Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Luuk van der Duim]

Adrian wrote:
--
The Linux kernel sometimes looses developers.

That seems to unavoidable, there are there are always problems like:
- Some developers leave the projects if their code was rejected because 
  it didn't match the standards and policies of the project.
- Some developers leave the project if other people's code that didn't 
  match the standards and policies of the project was accepted.

--


*Cough*DonaldBecker AndreHedrick*Cough*
..


   Luuk

-------------------------------------------------------------------------------------------------------------

Disclaimer:
By sending an email to ANY of my addresses you are agreeing that:

   1. I am by definition, "the intended recipient"
   2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it on usenet.
   3. I may take the contents as representing the views of your company.
   4. This overrides any disclaimer or statement of confidentiality that may be included on your message. 



                                        

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[greg]

On Jul 31,  3:41pm, Theodore Tso wrote:
} Subject: Re: the " 'official' point of view" expressed by kernelnewbies.or

> On Mon, Jul 31, 2006 at 06:54:06PM +0200, Matthias Andree wrote:
> > > > This looks rather like an education issue rather than a technical limit.
> > > 
> > > We aren't talking about the same issue: I was asking to do it
> > > on-the-fly. Umounting the filesystem, running e2fsck and resize2fs
> > > is something different ;-)
> > 
> > There was stuff by Andreas Dilger, to support "online" resizing of
> > mounted ext2 file systems. I never cared to look for this (does it
> > support ext3, does it work with current kernels, merge status) since
> > offline resizing was always sufficient for me.

> With the latest e2fsprogs and 2.6 kernels, the online resizing
> support has been merged in, and as long as the filesystem was
> created with space reserved for growing the filesystem (which is now
> the default, or if the filesystem has the off-line prepration step
> ext2prepare run on it), you can run resize2fs on a mounted
> filesystem and grow an ext2/3 filesystem on-line.  And yes, you get
> more inodes as you add more disk blocks, using the original inode
> ratio that was established when the filesystem was created.

Are all the necessary tools in and documented in e2fsprogs?

It seems that finding all the bits and pieces to do ext3 on-line
expansion has been a study in obfuscation.  Somewhat surprising since
this feature is a must for enterprise class storage management.

> 						- Ted

Best wishes for a productive week.

}-- End of excerpt from Theodore Tso

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Ooohh.. FreeBSD is faster over loopback, when compared to Linux over
the wire.  Film at 11."
                                -- Linus Torvalds

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan Engelhardt]

>> With the latest e2fsprogs and 2.6 kernels, the online resizing
>> support has been merged in, and as long as the filesystem was
>> created with space reserved for growing the filesystem (which is now
>> the default, or if the filesystem has the off-line prepration step
>> ext2prepare run on it), you can run resize2fs on a mounted
>> filesystem and grow an ext2/3 filesystem on-line.  And yes, you get
>> more inodes as you add more disk blocks, using the original inode
>> ratio that was established when the filesystem was created.
>
>Are all the necessary tools in and documented in e2fsprogs?
>
>It seems that finding all the bits and pieces to do ext3 on-line
>expansion has been a study in obfuscation.  Somewhat surprising since
>this feature is a must for enterprise class storage management.

Enterprise will hardly use ext3 on the big ones, but one of the "more
commercial" things.



Jan Engelhardt
-- 

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

greg@enjellic.com wrote:

> It seems that finding all the bits and pieces to do ext3 on-line
> expansion has been a study in obfuscation.  Somewhat surprising since
> this feature is a must for enterprise class storage management.

Not really.  Having people who can dig through the obfuscation is also a 
must for enterprise class anything.

The desktop is where it's really crucial to have good documentation and 
ease of use.  The enterprise can afford to pay people who already knew 
it well, helped to develop it...  Grandma probably got Linux because she 
couldn't afford a new OS, or computer.

Of course, I won't go so far as to try to say "Linux should focus on 
this."  Linux should focus on whatever Linux developers feel like 
focusing on.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Bodo Eggert]

Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> On Mon, 2006-07-31 12:17:12 -0700, Clay Barnes <clay.barnes@gmail.com> wrote:
>> On 20:43 Mon 31 Jul     , Jan-Benedict Glaw wrote:
>> > On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de>
>> > > Jan-Benedict Glaw schrieb am 2006-07-31:

> [Crippled DMA writes]
>> > > Massive hardware problems don't count. ext2/ext3 doesn't look much better
>> > > in such cases. I had a machine with RAM gone bad (no ECC - I wonder what
>> > 
>> > They do! Very much, actually. These happen In Real Life, so I have to
>> 
>> I think what he meant was that it is unfair to blame reiser3 for data
>> loss in a massive failure situation as a case example by itself.  What

<snip>

> The point is that it's quite hard to really fuck up ext{2,3} with only
> some KB being written while it seems (due to the
> fragile^Wsophisticated on-disk data structures) that it's just easy to
> kill a reiser3 filesystem.

- Once I had dying hdd without realizing this (I asumed heat problems or a
  failing power supply), and that caused the fs to become unaccessible.
  --rebuild-tree did the trick.

- I had a LVM on a set of some crappy disks with a reiser3fs-formated LV.  
  Windows decided it had to format the LVM partitions, and the reiserfs
  survived almost undamaged.

- I sometimes had errors on reiserfs resulting in inaccessible
  directories. I could fix that by moving them out of the way. (Maybe I
  could also have used --clean-attributes, I don't remember trying. OTOH,
  maybe that option is too new (2003).)

- I have an ext3 that can't be fixed by e2fsck (see below). fsck will fix
  some errors, trash some files and leave a fs waiting to throw the same
  error again. I'm fixing it using mkreiserfs now.

---------------------------------


Aug  2 15:15:23 server kernel: EXT3-fs error (device md(9,3)): ext3_free_blocks:
bit already cleared for block 13084101
Aug  2 15:15:23 server kernel: Aborting journal on device md(9,3).
Aug  2 15:15:23 server kernel: Remounting filesystem read-only
Aug  2 15:15:23 server kernel: ext3_reserve_inode_write: aborting transaction:
Journal has aborted in __ext3_journal_get_write_access<2>EXT3-fs error (device
md(9,3)) in ext3_reserve_inode_write: Journal has aborted
Aug  2 15:15:23 server kernel: EXT3-fs error (device md(9,3)) in ext3_truncate:
Journal has aborted
Aug  2 15:15:23 server kernel: ext3_reserve_inode_write: aborting transaction:
Journal has aborted in __ext3_journal_get_write_access<2>EXT3-fs error (device
md(9,3)) in ext3_reserve_inode_write: Journal has aborted
Aug  2 15:15:23 server kernel: EXT3-fs error (device md(9,3)) in
ext3_orphan_del: Journal has aborted
Aug  2 15:15:23 server kernel: ext3_reserve_inode_write: aborting transaction:
Journal has aborted in __ext3_journal_get_write_access<2>EXT3-fs error (device
md(9,3)) in ext3_reserve_inode_write: Journal has aborted
Aug  2 15:15:23 server kernel: EXT3-fs error (device md(9,3)) in
ext3_delete_inode: Journal has aborted



-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Luuk van der Duim]

I think the whole thing resembles the ALSA merge case(?)
Also a big blob which was offered at once developped seperately in alsa-devel.

It took Jaroslav and others a few years to convince Linus, him disliking music, he had no appetite for ALSA. 
Did Alan end up feeding it to Linus by putting cream on top?
Truth is he was busy being annoyed over scsi layer and other things. He ended up having no trouble at all
with the 79,000 line patch.. It was merged in 2.5.4 

I tried reiserV4 in the earlier days of V4  (2000?), I only used it on a non-critical partition but it worked fine for me although tools were still in development then
and _to me_ performance benefits were irreproduceable.

Even rmap had to go out of 2.4.10 because Linus thought the carpet would never fit the floor, no matter how many patches were to come.
I can imagine he's not to keen experimenting on things better than sliced bread (without cream).

Development trees also weren't an option because people kept being annoyed over too many things being broke at the same time.


  Luuk

---------------------------------

Disclaimer:
By sending an email to ANY of my addresses you are agreeing that:

   1. I am by definition, "the intended recipient"
   2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it on usenet.
   3. I may take the contents as representing the views of your company.
   4. This overrides any disclaimer or statement of confidentiality that may be included on your message. 



                                        

RE: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[André Goddard Rosa]

Jeff Garzik wrote:
> Hans Reiser wrote:
> > Theodore Tso wrote:
> >
> >> Actually, the first bits
> >>
> > yes, the first bits....   other people send in completed filesystems....
>
> Completed filesystems have a much higher barrier to entry, because they
> require a fresh review.
>
> ext4 will go upstream MUCH faster, because it follows the standard
> process of Linux evolution, building on top of existing code with
> progressive changes:

Hi, Hans!

    I think you are mostly right in your conclusions, it is sad but
true that we lose very good developers sometimes. I hope this
can be changed by emails like yours in the long run.

   I would like to see you and Christoph working again with synergy
(with harmony), but you have insulted him in the past. I regret this
happened and I think reiser4 would be already with us all if you had
focused at the technical discussion with him. I know that sometimes
it is difficult but we must all learn with Greg Kroah-Hartman, one of
the major contributors in volume to the kernel: be highly diplomatic
and respectfull. See more here:

    http://os.newsforge.com/os/06/07/23/1212252.shtml?tid=2&tid=138

    The arguments Jeff presented are super strong and are backed by an
entire past thread where Linus and others share a sharp clear vision
which I think that works very well in practice:

http://www.kernel-traffic.org/kernel-traffic/kt19991101_41.html#6

    They are _right_ (tm) here. We all know they are from our experiences
in past mistakes.

   I would like to thank you for reiser4, it is great! Please use your
diplomacy to get more reviews. After getting the code reviewed, fix the
complaints or discuss technically your POV.
   We all want reiser4 in (I do think it is great), but after reviewed by the
filesystem experts.

Thank you and the reiser4 team!

-- 
[]s,
André Goddard

the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Is http://wiki.kernelnewbies.org/WhyReiser4IsNotIn truly the "official"
point of view as claimed by its author?  An interesting method of
expression for it.  I heard about it from a user who suggested that I
respond before it got slashdotted.

Let me ask that one compare and contrast the ext4 integration procedure
outlined by Ted Tso, with the procedure experienced by other
filesystems.  The code isn't even written, benchmarked, or tested yet,
and it is going into the kernel already so that its developers don't
have to deal with maintaining patches separate from the tree.  Wow. 
Kind of hard to argue that it is not politically differentiated, isn't it?

Consider what happened with XFS as the article writer mentions.  I met
the original XFS team, led by two very senior developers (Jim Grey, and
another fellow whose name I am blanking on, forgive me, I learned much
from him in just a few conversations).  They were kind enough to
instruct me on what ideas I should take from XFS, and you know what, I
listened.  Reiser4's allocate on flush is the result of their kind
instruction.  I then took it a bit further, like a good student, and
Reiser4 also has balance on flush, compress on flush, etc.

These guys wanted to port XFS to Linux, but there was a problem, which
was that IRIX was better in some ways than Linux, and XFS depended on
those advantages.  Now I met them, and I have to tell you that it was
pretty obvious that these guys knew what they were doing.  Suggest that
these guys needed supervision --- sorry, no way, we needed their
supervision.  What happened?  They got hassled.  Instead of learning
from them, welcoming into our community two very senior developers who
knew a lot more than any of us about the topics they chose to speak
about, they got hassled, they get ignored, they felt rejected, and left
the Linux community forever, never to return.  XFS is still with us, but
the loss of those two could only have been devastating for their
project.    I think the whole kernel community suffered from their loss.

Linux has a problem, which is that with success it is attracting people
with more skill than what it started with, and it is not doing a very
good job of handling that.  In fact, it downright stinks at it, behaving
in the worst way it could choose for handling that.  We have lost quite
a number of FS developers who just don't want to deal with people who
know less than they do but are obnoxious and disrespectful to
submissions because they enjoy powertripping.  We lost David Mazieres,
for example, because he is very very bright, is one of DARPA's most
promising security researchers, and he does not want to be bothered with
Viro et al. so he develops for BSD instead.  Linus, if you really want
to prove that Linux welcomes talented people, go sweet talk Mazieres
into giving Linux another try, you might succeed if you try.  The odd
thing is that Viro is not obnoxious at all in person.  lkml suffers from
email disease, and we need to make conscious efforts to reduce that.

Regarding distros accepting filesystems first, that is just completely
backwards from what it ought to be.  Linus, I respect you a lot, but I
know this one is your idea, and some things I disagree with you on. 
Distros are marketed towards people who do not know how to tar and untar
if an FS is dropped.  A reasonable approach would be to say that any
filesystem marked as experimental can be dropped at any time, so if you
aren't able to tar and untar the partition it is on when a new kernel
comes out, you should not use experimental filesystems.  Then most
distros will not make the experimental FS visible to users who don't
press three buttons acknowledging that they were warned....  Linspire's
view is pretty simple, they need to know that Reiser4 will be accepted
BEFORE they make their distro depend on it.  This is being responsible
to their users.  I could go ask Debian, etc., to include Reiser4, but it
is the wrong way, so I am shy about it.

I am not saying that ext4 should not be accepted as an experimental FS,
I don't even really believe that ext4 should only be accepted when it is
higher performance than Reiser4, I am saying that the process should be
the same for everyone.  Reiser4 is the upgrade for ReiserFS V3, in which
we fix all of V3's flaws without disturbing the mission critical servers
using V3 by changing the V3 code underneath them.  (Things like the bug
affecting MythTV users on V3 at the moment just should not be
happening.  Experiments belong in V4, and I wish there was more respect
for my views on this.).  V4 contains bug fixes for several V3 bugs that
are too deep to fix without deep rewrite, and since V3 does not have
plugins, disk format changes should not get added to a stable branch. 
When submitted Reiser4 was more stable than V3 was when it was
accepted.  (This is because we now have a much better test suite.   I
would never submit code that I know has a bug unfixed.  At the moment we
can crash Reiser4 using our test suite, as some of the linux kernel
inclusion related changes made recently were extensive, I hope we have
that bug fixed by next week.)

We should develop a culture in which acceptance is more based on whose
code measurably performs well than on who is friends with whom.  We
should not think that such a culture will develop without an effort
being made to grow it.

Actually, if we just had a few more akpms to go around, things would be
a lot better..... oh well.  We need to recruit more people like him. 
You can't really have code reviewed by persons less experienced and
proven than those being reviewed, it just doesn't work too well.  Linux
needs to look outward more, and welcome persons who have proven
themselves in other arenas as though we were lucky to get their time. 
Because we are.  Maybe when we don't have people with the expertise to
review something, we should go outside the Linux community, like the way
academic journals will solicit outside reviewers for particular articles
as they need them.  Our current attitude resembles that of BSD before it
lost the market to Linux, I remember it well, there was a reason why I
developed for Linux instead.

Avoiding the problems that some large corporations have with politics
does not happen automatically as a result of it being free software, it
requires as much effort as it does in the successful large
corporations.  Non-profits are in no way immune to being harmed by
internal politics.

If it is true that Reiser4 is likely to go in for 2.6.19, this is good
to hear, though an odd source to hear it from.  If it is true, then I
will skip lobbying distros to accept Reiser4 before the kernel does,
because really it makes little sense for them to do so.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote:

> Is http://wiki.kernelnewbies.org/WhyReiser4IsNotIn truly the "official"
> point of view as claimed by its author?  An interesting method of
> expression for it.  I heard about it from a user who suggested that I
> respond before it got slashdotted.

After two users asked the "Why is Reiser4 still not included?" question 
within a very short amount of time on this list (mailing lists aren't 
write-only, and this issue has been discussed often before...), Diego 
wrote an FAQ for this issue.

Emails by people like Linus, Andrew, Christoph or Al are what comes 
nearest to an official statement.

If there are factual errors or things you consider offensive in the FAQ, 
please try to discuss them with Diego privately first. But changing 
contents of this FAQ wouldn't change anything regarding the Reiser4 
inclusion - the FAQ is only a high level description of the Reiser4 
situation for end users.

> Let me ask that one compare and contrast the ext4 integration procedure
> outlined by Ted Tso, with the procedure experienced by other
> filesystems.  The code isn't even written, benchmarked, or tested yet,
> and it is going into the kernel already so that its developers don't
> have to deal with maintaining patches separate from the tree.  Wow. 
> Kind of hard to argue that it is not politically differentiated, isn't it?

The main difference seems to be that ext4 is being developed by people 
that have already shown and are trusted to develop a filesystem that 
follows the Linux way in all respects.

Note that I didn't say "right way" but "Linux way".

No matter whether it's coding style or the discussion which 
functionality belongs to the VFS level, the Linux kernel has it's (often 
unwritten) rules - but the same is true with other rules for any other 
operating system.

>...
> Actually, if we just had a few more akpms to go around, things would be
> a lot better..... oh well.  We need to recruit more people like him. 
> You can't really have code reviewed by persons less experienced and
> proven than those being reviewed, it just doesn't work too well.  Linux
> needs to look outward more, and welcome persons who have proven
> themselves in other arenas as though we were lucky to get their time. 
> Because we are.  Maybe when we don't have people with the expertise to
> review something, we should go outside the Linux community, like the way
> academic journals will solicit outside reviewers for particular articles
> as they need them.  Our current attitude resembles that of BSD before it
> lost the market to Linux, I remember it well, there was a reason why I
> developed for Linux instead.

A very important part of a review is whether it follows the 
"Linux way" that might be quite different from what someone who comes 
from outwards has to learn before he can start doing a review.

Finding people for the cool stuff like developing a new filesystem is 
relatively easy compared to finding people why are both capable and 
willing to do the boring work of reviewing other people's code.

So we'd need people who are already acknowleged experts in the area, who 
are willing to learn the Linux way, and who will then do the not-fun 
work of reviewing other people's code.

How should this work?
Someone has to offer well-paid jobs for such people?

> Avoiding the problems that some large corporations have with politics
> does not happen automatically as a result of it being free software, it
> requires as much effort as it does in the successful large
> corporations.  Non-profits are in no way immune to being harmed by
> internal politics.

In my experience, the Linux kernel is a big open source project with a 
working structure.

The Linux kernel sometimes looses developers.

That seems to unavoidable, there are there are always problems like:
- Some developers leave the projects if their code was rejected because 
  it didn't match the standards and policies of the project.
- Some developers leave the project if other people's code that didn't 
  match the standards and policies of the project was accepted.

And it's not as if open source projects had in any respect better 
prerequisites than large corporations - open source lacks the "it's our 
job" glue forcing people to work together in large corporations.

>...
> Hans

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Diego Calleja]

El Sat, 22 Jul 2006 02:18:19 +0200,
Adrian Bunk <bunk@stusta.de> escribió:

> After two users asked the "Why is Reiser4 still not included?" question 
> within a very short amount of time on this list (mailing lists aren't 
> write-only, and this issue has been discussed often before...), Diego 
> wrote an FAQ for this issue.
> 
> Emails by people like Linus, Andrew, Christoph or Al are what comes 
> nearest to an official statement.

[I'm the guy who wrote the doc]

I didn't write "It could possibly be ready as soon as 2.6.19" literally,
that was someone that reworked my (ugly) english. Being fair what make
me wrote that pages was the huge amount of people FUDing about linux
developers in online forums (including this list)

> > You can't really have code reviewed by persons less experienced and
> > proven than those being reviewed, it just doesn't work too well.  Linux

Hans, stop that "we're smarter than you" attitude. It's not surprising
that reiser 4 creates so many flames and that it's getting so hard to make
progress, with such strong arguments. As far as I can tell, most of the
kernel hackers trust Al Viro and hch on their opinions, specially in
fs-related issues - and for good reasons. They know linux and they 
know how a linux fs must behave. As long as a fs plays well with the
rest of the system and the code doesn't suck, I doubt they care too
much if it's very advanced or arcane, and the huge variety of
filesystem available in linux confirms that.

> > as they need them.  Our current attitude resembles that of BSD before it
> > lost the market to Linux, I remember it well, there was a reason why I
> > developed for Linux instead.

Hans, I don't agree. If anything, the problem is that right now there's
not a "development" stage: people just takes more care about what goes
in, it wouldn't happen the same under a development stage. That certainly
could make the job of big projects like reiser 4 much harder.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Theodore Tso]

On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote:
> Let me ask that one compare and contrast the ext4 integration procedure
> outlined by Ted Tso

"integration procedure" is hardly an accurate description, rather it
is a development procedure that was developed after discussion and
consensus building across LKML and the ext2/3/4 development team.  It
was not the original plan put forth by the ext2 developers, but after
listening to the concerns and suggestions, we did not question the
motives of the people making suggestions; we listened.

> The code isn't even written, benchmarked, or tested yet,

Actually, the first bits that we plan to merge have already been
written and in use by hundreds of clusterfs customers, posted to LKML
for comments (and we don't attack our reviewers, we thank them for
their comments), and in fact they were written about at last year's
OLS complete with benchmarks and graphs.
(http://ext2.sourceforge.net/2005-ols/2005-ols-ext3.html)

> Consider what happened with XFS as the article writer mentions.  I met
> the original XFS team, led by two very senior developers (Jim Grey, and
> another fellow whose name I am blanking on, forgive me, I learned much
> from him in just a few conversations).  

I believe you are referring to Jim Mostek and Steve Lord, and yes,
they were very talented developers and engineers.  I very much enjoyed
talking to them at various filesystem and Linux conferences and
workshops.

> supervision.  What happened?  They got hassled.  Instead of learning
> from them, welcoming into our community two very senior developers who
> knew a lot more than any of us about the topics they chose to speak
> about, they got hassled, they get ignored, they felt rejected, and left
> the Linux community forever, never to return.  

That's hardly what happened.  SGI went through layoffs, and they were
hit.  See:  http://slashdot.org/articles/01/05/26/0743254.shtml

> A reasonable approach would be to say that any
> filesystem marked as experimental can be dropped at any time, so if you
> aren't able to tar and untar the partition it is on when a new kernel
> comes out, you should not use experimental filesystems.  Then most
> distros will not make the experimental FS visible to users who don't
> press three buttons acknowledging that they were warned....  Linspire's
> view is pretty simple, they need to know that Reiser4 will be accepted
> BEFORE they make their distro depend on it.  

You do realize these two statements are completely contradictory,
don't you?  If an experimental filesystem can be dropped at any time,
then Linspire can't depend on it from the point of view of supporting
their users.  If there is a huge user base, then someone is going to
have to maintain it, even if the developer community has moved on to
supporting the next new exciting filesystem thing.  Hence, it is
critical that the resulting filesystem be fully maintainable before it
is integrated.  To put it in your words, it wouldn't be responsible to
the user base to do otherwise.

							- Ted

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Eric Sandeen]

Theodore Tso wrote:
> On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote:
>> Consider what happened with XFS as the article writer mentions.  I met
>> the original XFS team, led by two very senior developers (Jim Grey, and
>> another fellow whose name I am blanking on, forgive me, I learned much
>> from him in just a few conversations).  
> 
> I believe you are referring to Jim Mostek and Steve Lord, and yes,
> they were very talented developers and engineers.  I very much enjoyed
> talking to them at various filesystem and Linux conferences and
> workshops.
> 
>> supervision.  What happened?  They got hassled.  Instead of learning
>> from them, welcoming into our community two very senior developers who
>> knew a lot more than any of us about the topics they chose to speak
>> about, they got hassled, they get ignored, they felt rejected, and left
>> the Linux community forever, never to return.  
> 
> That's hardly what happened.  SGI went through layoffs, and they were
> hit.  See:  http://slashdot.org/articles/01/05/26/0743254.shtml

Jim & Steve were never laid off from SGI.  SGI mgmt -was- smarter than that ;)
Neither account above is 100% correct, but I won't speak further on behalf of 
these gentlemen...

-Eric

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Theodore Tso wrote:

>
>Actually, the first bits
>
yes, the first bits....   other people send in completed filesystems....

> that we plan to merge
>
I don't actually think that your merge approach is the wrong one, I
think that it being exclusive to you is what is wrong.

>
>  
>
>>Consider what happened with XFS as the article writer mentions.  I met
>>the original XFS team, led by two very senior developers (Jim Grey, and
>>another fellow whose name I am blanking on, forgive me, I learned much
>>from him in just a few conversations).  
>>    
>>
>
>I believe you are referring to Jim Mostek
>
Ah, Jim Mostek and Jim Gray.  (Steve Lord was not a senior guy back
then, and he is still with SGI last I heard....  I actually don't know
Steve very well, hmm, maybe some future conference....)  Thanks.

>That's hardly what happened.  SGI went through layoffs, and they were
>hit.  See:  http://slashdot.org/articles/01/05/26/0743254.shtml
>  
>
As the other poster mentioned, they went off to startups, and did not
become part of our community.  How much of that was because their
contributions were more hassled than welcomed, I cannot say with
certainty, I can only say that they were discouraged by the difficulty
of getting their stuff in, and this was not as it should have been. 
They were more knowledgeable than we were on the topics they spoke on,
and this was not recognized and acknowledged.

Outsiders are not respected by the kernel community.  This means we miss
a lot.

>  
>
>>A reasonable approach would be to say that any
>>filesystem marked as experimental can be dropped at any time, so if you
>>aren't able to tar and untar the partition it is on when a new kernel
>>comes out, you should not use experimental filesystems.  Then most
>>distros will not make the experimental FS visible to users who don't
>>press three buttons acknowledging that they were warned....  Linspire's
>>view is pretty simple, they need to know that Reiser4 will be accepted
>>BEFORE they make their distro depend on it.  
>>    
>>
>
>You do realize these two statements are completely contradictory,
>don't you? 
>
No, because distros would wait until it is not experimental before
giving it to their users by default, in my proposed release model.  lkml
is populated with people FAR more suited to experimenting with
experimental filesystems than typical distro customer lists are.  It is
commercial and political reasons that motivate distros being the first
with patches not tried yet by lkml, not the interests of the users.

Now, for other patches these commercial and political reasons may need
to be catered to as the price of getting the Redhats of the world to
fund kernel development, but that logic does not apply to Reiser4's
particulars.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff Garzik]

Hans Reiser wrote:
> Theodore Tso wrote:
> 
>> Actually, the first bits
>>
> yes, the first bits....   other people send in completed filesystems....

Completed filesystems have a much higher barrier to entry, because they 
require a fresh review.

ext4 will go upstream MUCH faster, because it follows the standard 
process of Linux evolution, building on top of existing code with 
progressive changes:

	cp -a ext3 ext4
	update ext4
	update ext4
	update ext4
	...

This process builds upon existing reviews and knowledge of existing 
code.  This process also guarantees a higher degree of stability during 
development, because the interim changes must always form a complete, 
working, usable filesystem.


> As the other poster mentioned, they went off to startups, and did not
> become part of our community.  How much of that was because their
> contributions were more hassled than welcomed, I cannot say with
> certainty, I can only say that they were discouraged by the difficulty
> of getting their stuff in, and this was not as it should have been. 
> They were more knowledgeable than we were on the topics they spoke on,
> and this was not recognized and acknowledged.
> 
> Outsiders are not respected by the kernel community.  This means we miss
> a lot.

Anyone who fails to respect the kernel development process, the process 
of building consensus, is turn not respected, flamed, and/or ignored.

If you don't respect us, why should we respect you?


> No, because distros would wait until it is not experimental before
> giving it to their users by default, in my proposed release model.  lkml

Distros follow their own release model, and don't have a care about what 
Hans Reiser thinks they should do.

<vendor hat on>
Red Hat has a pipeline in place for offering new technologies to users: 
  Fedora Core -> RHEL, and sometimes RHEL technology previews.  SuSE 
presumably does something similar with OpenSUSE.
</vendor hat>

There is PLENTY of opportunity to be experimental.


> is populated with people FAR more suited to experimenting with
> experimental filesystems than typical distro customer lists are.  It is
> commercial and political reasons that motivate distros being the first
> with patches not tried yet by lkml, not the interests of the users.

> Now, for other patches these commercial and political reasons may need
> to be catered to as the price of getting the Redhats of the world to
> fund kernel development, but that logic does not apply to Reiser4's
> particulars.

I always feel sad to hear technologists wail about politics.

In my experience, the cause of such is almost always the fault of the 
submittor, ignoring consensus.  But once the submittor has decided that 
"politics" are cause of their troubles, the submittor focuses on that 
rather than addressing the technology objections that were raised.

With you in particular, you demonstrated NO interest in maintaining 
reiser3, once reiser4 began to make a splash.  Linux kernel code exists 
for DECADES, and as such, long term maintenance is a CRITICAL aspect of 
development.

Regardless of whatever new whiz-bang technology exists in reiser4, there 
is a very real worry that you will abandon reiser4 once its in the tree 
for a few years, just like what happened with reiser3.

	Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Jeff, I think that a large part of what is going on is that any patch
that can be read in 15 minutes gets reviewed immediately, and any patch
that is worked on for 5 years and then takes a week to read gets
neglected.  This is true even if line for line the 1 week to read patch
is more valuable.    What is more is that people know this is
irrational, but aren't able to cure it in themselves.  Even I have a
problem of paying too much attention to endless 5 minute emails when I
know I should instead, say, read the compression plugin from beginning
to end.

There is nothing about small patches that makes them better code.  There
is no reason we should favor them, if the developers are willing to work
on something for 5 years to escape a local optimum, that is often the
RIGHT thing to do.

It is importand that we embrace our diversity, and be happy for the
strength it gives us.  Some of us are good at small patches that evolve,
and some are good at escaping local optimums.  We all have value, both
trees and grass have their place in the world.


>
>
> With you in particular, you demonstrated NO interest in maintaining
> reiser3, once reiser4 began to make a splash.  Linux kernel code
> exists for DECADES, and as such, long term maintenance is a CRITICAL
> aspect of development.

You are rejecting the development model which is based on stable
branches getting only bugfixes.  V3 is a stable branch.  It just had a
feature added to it which added a bug that MythTV users are hitting. 
Some of them are responding to it by walking away from Reiser3, and no
doubt muttering about what an unstable pile of shit our code is.  On
monday one of my guys is stopping work on V4 to send in a bug fix for a
feature that should have gone into V4 first, and then maybe gotten
backported after it was proven in V4.

So, given that Jeff and Chris can often be gotten to fix bugs, do I ask
them to do it whenever there is a bug to fix and they will fix it?  Oh
yes!  The despiriting thing though is that there is usually another
reason to let them fix it, which is that almost all v3 bugs are in
features they have added to what ought to have been a stable branch, and
since it is their code, they should be the ones to fix it.  We might,
maybe, get one bug report a year in code written by Namesys before  I
announced code freeze on V3.

I just got an email from the programmer who wrote the MythTV bug saying
that he is just too busy to bother fixing the bug in his code.....  so
my response is that a Namesys programmer is going to fix it on Monday.

All this talk about how you guys worry that code is going to be
abandoned, you know, try policing the kids in their 20's who do it, not
those who have been working since 1984 on developing the thing you
somehow are worried they will abandon.  I am not 20 something anymore, I
am getting fat no matter how much I exercise, and I stick with things,
and I only wish some things didn't stick so much with my middle....

>
> Regardless of whatever new whiz-bang technology exists in reiser4,
> there is a very real worry that you will abandon reiser4 once its in
> the tree for a few years, just like what happened with reiser3.

And look at how Linus abandoned 2.4!  Users of 2.4 needed so many
features that were put into 2.6 instead, and they were just abandoned
and neglected and....  Do you think he will abandon 2.6.18 also?

The stable branch of code getting only bugfixes and the development
branch getting all the new features model of development is something
most release management professionals agree is the right way to do
things.  I worked with release management teams some, and I have to say
that the dominant paradigm in the software industry is, in this case,
the best one yet.

Of course, I want to make it a little better, you know how I am, and as
I was just discussing on the reiserfs-list, with plugins we can now move
to a model in which if you mount reiser4 using the -o reiser4.1-beta
mount option, it changes what the default plugin is, and that is how we
do releases, we put our beta code in different plugins, and let the user
choose whether to upgrade to a new release by just choosing what plugins
to use as his default.  Now that we paid the 5 year development price
tag to get everything as plugins, we can now upgrade in littler pieces
than any other FS.  Hmm, I need a buzz phrase, its not extreme
programming, maybe "moderate programming".  Does that sound exciting to
others.;-)  Seriously though, I am curious to see whether plugin based
release management works out as pleasantly for users as I am hoping it will.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matt Heler]

On Sunday 23 July 2006 12:20 am, Hans Reiser wrote:
> I just got an email from the programmer who wrote the MythTV bug saying
> that he is just too busy to bother fixing the bug in his code.....  so
> my response is that a Namesys programmer is going to fix it on Monday.

The way you wrote this, makes it sound like a userspace issue, and _not_ a 
problem with reiserfs.

> And look at how Linus abandoned 2.4!  Users of 2.4 needed so many
> features that were put into 2.6 instead, and they were just abandoned
> and neglected and....  Do you think he will abandon 2.6.18 also?

Not entirely true, he did not abandon the 2.4 kernel branch, he passed on 
maintainership to Marcelo. Similar to how he passed the torch on the 2.2 
kernel branch to Alan Cox. Also on a side note, many new features ( and a ton 
of bug fixes !! ) were added to the 2.4 series _after_ Linus started working 
on the 2.5 branch.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Matt Heler wrote:

>On Sunday 23 July 2006 12:20 am, Hans Reiser wrote:
>  
>
>
>The way you wrote this, makes it sound like a userspace issue, and _not_ a 
>problem with reiserfs.
>  
>
It was a problem with reiserfs.  Code was added to search for the
perfect spot to fit a file.  If there is no perfect spot, it searches
every bitmap for that spot before giving up.  However, Jeff kindly gave
us a little patch to fix this and made the whole issue moot.  It also
seems I was in error, and we actually have had this problem since 2002. 
Now some past remarks from users about fragmentation make more sense. 
What can I say, since I have no MP3s I never get anywhere near full on
my personal hard drive.

>  
>
>>And look at how Linus abandoned 2.4!  Users of 2.4 needed so many
>>features that were put into 2.6 instead, and they were just abandoned
>>and neglected and....  Do you think he will abandon 2.6.18 also?
>>    
>>
>
>Not entirely true, he did not abandon the 2.4 kernel branch, he passed on 
>maintainership to Marcelo. Similar to how he passed the torch on the 2.2 
>kernel branch to Alan Cox. Also on a side note, many new features ( and a ton 
>of bug fixes !! ) were added to the 2.4 series _after_ Linus started working 
>on the 2.5 branch.
>  
>
You missed the sarcasm in my voice, my apologies, it is the trouble I
have with email.

Just to balance everything with some nuance, let me add that when a
development branch is first opened, there is usually a bit of gray as to
whether particular small features should go into the development branch
or the stable branch.  As the stable branch gets more stable the
incentive to not destabilize it increases, and as a development branch
becomes usable, the delay to users due to putting features only there
reduces.

I want reiserfs to be the filesystem that professional system
administrators view as the one with both the fastest technological pace,
and the most conservative release management.

I apologize to users  that the technology required a 5 year gap between
releases.   It just did, an outsider may not realize how deep the
changes we made were.  Things like per node locking based on a whole new
approach to tree locking that goes bottom up instead of the usual top
down are big tasks.    Dancing trees are a big change, getting rid of
blobs is a big change, wandering logs.....  We did a lot of things like
that, and got very fortunate with them.  If we had tried to add such
changes to V3, the code would have been unstable the whole 5 years, and
would not have come out right.

Experienced writers know that often, if you want to fix a passage, even
a passage that is quite good in some parts, sometimes it is better to
write the whole passage again without looking at the text of the first
draft of the old passage, because sometimes your muse just needs the
freedom, and without the freedom the awkwardness of the old passage is
incurable.  Probably there is some very sophisticated neurological
reason why that is.  Code can be the same.  Sometimes.  I knew that
reiser4 HAD to be written from scratch without reference to the old code
if it was to come out right. 

If I cannot be a great artist, at least I can try to have the
temperament of one, yes? :-)

I sincerely hope that using mount options to select default plugins, and
making development code go into new plugins means that releases after
this can be roughly quarterly, and that we can start doing a whole bunch
of quick little plugins.  Technically, I think it is going to be
downhill skiing from here, and some very visible bits of functionality
will get added much more easily than this difficult infrastructure we
just coded.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Sun, 23 Jul 2006, Hans Reiser wrote:

> I want reiserfs to be the filesystem that professional system
> administrators view as the one with both the fastest technological pace,
> and the most conservative release management.

Well, I, with the administrator hat on, phased out all reiserfs file
systems and replaced them by ext3. This got me rid of silent
corruptions, immature reiserfsprogs and hash collision chain limits.

> I apologize to users  that the technology required a 5 year gap between
> releases.   It just did, an outsider may not realize how deep the
> changes we made were.  Things like per node locking based on a whole new
> approach to tree locking that goes bottom up instead of the usual top
> down are big tasks.    Dancing trees are a big change, getting rid of
> blobs is a big change, wandering logs.....  We did a lot of things like
> that, and got very fortunate with them.  If we had tried to add such
> changes to V3, the code would have been unstable the whole 5 years, and
> would not have come out right.

And that is something that an administrator does not care the least
about. It must simply work, and the tools must simply work. Once I hit
issues like "xfs_check believes / were mounted R/W (not ignoring rootfs)
and refuses the R/O check", "reiserfsck can't fix a R/O file system"
(I believed this one got fixed before 3.6.19) or particularly silent
corruptions that show up later in a routine fsck --check after a kernel
update, the filesystem and its tools appear in a bad light. I've never
had such troubles with ext2fs or ext3fs or FreeBSD's or Solaris's ufs.

I'm not sure what patches Chris added to SUSE's reiserfs, nor do I care
any more. The father declared his child unsupported, and that's the end
of the story for me. There's nothing wrong about focusing on newer code,
but the old code needs to be cared for, too, to fix remaining issues
such as the "can only have N files with the same hash value". (I am well
aware this is exploiting worst-case behavior in a malicious sense but I
simply cannot risk such nonsense on a 270 GB RAID5 if users have shared
work directories.)

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Matthias Andree wrote:

> The father declared his child unsupported, 
>
I never did that.

>and that's the end
>of the story for me. There's nothing wrong about focusing on newer code,
>but the old code needs to be cared for, too, to fix remaining issues
>such as the "can only have N files with the same hash value". 
>
Requires a disk format change, in a filesystem without plugins, to fix it.

>(I am well
>aware this is exploiting worst-case behavior in a malicious sense but I
>simply cannot risk such nonsense on a 270 GB RAID5 if users have shared
>work directories.)
>
>  
>
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Mon, 24 Jul 2006, Hans Reiser wrote:

> >and that's the end
> >of the story for me. There's nothing wrong about focusing on newer code,
> >but the old code needs to be cared for, too, to fix remaining issues
> >such as the "can only have N files with the same hash value". 
>
> Requires a disk format change, in a filesystem without plugins, to fix it.

You see, I don't care a iota about "plugins" or other implementation details.

The bottom line is reiserfs 3.6 imposes practial limits that ext3fs
doesn't impose and that's reason enough for an administrator not to
install reiserfs 3.6. Sorry.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Christian Iversen]

On Monday 24 July 2006 12:25, Matthias Andree wrote:
> On Mon, 24 Jul 2006, Hans Reiser wrote:
> > >and that's the end
> > >of the story for me. There's nothing wrong about focusing on newer code,
> > >but the old code needs to be cared for, too, to fix remaining issues
> > >such as the "can only have N files with the same hash value".
> >
> > Requires a disk format change, in a filesystem without plugins, to fix
> > it.
>
> You see, I don't care a iota about "plugins" or other implementation
> details.
>
> The bottom line is reiserfs 3.6 imposes practial limits that ext3fs
> doesn't impose and that's reason enough for an administrator not to
> install reiserfs 3.6. Sorry.

And what do you do if you, say, run of of inodes on ext3? Do you think the 
users will care about that? Or what if the number of files in your mail queue 
or proxy cache* become large enough for your fs operations to slow to a 
crawl?


* Yes I know most programs work around this by using many subdirs, but that's 
really a bandaid solution.

-- 
Regards,
Christian Iversen

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Erik Mouw]

On Mon, Jul 24, 2006 at 01:34:11PM +0200, Christian Iversen wrote:
> On Monday 24 July 2006 12:25, Matthias Andree wrote:
> > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs
> > doesn't impose and that's reason enough for an administrator not to
> > install reiserfs 3.6. Sorry.
> 
> And what do you do if you, say, run of of inodes on ext3? Do you think the 
> users will care about that?

>From what I've seen from our customers, that never happens. Yes, there
are sometimes people with a million inodes in use, and we've seen four
million once, but that's never been a problem, even not with a huge
mail server with thousands of users having mailboxes in maildir format.

We usually limit our own filesystems to 12 million inodes and it's
never been a problem to store files from our customers.

> Or what if the number of files in your mail queue 
> or proxy cache* become large enough for your fs operations to slow to a 
> crawl?

Not a problem anymore with htree dirextory indexing. If it's not yet
enabled (dumpe2fs the filesystem and look for the "dir_index" feature),
enable it with:

  tune2fs -O dir_index /dev/whatever
    
After the next mount the filesystem will use it for new directories. To
optimize existing directories, run e2fsck -D.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Mike Benoit]

[-- Attachment #1: Type: text/plain, Size: 1708 bytes --]

On Mon, 2006-07-24 at 12:25 +0200, Matthias Andree wrote:
> On Mon, 24 Jul 2006, Hans Reiser wrote:
> 
> > >and that's the end
> > >of the story for me. There's nothing wrong about focusing on newer code,
> > >but the old code needs to be cared for, too, to fix remaining issues
> > >such as the "can only have N files with the same hash value". 
> >
> > Requires a disk format change, in a filesystem without plugins, to fix it.
> 
> You see, I don't care a iota about "plugins" or other implementation details.
> 
> The bottom line is reiserfs 3.6 imposes practial limits that ext3fs
> doesn't impose and that's reason enough for an administrator not to
> install reiserfs 3.6. Sorry.
> 

And EXT3 imposes practical limits that ReiserFS doesn't as well. The big
one being a fixed number of inodes that can't be adjusted on the fly,
which was reason enough for me to not use EXT3 and use ReiserFS
instead. 

Do you consider the EXT3 developers to have "abandoned" it because they
haven't fixed this issue? I don't, I just think of it as using the right
tool for the job.

I've been bitten by running out of inodes on several occasions, and by
switching to ReiserFS it saved one company I worked for over $250,000
because they didn't need to buy a totally new piece of software.

I haven't been able to use EXT3 on a backup server for the last ~5 years
due to inode limitations. Instead, ReiserFS has been filling that spot
like a champ. 

The bottom line is that every file system imposes some sort of limits
that bite someone. In your case it sounds like EXT3 limits weren't an
issue for you, in my case they were. Thats life. 

-- 
Mike Benoit <ipso@snappymail.ca>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Mike Benoit wrote:

> I've been bitten by running out of inodes on several occasions, and by
> switching to ReiserFS it saved one company I worked for over $250,000
> because they didn't need to buy a totally new piece of software.

ext3fs's inode density is configurable, reiserfs's hash overflow chain
length is not, and it doesn't show in df -i either.

If you need lots of inodes, mkfs for lots. That's old Unix lore.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 573 bytes --]

On Mon, 24 Jul 2006 19:35:24 +0200, Matthias Andree said:
> Mike Benoit wrote:
> 
> > I've been bitten by running out of inodes on several occasions, and by
> > switching to ReiserFS it saved one company I worked for over $250,000
> > because they didn't need to buy a totally new piece of software.
> 
> ext3fs's inode density is configurable, reiserfs's hash overflow chain
> length is not, and it doesn't show in df -i either.

Equally important - you can usually *see* "out of inodes" coming on a 'df -i'
long before a reiser3 filesystem hits the wall on a hash issue.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Mike Benoit <ipso@snappymail.ca> wrote:
> On Mon, 2006-07-24 at 12:25 +0200, Matthias Andree wrote:
> > On Mon, 24 Jul 2006, Hans Reiser wrote:
> > 
> > > >and that's the end
> > > >of the story for me. There's nothing wrong about focusing on newer code,
> > > >but the old code needs to be cared for, too, to fix remaining issues
> > > >such as the "can only have N files with the same hash value". 
> > >
> > > Requires a disk format change, in a filesystem without plugins, to fix it.
> > You see, I don't care a iota about "plugins" or other implementation details.
> > 
> > The bottom line is reiserfs 3.6 imposes practial limits that ext3fs
> > doesn't impose and that's reason enough for an administrator not to
> > install reiserfs 3.6. Sorry.

> And EXT3 imposes practical limits that ReiserFS doesn't as well. The big
> one being a fixed number of inodes that can't be adjusted on the fly,

Right. Plan ahead.

> which was reason enough for me to not use EXT3 and use ReiserFS
> instead. 

I don't see this following in any way.

> Do you consider the EXT3 developers to have "abandoned" it because they
> haven't fixed this issue? I don't, I just think of it as using the right
> tool for the job.

Dangerous parallel, that one...

> I've been bitten by running out of inodes on several occasions,

Me too. It was rather painful each time, but fixable (and in hindsight,
dumb user (setup) error).

>                                                                 and by
> switching to ReiserFS it saved one company I worked for over $250,000
> because they didn't need to buy a totally new piece of software.

How can a filesystem (which by basic requirements and design is almost
transparent to applications) make such a difference?!

> I haven't been able to use EXT3 on a backup server for the last ~5 years
> due to inode limitations.

See comment above. Read mke2fs(8) with care.

>                           Instead, ReiserFS has been filling that spot
> like a champ. 

Nice for you.

> The bottom line is that every file system imposes some sort of limits
> that bite someone.

Mostly that infinite disks are hard to come by ;-)

>                    In your case it sounds like EXT3 limits weren't an
> issue for you, in my case they were.

I'd suspect the limits you ran into weren't exactly in ext3.

>                                      Thats life. 
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Mike Benoit]

[-- Attachment #1: Type: text/plain, Size: 2406 bytes --]

On Mon, 2006-07-24 at 14:06 -0400, Horst H. von Brand wrote:
> > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big
> > one being a fixed number of inodes that can't be adjusted on the fly,
> 
> Right. Plan ahead.

That is great in theory. But back to reality, when your working for a
company that is growing by leaps and bounds that isn't always possible.
Why would I want to intentionally limit myself to a set number of inodes
when I can get a performance increase and not have to worry about it by
simply using ReiserFS? 

It all boils down to using the right tool for the job, ReiserFS was the
right tool for this job. 

> > I've been bitten by running out of inodes on several occasions,
> 
> Me too. It was rather painful each time, but fixable (and in hindsight,
> dumb user (setup) error).
> 
> >                                                                 and by
> > switching to ReiserFS it saved one company I worked for over $250,000
> > because they didn't need to buy a totally new piece of software.
> 
> How can a filesystem (which by basic requirements and design is almost
> transparent to applications) make such a difference?!

Very easily, the backup software the company had spent ~$75,000 on
before I started working there used tiny little files as its "database".
Once the inodes ran out the entire system pretty much came to a
screeching halt. We basically had two options, use ReiserFS, or find
another piece of software that didn't use tiny little files as its
database. If I recall correctly the database went from about 2million
files to over 40 million in the span of just a few months. No one could
have predicted that. I believe this database was on an 18GB SCSI drive,
so even using 1K blocks wouldn't have worked. According to the mke2fs
man page:

"-i bytes-per-inode
This value generally shouldn't be smaller than the blocksize  of
the filesystem,  since  then  too many inodes will be made."

The only other option at that time was to purchase Veritas backup and
its not cheap. We ended up switching to ReiserFS, it increased our
backup/restore time immensely and also bought us about 1year before the
system finally outgrew itself for good. By that time the company could
afford to drop $250,000 on high end backup software so we could grow
past 10TB.

-- 
Mike Benoit <ipso@snappymail.ca>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

On Mon, 2006-07-24 13:37:13 -0700, Mike Benoit <ipso@snappymail.ca> wrote:
> On Mon, 2006-07-24 at 14:06 -0400, Horst H. von Brand wrote:
> The only other option at that time was to purchase Veritas backup and
> its not cheap. We ended up switching to ReiserFS, it increased our
> backup/restore time immensely and also bought us about 1year before the
> system finally outgrew itself for good. By that time the company could
> afford to drop $250,000 on high end backup software so we could grow
> past 10TB.

Erm, how does your data look like and why does it require a $250,000
backup solution? Sounds like I'm in the wrong business...

But that's for sure not lkml related...

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
 Signature of:                       Don't believe in miracles: Rely on them!
 the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Mike Benoit <ipso@snappymail.ca> wrote:
> On Mon, 2006-07-24 at 14:06 -0400, Horst H. von Brand wrote:
> > > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big
> > > one being a fixed number of inodes that can't be adjusted on the fly,

> > Right. Plan ahead.

> That is great in theory. But back to reality, when your working for a
> company that is growing by leaps and bounds that isn't always possible.
> Why would I want to intentionally limit myself to a set number of inodes
> when I can get a performance increase and not have to worry about it by
> simply using ReiserFS? 

Place your filesystems on LVN, when they grow the number of inodes grows to
match. Or did you run out of inodes and not of diskspace? Only ever
happened to me on (static) /dev, or (wrong configured) newsspools...

> It all boils down to using the right tool for the job, ReiserFS was the
> right tool for this job. 

/One/ tool that did the job. Not "the" right one, perhaps not even the best
one.

> > > I've been bitten by running out of inodes on several occasions,

> > Me too. It was rather painful each time, but fixable (and in hindsight,
> > dumb user (setup) error).

> > >                                                                 and by
> > > switching to ReiserFS it saved one company I worked for over $250,000
> > > because they didn't need to buy a totally new piece of software.

> > How can a filesystem (which by basic requirements and design is almost
> > transparent to applications) make such a difference?!

> Very easily, the backup software the company had spent ~$75,000 on
> before I started working there used tiny little files as its "database".

If you know that, you configure your filesystem like a newsspool or some
such.

> Once the inodes ran out the entire system pretty much came to a
> screeching halt.

Get a clue by for, apply to the vendor (for the design, or at the very
least for not warning unsuspecting users)?

>                  We basically had two options, use ReiserFS, or find
> another piece of software that didn't use tiny little files as its
> database.

Or reconfigure the filesystem with more inodes (you were willing to rebuild
the filesystem in any case, so...)

>           If I recall correctly the database went from about 2million
> files to over 40 million in the span of just a few months. No one could
> have predicted that. I believe this database was on an 18GB SCSI drive,
> so even using 1K blocks wouldn't have worked. According to the mke2fs
> man page:

> "-i bytes-per-inode
> This value generally shouldn't be smaller than the blocksize  of
> the filesystem,  since  then  too many inodes will be made."

18GiB = 18 million KiB, you do have a point there. But 40 million files on
that, with some space to spare, just doesn't add up.

> The only other option at that time was to purchase Veritas backup

... or a larger disk...

>                                                                   and
> its not cheap. We ended up switching to ReiserFS, it increased our
> backup/restore time immensely and also bought us about 1year before the
> system finally outgrew itself for good. By that time the company could
> afford to drop $250,000 on high end backup software so we could grow
> past 10TB.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Denis Vlasenko]

On Monday 24 July 2006 23:51, Horst H. von Brand wrote:
> > Once the inodes ran out the entire system pretty much came to a
> > screeching halt.
> 
> Get a clue by for, apply to the vendor (for the design, or at the very
> least for not warning unsuspecting users)?
> 
> >                  We basically had two options, use ReiserFS, or find
> > another piece of software that didn't use tiny little files as its
> > database.
> 
> Or reconfigure the filesystem with more inodes (you were willing to rebuild
> the filesystem in any case, so...)

The fact that _many_ UNIX filesystem formats have inode number limit
(configurable only at mkfs time) doesn't make it a good design.

By the same virtue you may declare that it is okay to smoke
and drink lots of vodka. Why not? Sizable portion if population does that...

I, on the contrary, want software to impose as few limits on me
as possible.
--
vda

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Tue, 25 Jul 2006, Denis Vlasenko wrote:

> I, on the contrary, want software to impose as few limits on me
> as possible.

As long as it's choosing some limit, I'll pick the one with fewer
surprises.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

[-- Attachment #1: Type: text/plain, Size: 383 bytes --]

Matthias Andree wrote:
> On Tue, 25 Jul 2006, Denis Vlasenko wrote:
> 
>> I, on the contrary, want software to impose as few limits on me
>> as possible.
> 
> As long as it's choosing some limit, I'll pick the one with fewer
> surprises.

Running out of inodes would be pretty surprising for me.

But then, I guess it's a good thing I don't admin for a living anymore.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 890 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Tue, 25 Jul 2006, David Masover wrote:

> Matthias Andree wrote:
> > On Tue, 25 Jul 2006, Denis Vlasenko wrote:
> > 
> >> I, on the contrary, want software to impose as few limits on me
> >> as possible.
> > 
> > As long as it's choosing some limit, I'll pick the one with fewer
> > surprises.
> 
> Running out of inodes would be pretty surprising for me.

No offense: Then it was a surprise for you because you closed your eyes
and didn't look at df -i or didn't have monitors in place.

There is no way to ask how many files with particular hash values you
can still stuff into a reiserfs 3.X. There, you're running into a brick
wall that only your forehead will "see" when you touch it.

Of course, different sites have different needs and if you need
gazillions of inodes or file names, you may see trouble.

But the assertion that some backup was the cause for inode exhaustion on
ext? is not very plausible since hard links do not take up inodes,
symlinks are not backups and everything else requires disk blocks. So,
since reformatting ext2/ext3 to one inode per block is possible
(regardless of disk capacity), I see no way how a reformatted file
system might run out of inodes before it runs out of blocks.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Wed, 26 Jul 2006, Matthias Andree wrote:

> But the assertion that some backup was the cause for inode exhaustion on
> ext? is not very plausible since hard links do not take up inodes,
> symlinks are not backups and everything else requires disk blocks. So,
> since reformatting ext2/ext3 to one inode per block is possible
> (regardless of disk capacity), I see no way how a reformatted file
> system might run out of inodes before it runs out of blocks.

OK; ext2/ext3 require 1k blocks, but still you need heaps of files < 1k
to run out of inodes without running of space.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Bernd Eckenfels]

Hello,

I know thats not relevant for the discussion, but I wanted to share my
experiences anyway (to emphasis how important df-i monitoring on smaller
filesystems is):

Matthias Andree <matthias.andree@gmx.de> wrote:
> But the assertion that some backup was the cause for inode exhaustion on
> ext? is not very plausible since hard links do not take up inodes,
> symlinks are not backups and everything else requires disk blocks. So,
> since reformatting ext2/ext3 to one inode per block is possible
> (regardless of disk capacity), I see no way how a reformatted file
> system might run out of inodes before it runs out of blocks.

Well I had actually the problem on a tmpfs where I had too many zero byte
files...

Gruss
Bernd



> 

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Buddy Lucas]

On 7/26/06, Bernd Eckenfels <be-news06@lina.inka.de> wrote:
>
> Matthias Andree <matthias.andree@gmx.de> wrote:
> > But the assertion that some backup was the cause for inode exhaustion on
> > ext? is not very plausible since hard links do not take up inodes,
> > symlinks are not backups and everything else requires disk blocks. So,
> > since reformatting ext2/ext3 to one inode per block is possible
> > (regardless of disk capacity), I see no way how a reformatted file
> > system might run out of inodes before it runs out of blocks.
>
> Well I had actually the problem on a tmpfs where I had too many zero byte
> files...

Yes, I once ran out of inodes because logrotate kept rotating and
compressing already compressed and empty logfiles. I can't remember
how many seconds it took me to add 'df -i' to our monitoring system.

This, however, was not a feature of the software. I assume.

So, any company that considers the remote possibility of seeking a
$250,000 solution, where the alternative is to buy a 36GB hard drive,
please give me a call.


Cheers,
Buddy

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Matthias Andree wrote:
> On Tue, 25 Jul 2006, David Masover wrote:
> 
>> Matthias Andree wrote:
>>> On Tue, 25 Jul 2006, Denis Vlasenko wrote:
>>>
>>>> I, on the contrary, want software to impose as few limits on me
>>>> as possible.
>>> As long as it's choosing some limit, I'll pick the one with fewer
>>> surprises.
>> Running out of inodes would be pretty surprising for me.
> 
> No offense: Then it was a surprise for you because you closed your eyes
> and didn't look at df -i or didn't have monitors in place.

Or because my (hypothetical) business exploded before I had the chance.

After all, you could make the same argument about bandwidth, until you 
get Slashdotted.  Surprise!

> There is no way to ask how many files with particular hash values you
> can still stuff into a reiserfs 3.X. There, you're running into a brick
> wall that only your forehead will "see" when you touch it.

That's true, so you may be correct about "less" surprises.  So, it 
depends which is more valuable -- fewer surprises, or fewer limits?

That's not a hypothetical statement, and I don't really know.  I can see 
both sides of this one.  But I do hope that once Reiser4 is stable 
enough for you, it will be predictable enough.

> But the assertion that some backup was the cause for inode exhaustion on
> ext? is not very plausible since hard links do not take up inodes,
> symlinks are not backups and everything else requires disk blocks. So,

Ok, where's the assertion that symlinks are not backups?  Or not used in 
backup software?  What about directories full of hardlinks -- the dirs 
themselves must use something, right?

Anyway, it wasn't my project that hit this limit.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

Horst H. von Brand wrote:

> 18GiB = 18 million KiB, you do have a point there. But 40 million files on
> that, with some space to spare, just doesn't add up.

Right, ok...

Here's a quick check of my box.  I've explicitly stated which root-level
directories to search, to avoid nfs mounts, chrooted OSes, and virtual
filesystems like /proc and /sys.

elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt
/root /sbin /tmp /usr /var -type f -size 1 | wc -l
246127

According to the "find" manpage:

-size n[bckw]
      File uses n units of space.  The units are  512-byte  blocks  by
      default  or  if `b' follows n, bytes if `c' follows n, kilobytes
      if `k' follows n, or 2-byte words if `w' follows  n.   The  size
      does  not  count  indirect  blocks,  but it does count blocks in
      sparse files that are not actually allocated.


And I certainly didn't plan it that way.  And this is my desktop box,
and I'm just one user.  Most of the space is taken up by movies.

And yet, I have almost 250k files at the moment whose size is less than
512 bytes.  And this is a normal usage pattern.  It's not hard to
imagine something prone to creating lots of tiny files, combined with
thousands of users, easily hitting some 40 mil files -- and since none
of them are movies, it could fit in 18 gigs.

I mean, just for fun:

elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt
/root /sbin /tmp /usr /var | wc -l
866160

It may not be a good idea, but it's possible.  And one of the larger
reasons it's not a good idea is that most filesystems can't handle it.
Kind of like how BitTorrent is a very bad idea on dialup, but a very
good idea on broadband.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 890 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Lang]

On Tue, 25 Jul 2006, David Masover wrote:

> Horst H. von Brand wrote:
>
>> 18GiB = 18 million KiB, you do have a point there. But 40 million files on
>> that, with some space to spare, just doesn't add up.

if you have 18 million KiB and each file is a single block (512 Bytes = 0.5 Kib) 
then assuming zero overhead you could fit 18 Million KiB / 0.5 KiB = 36 Million 
files on the drive.

thus being scheptical about 40 million files on a 18G drive.

this is only possible if you are abel to have multiple files per 512 byte block.

David Lang

> Right, ok...
>
> Here's a quick check of my box.  I've explicitly stated which root-level
> directories to search, to avoid nfs mounts, chrooted OSes, and virtual
> filesystems like /proc and /sys.
>
> elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt
> /root /sbin /tmp /usr /var -type f -size 1 | wc -l
> 246127
>
> According to the "find" manpage:
>
> -size n[bckw]
>      File uses n units of space.  The units are  512-byte  blocks  by
>      default  or  if `b' follows n, bytes if `c' follows n, kilobytes
>      if `k' follows n, or 2-byte words if `w' follows  n.   The  size
>      does  not  count  indirect  blocks,  but it does count blocks in
>      sparse files that are not actually allocated.
>
>
> And I certainly didn't plan it that way.  And this is my desktop box,
> and I'm just one user.  Most of the space is taken up by movies.
>
> And yet, I have almost 250k files at the moment whose size is less than
> 512 bytes.  And this is a normal usage pattern.  It's not hard to
> imagine something prone to creating lots of tiny files, combined with
> thousands of users, easily hitting some 40 mil files -- and since none
> of them are movies, it could fit in 18 gigs.
>
> I mean, just for fun:
>
> elite ~ # find /bin/ /boot/ /dev/ /emul/ /etc/ /home /lib32 /lib64 /opt
> /root /sbin /tmp /usr /var | wc -l
> 866160
>
> It may not be a good idea, but it's possible.  And one of the larger
> reasons it's not a good idea is that most filesystems can't handle it.
> Kind of like how BitTorrent is a very bad idea on dialup, but a very
> good idea on broadband.
>
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

David Lang wrote:
> On Tue, 25 Jul 2006, David Masover wrote:
> 
>> Horst H. von Brand wrote:
>>
>>> 18GiB = 18 million KiB, you do have a point there. But 40 million
>>> files on
>>> that, with some space to spare, just doesn't add up.
> 
> if you have 18 million KiB and each file is a single block (512 Bytes =
> 0.5 Kib) then assuming zero overhead you could fit 18 Million KiB / 0.5
> KiB = 36 Million files on the drive.
> 
> thus being scheptical about 40 million files on a 18G drive.
> 
> this is only possible if you are abel to have multiple files per 512
> byte block.

I believe Reiser4 does this.  Does ext3?  I know I heard somewhere in
this thread that you can't set the blocksize lower than 1k...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 890 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

> > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big
> > one being a fixed number of inodes that can't be adjusted on the fly,
> 
> Right. Plan ahead.

Ok: Assume that i've read the mke2fs manpage and added more inodes to
my filesystem.

So: What happens if i need to grow my filesystem by 200% after 1-2
years? Can i add more inodes to Ext3 on-the-fly ?

A filesystem with a fixed number of inodes (= not readjustable while
mounted) is ehr.. somewhat unuseable for a lot of people with
big and *flexible* storage needs (Talking about NetApp/EMC owners)

Why are a lot of Solaris-people using (buying) VxFS? Maybe because UFS
also has such silly limitations? (..and performs awkward with trillions
of files..?..)


Ext3 may be a fine and stable Filesystem and works well for a lot of
people. But there are also a lot of people who need 'something' better
like: VxFS, WAFL and Reiser4.

Btw: Do you know Adics 'StorNext Filesystem' ? 
IMHO Ext3 will never be able to do such things...
But with Reiser4.. .. if someone wrote a plugin .. ;-)



Regards,
 Adrian

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Adrian Ulrich schrieb am 2006-07-31:

> > > And EXT3 imposes practical limits that ReiserFS doesn't as well. The big
> > > one being a fixed number of inodes that can't be adjusted on the fly,
> > 
> > Right. Plan ahead.
> 
> Ok: Assume that i've read the mke2fs manpage and added more inodes to
> my filesystem.
> 
> So: What happens if i need to grow my filesystem by 200% after 1-2
> years? Can i add more inodes to Ext3 on-the-fly ?

Since you "grow", you'll be using resize2fs (or growfs or mkfs -G for
UFS). resize2fs and the other tools do exactly that: add inodes - and
you could easily have told this either from reading the resize2fs code
or just trying it on a temp file:

  -- create file system
  dd if=/dev/zero of=/tmp/foo bs=1k count=50000
  /sbin/mke2fs -F -j /tmp/foo

  -- check no. of inodes
  /sbin/tune2fs -l /tmp/foo | grep -i inode | head -2
  # Inode count:              12544
  # Free inodes:              12533

  -- resize
  /sbin/e2fsck -f /tmp/foo
  dd if=/dev/zero bs=1k count=50000 >>/tmp/foo
  /sbin/resize2fs /tmp/foo

  -- check no. of inodes
  /sbin/tune2fs -l /tmp/foo | grep -i inode
  # Inode count:              23296
  # Free inodes:              23285

Trying the same after mke2fs -b 1024 -i 1024 shows that the inode
density will continue to be respected.

FreeBSD 6.1's growfs(8) increases the number of inodes. This is
documented to work since 4.4.

Solaris 8's mkfs -G also increases the number of inodes and apparently
also works for mounted file systems.

This looks rather like an education issue rather than a technical limit.

> A filesystem with a fixed number of inodes (= not readjustable while
> mounted) is ehr.. somewhat unuseable for a lot of people with
> big and *flexible* storage needs (Talking about NetApp/EMC owners)

Which is untrue at least for Solaris, which allows resizing a life file
system. FreeBSD and Linux require an unmount.

> Why are a lot of Solaris-people using (buying) VxFS? Maybe because UFS
> also has such silly limitations? (..and performs awkward with trillions
> of files..?..)

Well, such "silly limitations"... looks like they are mostly hot air
spewn by marketroids that need to justify people spending money on their
new filesystem.

The only problem remains if you grossly overestimate the average file
size and with it underestimate the number of inodes needed. But even
then, I'd be interested to know if that's a real problem for systems
such as ZFS.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

Hello Matthias,

> This looks rather like an education issue rather than a technical limit.

We aren't talking about the same issue: I was asking to do it
on-the-fly. Umounting the filesystem, running e2fsck and resize2fs
is something different ;-)

> Which is untrue at least for Solaris, which allows resizing a life file
> system. FreeBSD and Linux require an unmount.

Correct: You can add more inodes to a Solaris UFS on-the-fly if you are
lucky enough to have some free space available.

A colleague of mine happened to create a ~300gb filesystem and started
to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
to the new LUN. At about 70% the filesystem ran out of inodes; Not a
big deal with VxFS because such a problem is fixable within seconds.
What would have happened if he had used UFS? mkfs -G wouldn't work
because he had no additional Diskspace left... *ouch*..

> Well, such "silly limitations"... looks like they are mostly hot air
> spewn by marketroids that need to justify people spending money on their
> new filesystem.

Have you ever seen VxFS or WAFL in action?


> But even then, I'd be interested to know if that's a real problem for systems
> such as ZFS.

ZFS uses 'dnodes'. The dnodes are allocated on demand from your
available space so running out of [di]nodes is impossible.

Great to see that Sun ships a state-of-the-art Filesystem with
Solaris... I think linux should do the same...

Regards,
 Adrian

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 610 bytes --]

On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote:
> A colleague of mine happened to create a ~300gb filesystem and started
> to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> to the new LUN. At about 70% the filesystem ran out of inodes; Not a

So preparation work wasn't done.

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
Signature of:            Ich hatte in letzter Zeit ein bisschen viel Realitycheck.
the second  :                 Langsam möchte ich mal wieder weiterträumen können.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Jan-Benedict Glaw wrote:
> On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote:
>> A colleague of mine happened to create a ~300gb filesystem and started
>> to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
>> to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> 
> So preparation work wasn't done.

So what?

Yes, you need to do preparation.  But it is really nice if the 
filesystem can do that work for you.

Let me put it this way -- You're back in college, and it's time to write 
a thesis.  You have a choice of software packages:



Package A:  You have to specify how many pages, and how many words, 
you're likely to use before you start typing.  Guess too high, and 
you'll print out a bunch of blank pages at the end.  Guess too low, and 
you'll run out of space and have to start over, copy and paste your 
document back in, and hope it gets all the formatting right, which it 
probably won't.

Package B:  Your document grows as you type.  When it's time to print, 
only the pages you've actually written something on -- but all of the 
pages you've actually written something on -- are printed.



All other things being equal, which would you choose?  Which one seems 
more modern?

Look, I understand the argument against ReiserFS v3 -- it has another 
limitation that you don't even know about.  That other limitation is 
scary -- that's like being able to type as many words as you want, but 
once you type enough pages (no way of knowing how many), pages start 
randomly disappearing from the middle of your document.

But the argument that no one cares about inode limits?  Really, stop 
kidding yourselves.  It's 2006.  The limits are starting to look 
ridiculous.  Just because they're workable doesn't mean we should have 
to live with them.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Bernd Eckenfels]

David Masover <ninja@slaphack.com> wrote:
> Yes, you need to do preparation.  But it is really nice if the 
> filesystem can do that work for you.

The filesystem defaults are more than useable for most situations. You
simply dont migrate a filesystem of that size without doing some
calculation.

Gruss
Bernd

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1729 bytes --]

On Mon, 2006-07-31 11:44:25 -0500, David Masover <ninja@slaphack.com> wrote:
> Jan-Benedict Glaw wrote:
> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich 
> > <reiser4@blinkenlights.ch> wrote:
> > > A colleague of mine happened to create a ~300gb filesystem and started
> > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> >
> > So preparation work wasn't done.
> 
> Let me put it this way -- You're back in college, and it's time to write 
> a thesis.  You have a choice of software packages:
> 
> Package A:  You have to specify how many pages, and how many words, 
> you're likely to use before you start typing.  Guess too high, and 
> you'll print out a bunch of blank pages at the end.  Guess too low, and 
> you'll run out of space and have to start over, copy and paste your 
> document back in, and hope it gets all the formatting right, which it 
> probably won't.
> 
> Package B:  Your document grows as you type.  When it's time to print, 
> only the pages you've actually written something on -- but all of the 
> pages you've actually written something on -- are printed.
> 
> All other things being equal, which would you choose?  Which one seems 
> more modern?

:)  Well, given that TeX needs two (or even three!) runs to get all
page references right, why should I choose MS Word, where you won't
see that problem at all?

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
 Signature of:                               If it doesn't work, force it.
 the second  :                      If it breaks, it needed replacing anyway.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Rudy Zijlstra]

On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote:

> On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote:
>> A colleague of mine happened to create a ~300gb filesystem and started
>> to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
>> to the new LUN. At about 70% the filesystem ran out of inodes; Not a
>
> So preparation work wasn't done.
>
> MfG, JBG

Of course you are right. Preparation work was not fully done. And using 
ext1 would also have been possible. I suspect you are still using ext1, 
cause with proper preparation it is perfectly usable.

Cheers,

Rudy

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]

On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote:
> On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote:
> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich 
> > <reiser4@blinkenlights.ch> wrote:
> > > A colleague of mine happened to create a ~300gb filesystem and started
> > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> > 
> > So preparation work wasn't done.
> 
> Of course you are right. Preparation work was not fully done. And using 
> ext1 would also have been possible. I suspect you are still using ext1, 
> cause with proper preparation it is perfectly usable.

Nope, I'm a quite happy ext3 user for quite divergent workloads.
Starting with storing loads of small files in a hugh number of
directories (mkfs -T news) over hugh numbers of files in few
directories (mkfs -O dir_index) to only a small number of really hugh
files (mkfs -T largefile4).

All that I still need to catch up with is 4TB sized files. But I guess
this is really an uncommon workload, but ISTR that somebody even works
on this.

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
 Signature of:                       Don't believe in miracles: Rely on them!
 the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2352 bytes --]

On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote:
> On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote:
> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich 
> > <reiser4@blinkenlights.ch> wrote:
> > > A colleague of mine happened to create a ~300gb filesystem and started
> > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> >
> > So preparation work wasn't done.
> 
> Of course you are right. Preparation work was not fully done. And using 
> ext1 would also have been possible. I suspect you are still using ext1, 
> cause with proper preparation it is perfectly usable.

Oh, and before people start laughing at me, here are some personal or
friend's experiences with different filesystems:

  * reiser3: A HDD containing a reiser3 filesystem was tried to be
    booted on a machine that fucked up DMA writes. Fortunately, it
    crashed really soon (right after going for read-write.)  After
    rebooting the HDD on a sane PeeCee, it refused to boot. Starting
    off some rescue system showed an _empty_ root filesystem.

  * A friend's XFS data partition (portable USB/FireWire HDD) once
    crashed due to being hot-unplugged off the USB.  The in-kernel XFS
    driver refused to mount that thing again, and the tools also
    refused to fix any errors. (Don't ask, no details at my hands...)

  * JFS just always worked for me. Though I've never ever had a broken
    HDD where it (or it's tools) could have shown how well-done they
    were, so from a crash-recovery point of view, it's untested.

  * Being a regular ext3 user, I had lots of broken HDDs containing
    ext3 filesystems. For every single case, it has been easy fixing
    the filesystem after cloning. Just _once_, fsck wasn't able to fix
    something, so I did it manually with some disk editor. This worked
    well because the on-disk data structures are actually as simple as
    they are.

ext3 always worked well for me, so why should I abandon it?

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
 Signature of:                               If it doesn't work, force it.
 the second  :                      If it breaks, it needed replacing anyway.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Dan Oglesby]

On Mon, 2006-07-31 at 19:32 +0200, Jan-Benedict Glaw wrote:
> On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote:
> > On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote:
> > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich 
> > > <reiser4@blinkenlights.ch> wrote:
> > > > A colleague of mine happened to create a ~300gb filesystem and started
> > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> > >
> > > So preparation work wasn't done.
> > 
> > Of course you are right. Preparation work was not fully done. And using 
> > ext1 would also have been possible. I suspect you are still using ext1, 
> > cause with proper preparation it is perfectly usable.
> 
> Oh, and before people start laughing at me, here are some personal or
> friend's experiences with different filesystems:
> 
>   * reiser3: A HDD containing a reiser3 filesystem was tried to be
>     booted on a machine that fucked up DMA writes. Fortunately, it
>     crashed really soon (right after going for read-write.)  After
>     rebooting the HDD on a sane PeeCee, it refused to boot. Starting
>     off some rescue system showed an _empty_ root filesystem.
> 
>   * A friend's XFS data partition (portable USB/FireWire HDD) once
>     crashed due to being hot-unplugged off the USB.  The in-kernel XFS
>     driver refused to mount that thing again, and the tools also
>     refused to fix any errors. (Don't ask, no details at my hands...)
> 
>   * JFS just always worked for me. Though I've never ever had a broken
>     HDD where it (or it's tools) could have shown how well-done they
>     were, so from a crash-recovery point of view, it's untested.
> 
>   * Being a regular ext3 user, I had lots of broken HDDs containing
>     ext3 filesystems. For every single case, it has been easy fixing
>     the filesystem after cloning. Just _once_, fsck wasn't able to fix
>     something, so I did it manually with some disk editor. This worked
>     well because the on-disk data structures are actually as simple as
>     they are.
> 
> ext3 always worked well for me, so why should I abandon it?
> 
> MfG, JBG

I've lost EXT2 and EXT3 filesystems from machines with no bad hardware
(power loss during writes).

I've recovered all but a handful of files from a RAID-5 array running
ReiserFS v3 that had two drives fail in rapid succession with bad
sectors.

Sometimes you're lucky, sometimes you're not.

--Dan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Jan-Benedict Glaw schrieb am 2006-07-31:

> On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl> wrote:
> > On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote:
> > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich 
> > > <reiser4@blinkenlights.ch> wrote:
> > > > A colleague of mine happened to create a ~300gb filesystem and started
> > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> > >
> > > So preparation work wasn't done.
> > 
> > Of course you are right. Preparation work was not fully done. And using 
> > ext1 would also have been possible. I suspect you are still using ext1, 
> > cause with proper preparation it is perfectly usable.
> 
> Oh, and before people start laughing at me, here are some personal or
> friend's experiences with different filesystems:
> 
>   * reiser3: A HDD containing a reiser3 filesystem was tried to be
>     booted on a machine that fucked up DMA writes. Fortunately, it
>     crashed really soon (right after going for read-write.)  After
>     rebooting the HDD on a sane PeeCee, it refused to boot. Starting
>     off some rescue system showed an _empty_ root filesystem.

Massive hardware problems don't count. ext2/ext3 doesn't look much better in
such cases. I had a machine with RAM gone bad (no ECC - I wonder what
idiot ordered a machine without ECC for a server, but anyways) and it
fucked up every 64th bit - only in a certain region. Guess what happened
to the fs when it went into e2fsck after a reboot. Boom. Same with a
dead DPTA that lost every 16th block or so, the rescue in the first case
was swapping the RAM and "amrecover" and in the second swapping the
drive and "dsmc restore". OTOH, kernel panics on bad blocks are a no-no
of course.

>   * A friend's XFS data partition (portable USB/FireWire HDD) once
>     crashed due to being hot-unplugged off the USB.  The in-kernel XFS
>     driver refused to mount that thing again, and the tools also
>     refused to fix any errors. (Don't ask, no details at my hands...)

Don't use write caches then. (Though I've seen NUL-filled blocks in new
files or appended to files after in 2001 or 2002.)

>   * JFS just always worked for me. Though I've never ever had a broken
>     HDD where it (or it's tools) could have shown how well-done they
>     were, so from a crash-recovery point of view, it's untested.

SUSE removed JFS support from their installation tool for "technical
reasons" they didn't specify in the release notes. Whatever.

> ext3 always worked well for me, so why should I abandon it?

Plus, it and its tools are maintained.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1447 bytes --]

On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote:
> Jan-Benedict Glaw schrieb am 2006-07-31:
> >   * reiser3: A HDD containing a reiser3 filesystem was tried to be
> >     booted on a machine that fucked up DMA writes. Fortunately, it
> >     crashed really soon (right after going for read-write.)  After
> >     rebooting the HDD on a sane PeeCee, it refused to boot. Starting
> >     off some rescue system showed an _empty_ root filesystem.
> 
> Massive hardware problems don't count. ext2/ext3 doesn't look much better in
> such cases. I had a machine with RAM gone bad (no ECC - I wonder what

They do! Very much, actually. These happen In Real Life, so I have to
pay attention to them. Once you're in setups with > 10000 machines,
everything counts. At some certain point, you can even use HDD's
temperature sensors in old machines to diagnose dead fans.

Everything that eases recovery for whatever reason is something you
have to pay attention to. The simplicity of ext{2,3} is something I
really fail to find proper words for. As well as the really good fsck.
Once seen a SIGSEGV'ing fsck, you really don't want to go there.

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
 Signature of:                       Eine Freie Meinung in einem Freien Kopf
 the second  :                     für einen Freien Staat voll Freier Bürger.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Clay Barnes]

On 20:43 Mon 31 Jul     , Jan-Benedict Glaw wrote:
> On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote:
> > Jan-Benedict Glaw schrieb am 2006-07-31:
> > >   * reiser3: A HDD containing a reiser3 filesystem was tried to be
> > >     booted on a machine that fucked up DMA writes. Fortunately, it
> > >     crashed really soon (right after going for read-write.)  After
> > >     rebooting the HDD on a sane PeeCee, it refused to boot. Starting
> > >     off some rescue system showed an _empty_ root filesystem.
> > 
> > Massive hardware problems don't count. ext2/ext3 doesn't look much better in
> > such cases. I had a machine with RAM gone bad (no ECC - I wonder what
> 
> They do! Very much, actually. These happen In Real Life, so I have to
> pay attention to them. Once you're in setups with > 10000 machines,
> everything counts. At some certain point, you can even use HDD's
> temperature sensors in old machines to diagnose dead fans.

I think what he meant was that it is unfair to blame reiser3 for data
loss in a massive failure situation as a case example by itself.  What
would be more appropriate (and fair) is saying something along the lines
of "in a machine with DMA failure, reiser3 hosed a drive, while ext3
only lost x data (or nothing).  In my situation, every bit of massive
failure robustness counts... "  This of course assumes you actually had
the *exact* same problem with hardware under ext3, pretty much in every
detail.  Of course, so many subtleties interact in massive ways with
filesystems and hardware problems, so we have to keep in mind how much
difference that can make (all the difference in the world), and that,
really, without a statistically significant sample size and some real
analysis, hardware failure comparisons are impossible to do fairly.

Of course, if ext3 were proven to be more robust against failures, I bet
the reiser team would be very interested in all the forensic data you
can offer, since, from what I've seen, they are always trying to make
reiser as good as possible---in speed, flexability, *and* robustness.
If they didn't care about the last, they'd benifit greatly from not
journaling!  :-P

--Clay
> 
> Everything that eases recovery for whatever reason is something you
> have to pay attention to. The simplicity of ext{2,3} is something I
> really fail to find proper words for. As well as the really good fsck.
> Once seen a SIGSEGV'ing fsck, you really don't want to go there.
> 
> MfG, JBG
> 
> -- 
>        Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
>  Signature of:                       Eine Freie Meinung in einem Freien Kopf
>  the second  :                     für einen Freien Staat voll Freier Bürger.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]

On Mon, 2006-07-31 12:17:12 -0700, Clay Barnes <clay.barnes@gmail.com> wrote:
> On 20:43 Mon 31 Jul     , Jan-Benedict Glaw wrote:
> > On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote:
> > > Jan-Benedict Glaw schrieb am 2006-07-31:
[Crippled DMA writes]
> > > Massive hardware problems don't count. ext2/ext3 doesn't look much better in
> > > such cases. I had a machine with RAM gone bad (no ECC - I wonder what
> > 
> > They do! Very much, actually. These happen In Real Life, so I have to
> 
> I think what he meant was that it is unfair to blame reiser3 for data
> loss in a massive failure situation as a case example by itself.  What

Crippling a few KB of metadata in the ext{2,3} case probably wouldn't
fobar the filesystem...

> failure robustness counts... "  This of course assumes you actually had
> the *exact* same problem with hardware under ext3, pretty much in every
> detail.  Of course, so many subtleties interact in massive ways with

The point is that it's quite hard to really fuck up ext{2,3} with only
some KB being written while it seems (due to the
fragile^Wsophisticated on-disk data structures) that it's just easy to
kill a reiser3 filesystem.

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
   Signature of:                            Zensur im Internet? Nein danke!
   the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Jan-Benedict Glaw wrote:
> On Mon, 2006-07-31 12:17:12 -0700, Clay Barnes <clay.barnes@gmail.com> wrote:
>> On 20:43 Mon 31 Jul     , Jan-Benedict Glaw wrote:
>>> On Mon, 2006-07-31 20:11:20 +0200, Matthias Andree <matthias.andree@gmx.de> wrote:
>>>> Jan-Benedict Glaw schrieb am 2006-07-31:
> [Crippled DMA writes]
>>>> Massive hardware problems don't count. ext2/ext3 doesn't look much better in
>>>> such cases. I had a machine with RAM gone bad (no ECC - I wonder what
>>> They do! Very much, actually. These happen In Real Life, so I have to
>> I think what he meant was that it is unfair to blame reiser3 for data
>> loss in a massive failure situation as a case example by itself.  What
> 
> Crippling a few KB of metadata in the ext{2,3} case probably wouldn't
> fobar the filesystem...

Probably.  By the time a few KB of metadata are corrupted, I'm reaching 
for my backup.  I don't care what filesystem it is or how easy it is to 
edit the on-disk structures.

This isn't to say that having robust on-disk structures isn't a good 
thing.  I have no idea how Reiser4 will hold up either way.  But 
ultimately, what you want is the journaling (so power failure / crashes 
still leave you in an OK state), backups (so when blocks go bad, you 
don't care), and performance (so you can spend less money on hardware 
and more money on backup hardware).

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Bernd Schubert]

On Monday 31 July 2006 21:29, Jan-Benedict Glaw wrote:
>
> The point is that it's quite hard to really fuck up ext{2,3} with only
> some KB being written while it seems (due to the
> fragile^Wsophisticated on-disk data structures) that it's just easy to
> kill a reiser3 filesystem.
>

Well, I was once very 'luckily' and after a system crash (*) e2fsck put all 
files into lost+found. Sure, I never experienced this again, but I also never 
experienced something like this with reiserfs. So please, stop this kind of 
FUD against reiser3.6.
While filesystem speed is nice, it also would be great if reiser4.x would be 
very robust against any kind of hardware failures.

(*) The problem was a specific mainboard + video-card + driver combination. As 
soon as X started up, the system entirely crashed. I don't believe many bytes 
were written, but I also can't prove it.

-- 
Bernd Schubert
PCI / Theoretische Chemie
Universität Heidelberg
INF 229
69120 Heidelberg

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Bernd Schubert <bernd-schubert@gmx.de> wrote:
> On Monday 31 July 2006 21:29, Jan-Benedict Glaw wrote:
> > The point is that it's quite hard to really fuck up ext{2,3} with only
> > some KB being written while it seems (due to the
> > fragile^Wsophisticated on-disk data structures) that it's just easy to
> > kill a reiser3 filesystem.

> Well, I was once very 'luckily' and after a system crash (*) e2fsck put
> all files into lost+found. Sure, I never experienced this again, but I
> also never experienced something like this with reiserfs. So please, stop
> this kind of FUD against reiser3.6.

It isn't FUD. One data point doesn't allow you to draw conclusions.

Yes, I've seen/heard of ext2/ext3 failures and data loss too. But at least
the same number for ReiserFS. And I know it is outnumbered 10 to 1 or so in
my sample, so that would indicate at a 10 fold higher probability of
catastrophic data loss, other factors mostly the same.

> While filesystem speed is nice, it also would be great if reiser4.x would be 
> very robust against any kind of hardware failures.

Can't have both.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

> > While filesystem speed is nice, it also would be great if reiser4.x would be 
> > very robust against any kind of hardware failures.
> 
> Can't have both.

..and some people simply don't care about this:

If you are running a 'big' Storage-System with battery protected
WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc..
you don't need your filesystem beeing super-robust against bad sectors
and such stuff because:

 a) You've paid enough money to let the storage care about 
    Hardware issues.
 b) If your storage is on fire you can do a failover using the mirror.
 c) And if someone ran dd if=/dev/urandom of=/dev/sda you could
    even rollback your Snapshot.
    (Btw: i did this once to a Reiser4 filesystem (overwritten about
    1.2gb). fsck.reiser4 --rebuild-sb was able to fix it.)


..but what you really need is a flexible and **fast** filesystem: Like
Reiser4.

(Yeah.. yeah.. i know: ext3 is also flexible and fast.. but Reiser4
simply is *MUCH* faster than ext3 for 'my' workload/application).

Regards,
 Adrian

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Alan Cox]

Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich:
> WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc..
> you don't need your filesystem beeing super-robust against bad sectors
> and such stuff because:

You do it turns out. Its becoming an issue more and more that the sheer
amount of storage means that the undetected error rate from disks,
hosts, memory, cables and everything else is rising.

There has been a great deal of discussion about this at the filesystem
and kernel summits - and data is getting kicked the way of networking -
end to end not reliability in the middle.

The sort of changes this needs hit the block layer and ever fs.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Alan Cox wrote:
> Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich:
>> WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc..
>> you don't need your filesystem beeing super-robust against bad sectors
>> and such stuff because:
> 
> You do it turns out. Its becoming an issue more and more that the sheer
> amount of storage means that the undetected error rate from disks,
> hosts, memory, cables and everything else is rising.

Yikes.  Undetected.

Wait, what?  Disks, at least, would be protected by RAID.  Are you 
telling me RAID won't detect such an error?

It just seems wholly alien to me that errors would go undetected, and 
we're OK with that, so long as our filesystems are robust enough.  If 
it's an _undetected_ error, doesn't that cause way more problems 
(impossible problems) than FS corruption?  Ok, your FS is fine -- but 
now your bank database shows $1k less on random accounts -- is that ok?

> There has been a great deal of discussion about this at the filesystem
> and kernel summits - and data is getting kicked the way of networking -
> end to end not reliability in the middle.

Sounds good, but I've never let discussions by people smarter than me 
prevent me from asking the stupid questions.

> The sort of changes this needs hit the block layer and ever fs.

Seems it would need to hit every application also...

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Gregory Maxwell]

On 8/1/06, David Masover <ninja@slaphack.com> wrote:
> Yikes.  Undetected.
>
> Wait, what?  Disks, at least, would be protected by RAID.  Are you
> telling me RAID won't detect such an error?

Unless the disk ECC catches it raid won't know anything is wrong.

This is why ZFS offers block checksums... it can then try all the
permutations of raid regens to find a solution which gives the right
checksum.

Every level of the system must be paranoid and take measure to avoid
corruption if the system is to avoid it... it's a tough problem. It
seems that the ZFS folks have addressed this challenge by building as
much of what is classically separate layers into one part.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Gregory Maxwell wrote:

> This is why ZFS offers block checksums... it can then try all the
> permutations of raid regens to find a solution which gives the right
> checksum.
>
ZFS performance is pretty bad in the only benchmark I have seen of it. 
Does anyone have serious benchmarks of it?  I suspect that our
compression plugin (with ecc) will outperform it.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Gregory Maxwell wrote:
> On 8/1/06, David Masover <ninja@slaphack.com> wrote:
>> Yikes.  Undetected.
>>
>> Wait, what?  Disks, at least, would be protected by RAID.  Are you
>> telling me RAID won't detect such an error?
> 
> Unless the disk ECC catches it raid won't know anything is wrong.
> 
> This is why ZFS offers block checksums... it can then try all the
> permutations of raid regens to find a solution which gives the right
> checksum.

Isn't there a way to do this at the block layer?  Something in 
device-mapper?

> Every level of the system must be paranoid and take measure to avoid
> corruption if the system is to avoid it... it's a tough problem. It
> seems that the ZFS folks have addressed this challenge by building as
> much of what is classically separate layers into one part.

Sounds like bad design to me, and I can point to the antipattern, but 
what do I know?

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

> > This is why ZFS offers block checksums... it can then try all the
> > permutations of raid regens to find a solution which gives the right
> > checksum.
> 
> Isn't there a way to do this at the block layer?  Something in 
> device-mapper?

Remember: Suns new Filesystem + Suns new Volume Manager = ZFS

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Alan Cox]

Ar Maw, 2006-08-01 am 11:44 -0500, ysgrifennodd David Masover:
> Yikes.  Undetected.
> 
> Wait, what?  Disks, at least, would be protected by RAID.  Are you 
> telling me RAID won't detect such an error?

Yes.

RAID deals with the case where a device fails. RAID 1 with 2 disks can
in theory detect an internal inconsistency but cannot fix it.

> we're OK with that, so long as our filesystems are robust enough.  If 
> it's an _undetected_ error, doesn't that cause way more problems 
> (impossible problems) than FS corruption?  Ok, your FS is fine -- but 
> now your bank database shows $1k less on random accounts -- is that ok?

Not really no. Your bank is probably using a machine (hopefully using a
machine) with ECC memory, ECC cache and the like. The UDMA and SATA
storage subsystems use CRC checksums between the controller and the
device. SCSI uses various similar systems - some older ones just use a
parity bit so have only a 50/50 chance of noticing a bit error.

Similarly the media itself is recorded with a lot of FEC (forward error
correction) so will spot most changes.

Unfortunately when you throw this lot together with astronomical amounts
of data you get burned now and then, especially as most systems are not
using ECC ram, do not have ECC on the CPU registers and may not even
have ECC on the caches in the disks.

> > The sort of changes this needs hit the block layer and ever fs.
> 
> Seems it would need to hit every application also...

Depending how far you propogate it. Someone people working with huge
data sets already write and check user level CRC values for this reason
(in fact bitkeeper does it for one example). It should be relatively
cheap to get much of that benefit without doing application to
application just as TCP gets most of its benefit without going app to
app.

Alan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Alan, I have seen only anecdotal evidence against reiserfsck, and I have
seen formal tests from Vitaly  (which it seems a user has replicated)
where our fsck did better than ext3s.  Note that these tests are of the
latest fsck from us: I am sure everyone understands that it takes time
for an fsck to mature, and that our early fsck's were poor.  I will also
say the V4's fsck is more robust than V3's because we made disk format
changes specifically to help fsck.

Now I am not dismissing your anecdotes as I will never dismiss data I
have not seen, and it sounds like you have seen more data than most
people, but I must dismiss your explanation of them. 

Being able to throw away all of the tree but the leaves and twigs with
extent pointers and rebuild all of it makes V4 very robust, more so than
ext3.  This business of inodes not moving, I don't see what the
advantage is, we can lose the directory entry and rebuild just as well
as ext3, probably better because we can at least figure out what
directory it was in.

Vitaly can say all of this more expertly than I....

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Alan Cox wrote:
> Ar Maw, 2006-08-01 am 11:44 -0500, ysgrifennodd David Masover:
>> Yikes.  Undetected.
>>
>> Wait, what?  Disks, at least, would be protected by RAID.  Are you 
>> telling me RAID won't detect such an error?
> 
> Yes.
> 
> RAID deals with the case where a device fails. RAID 1 with 2 disks can
> in theory detect an internal inconsistency but cannot fix it.

Still, if it does that, that should be enough.  The scary part wasn't 
that there's an internal inconsistency, but that you wouldn't know.

And it can fix it if you can figure out which disk went.  Or give it 3 
disks and it should be entirely automatic -- admin gets paged, admin 
hotswaps in a new disk, done.

>> we're OK with that, so long as our filesystems are robust enough.  If 
>> it's an _undetected_ error, doesn't that cause way more problems 
>> (impossible problems) than FS corruption?  Ok, your FS is fine -- but 
>> now your bank database shows $1k less on random accounts -- is that ok?
> 
> Not really no. Your bank is probably using a machine (hopefully using a
> machine) with ECC memory, ECC cache and the like. The UDMA and SATA
> storage subsystems use CRC checksums between the controller and the
> device. SCSI uses various similar systems - some older ones just use a
> parity bit so have only a 50/50 chance of noticing a bit error.
> 
> Similarly the media itself is recorded with a lot of FEC (forward error
> correction) so will spot most changes.
> 
> Unfortunately when you throw this lot together with astronomical amounts
> of data you get burned now and then, especially as most systems are not
> using ECC ram, do not have ECC on the CPU registers and may not even
> have ECC on the caches in the disks.

It seems like this is the place to fix it, not the software.  If the 
software can fix it easily, great.  But I'd much rather rely on the 
hardware looking after itself, because when hardware goes bad, all bets 
are off.

Specifically, it seems like you do mention lots of hardware solutions, 
that just aren't always used.  It seems like storage itself is getting 
cheap enough that it's time to step back a year or two in Moore's Law to 
get the reliability.

>>> The sort of changes this needs hit the block layer and ever fs.
>> Seems it would need to hit every application also...
> 
> Depending how far you propogate it. Someone people working with huge
> data sets already write and check user level CRC values for this reason
> (in fact bitkeeper does it for one example). It should be relatively
> cheap to get much of that benefit without doing application to
> application just as TCP gets most of its benefit without going app to
> app.

And yet, if you can do that, I'd suspect you can, should, must do it at 
a lower level than the FS.  Again, FS robustness is good, but if the 
disk itself is going, what good is having your directory (mostly) intact 
if the files themselves have random corruptions?

If you can't trust the disk, you need more than just an FS which can 
mostly survive hardware failure.  You also need the FS itself (or maybe 
the block layer) to support bad block relocation and all that good 
stuff, or you need your apps designed to do that job by themselves.

It just doesn't make sense to me to do this at the FS level.  You 
mention TCP -- ok, but if TCP is doing its job, I shouldn't also need to 
implement checksums and other robustness at the protocol layer (http, 
ftp, ssh), should I?  Because in this analogy, it looks like TCP is the 
"block layer" and a protocol is the "fs".

As I understand it, TCP only lets the protocol/application know when 
something's seriously FUBARed and it has to drop the connection. 
Similarly, the FS (and the apps) shouldn't have to know about hardware 
problems until it really can't do anything about it anymore, at which 
point the right thing to do is for the FS and apps to go "oh shit" and 
drop what they're doing, and the admin replaces hardware and restores 
from backup.  Or brings a backup server online, or...



I guess my main point was that _undetected_ problems are serious, but if 
you can detect them, and you have at least a bit of redundancy, you 
should be good.  For instance, if your RAID reports errors that it can't 
fix, you bring that server down and let the backup server run.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Krzysztof Halasa]

David Masover <ninja@slaphack.com> writes:

>> RAID deals with the case where a device fails. RAID 1 with 2 disks
>> can
>> in theory detect an internal inconsistency but cannot fix it.
>
> Still, if it does that, that should be enough.  The scary part wasn't
> that there's an internal inconsistency, but that you wouldn't know.

RAID1 can do that in theory but it practice there is no verification,
so the other disk can perform another read simultaneously (thus
increasing performance).

Some high-end systems, maybe.

That would be hardly economical. Per-block checksums (like used by the
ZFS) are different story, they add only little additional load.

> And it can fix it if you can figure out which disk went.  Or give it 3
> disks and it should be entirely automatic -- admin gets paged, admin
> hotswaps in a new disk, done.

Yep, that could be done. Or with 2 disks with block checksums.
Actually, while I don't exactly buy their ads, I think ZFS employs
some useful ideas.

> And yet, if you can do that, I'd suspect you can, should, must do it
> at a lower level than the FS.  Again, FS robustness is good, but if
> the disk itself is going, what good is having your directory (mostly)
> intact if the files themselves have random corruptions?

With per-block checksum you will know. Of course, that's still not
end to end checksum.

> If you can't trust the disk, you need more than just an FS which can
> mostly survive hardware failure.  You also need the FS itself (or
> maybe the block layer) to support bad block relocation and all that
> good stuff, or you need your apps designed to do that job by
> themselves.

Drives have internal relocation mechanisms, I don't think the
filesystem needs to duplicate them (though it should try to work
with bad blocks - relocations are possible on write).

> It just doesn't make sense to me to do this at the FS level.  You
> mention TCP -- ok, but if TCP is doing its job, I shouldn't also need
> to implement checksums and other robustness at the protocol layer
> (http, ftp, ssh), should I?

Sure you have to, if you value your data.

> Similarly, the FS (and the apps) shouldn't have to know
> about hardware problems until it really can't do anything about it
> anymore, at which point the right thing to do is for the FS and apps
> to go "oh shit" and drop what they're doing, and the admin replaces
> hardware and restores from backup.  Or brings a backup server online,
> or...

I don't think so. Going read-only if the disk returns write error,
ok. But taking the fs offline? Why?

Continuous backups (or rather transaction logs) are possible but
who has them? Do you have them? Would you throw away several hours
of work just because some file (or, say, unused area) contained
unreadable block (which could probably be transient problem, and/or
could be corrected by write)?
-- 
Krzysztof Halasa

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Tue, 01 Aug 2006, David Masover wrote:

> >RAID deals with the case where a device fails. RAID 1 with 2 disks can
> >in theory detect an internal inconsistency but cannot fix it.
> 
> Still, if it does that, that should be enough.  The scary part wasn't 
> that there's an internal inconsistency, but that you wouldn't know.

You won't usually know, unless you run a consistency check: RAID-1 will
only read from one of the two drives for speed - except if you make the
system check consistency as it goes, which would imply waiting for both
disks at the same time. And in that case, you'd better look for drives
that allow to synchronize their platter staples in order to avoid the
read access penalty that waiting for two drives entails.

> And it can fix it if you can figure out which disk went.

If it's decent and detects a bad block, it'll log it and rewrite it with
data from the mirror and let the drive do the remapping through ARWE.

> >Depending how far you propogate it. Someone people working with huge
> >data sets already write and check user level CRC values for this reason
> >(in fact bitkeeper does it for one example). It should be relatively
> >cheap to get much of that benefit without doing application to
> >application just as TCP gets most of its benefit without going app to
> >app.
> 
> And yet, if you can do that, I'd suspect you can, should, must do it at 
> a lower level than the FS.  Again, FS robustness is good, but if the 
> disk itself is going, what good is having your directory (mostly) intact 
> if the files themselves have random corruptions?

Berkeley DB can, since version 4.1 (IIRC), write checksums (newer
versions document this as SHA1) on its database pages, to detect
corruptions and writes that were supposed to be atomic but failed
(because you cannot write 4K or 16K atomically on a disk drive).

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

> You do it turns out. Its becoming an issue more and more that the sheer
> amount of storage means that the undetected error rate from disks,
> hosts, memory, cables and everything else is rising.

IMHO the possibility to hit such a random-so-far-undetected-corruption
is very low with one of the big/expensive raid systems as they are
doing fancy stuff like 'disk scrubbing' and usually do fail disks
at very early stages..

 * I've seen storage systems from a BIG vendor die due to
   firmware bugs
 * I've seen FC-Cards die.. SAN-switches rebooted.. People used
   my cables to do rope skipping
 * We had Fire, non-working UPS and faulty diesel generators..

but so far the FSes (and applications) on the Storage never
complained about corrupted data.

..YMMV..

Btw: I don't think that Reiserfs really behaves this bad
with broken hardware. So far, Reiser3 survived 2 broken Harddrives
without problems while i've seen ext2/3 die 4 times so far...
(= everything inside /lost+found). Reiser4 survived
 # mkisofs . > /dev/sda

Lucky me.. maybe..


To get back on-topic:

Some people try very hard to claim that the world doesn't need
Reiser4 and that you can do everything with ext3.

Ext3 may be fine for them but some people (like me) really need Reiser4
because they got applications/workloads that won't work good (fast) on ext3.

Why is it such a big thing to include a filesystem?
Even if it's unstable: does anyone care? Eg: the HFS+ driver
is buggy (corrupted the FS of my OSX installation 3 times so far) but
does this buggyness affect people *not* using it? No.

Regards,
 Adrian

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Ric Wheeler]

Alan Cox wrote:
> Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich:
> 
>>WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc..
>>you don't need your filesystem beeing super-robust against bad sectors
>>and such stuff because:
> 
> 
> You do it turns out. Its becoming an issue more and more that the sheer
> amount of storage means that the undetected error rate from disks,
> hosts, memory, cables and everything else is rising.


I agree with Alan despite being an enthusiastic supporter of neat array 
based technologies.

Most people use absolutely giant disks in laptops and desktop systems 
(300GB & 500GB are common, 750GB on the way). File systems need to be as 
robust as possible for users of these systems as people are commonly 
storing personal "critical" data like photos mostly on these unprotected 
drives.

Even for the high end users, array based mirroring and so on can only do 
so much to protect you.

Mirroring a corrupt file system to a remote data center will mirror your 
corruption.

Rolling back to a snapshot typically only happens when you notice a 
corruption which can go undetected for quite a while, so even that will 
benefit from having "reliability" baked into the file system (i.e., it 
should grumble about corruption to let you know that you need to roll 
back or fsck or whatever).

An even larger issue is that our tools, like fsck, which are used to 
uncover these silent corruptions need to scale up to the point that they 
can uncover issues in minutes instead of days.  A lot of the focus at 
the file system workshop was around how to dramatically reduce the 
repair time of file systems.

In a way, having super reliable storage hardware is only as good as the 
file system layer on top of it - reliability needs to be baked into the 
entire IO system stack...

ric

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Ric Wheeler wrote:

> Alan Cox wrote:
>
>>
>>
>> You do it turns out. Its becoming an issue more and more that the sheer
>> amount of storage means that the undetected error rate from disks,
>> hosts, memory, cables and everything else is rising.
>
>
>
> I agree with Alan 

You will want to try our compression plugin, it has an ecc for every 64k....

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Tue, 01 Aug 2006, Hans Reiser wrote:

> You will want to try our compression plugin, it has an ecc for every 64k....

What kind of forward error correction would that be, and how much and
what failure patterns can it correct? URL suffices.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Edward Shishkin]

Matthias Andree wrote:
> On Tue, 01 Aug 2006, Hans Reiser wrote:
> 
> 
>>You will want to try our compression plugin, it has an ecc for every 64k....
> 
> 
> What kind of forward error correction would that be,


Actually we use checksums, not ECC. If checksum is wrong, then run
fsck - it will remove the whole disk cluster, that represent 64K of
data.


  and how much and
> what failure patterns can it correct? URL suffices.
> 

Checksum is checked before unsafe decompression (when trying to
decompress incorrect data can lead to fatal things). It can be
broken because of many reasons. The main one is tree corruption
(for example, when disk cluster became incomplete - ECC can not
help here). Perhaps such checksumming is also useful for other
things, I didnt classify the patterns..

Edward.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Edward Shishkin wrote:

> Matthias Andree wrote:
>
>> On Tue, 01 Aug 2006, Hans Reiser wrote:
>>
>>
>>> You will want to try our compression plugin, it has an ecc for every
>>> 64k....
>>
>>
>>
>> What kind of forward error correction would that be,
>
>
>
> Actually we use checksums, not ECC. If checksum is wrong, then run
> fsck - it will remove the whole disk cluster, that represent 64K of
> data.

How about we switch to ecc, which would help with bit rot not sector loss?

>
>
>  and how much and
>
>> what failure patterns can it correct? URL suffices.
>>
>
> Checksum is checked before unsafe decompression (when trying to
> decompress incorrect data can lead to fatal things). It can be
> broken because of many reasons. The main one is tree corruption
> (for example, when disk cluster became incomplete - ECC can not
> help here). Perhaps such checksumming is also useful for other
> things, I didnt classify the patterns..
>
> Edward.
>
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Edward Shishkin]

Hans Reiser wrote:
> Edward Shishkin wrote:
> 
> 
>>Matthias Andree wrote:
>>
>>
>>>On Tue, 01 Aug 2006, Hans Reiser wrote:
>>>
>>>
>>>
>>>>You will want to try our compression plugin, it has an ecc for every
>>>>64k....
>>>
>>>
>>>
>>>What kind of forward error correction would that be,
>>
>>
>>
>>Actually we use checksums, not ECC. If checksum is wrong, then run
>>fsck - it will remove the whole disk cluster, that represent 64K of
>>data.
> 
> 
> How about we switch to ecc, which would help with bit rot not sector loss?

Interesting aspect.

Yes, we can implement ECC as a special crypto transform that inflates
data. As I mentioned earlier, it is possible via translation of key
offsets with scale factor > 1.

Of course, it is better then nothing, but anyway meta-data remains
ecc-unprotected, and, hence, robustness is not increased..

Edward.

> 
>>
>> and how much and
>>
>>
>>>what failure patterns can it correct? URL suffices.
>>>
>>
>>Checksum is checked before unsafe decompression (when trying to
>>decompress incorrect data can lead to fatal things). It can be
>>broken because of many reasons. The main one is tree corruption
>>(for example, when disk cluster became incomplete - ECC can not
>>help here). Perhaps such checksumming is also useful for other
>>things, I didnt classify the patterns..
>>
>>Edward.
>>
>>
> 
> 
> 
> 

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Antonio Vargas]

On 8/4/06, Edward Shishkin <edward@namesys.com> wrote:
> Hans Reiser wrote:
> > Edward Shishkin wrote:
> >
> >
> >>Matthias Andree wrote:
> >>
> >>
> >>>On Tue, 01 Aug 2006, Hans Reiser wrote:
> >>>
> >>>
> >>>
> >>>>You will want to try our compression plugin, it has an ecc for every
> >>>>64k....
> >>>
> >>>
> >>>
> >>>What kind of forward error correction would that be,
> >>
> >>
> >>
> >>Actually we use checksums, not ECC. If checksum is wrong, then run
> >>fsck - it will remove the whole disk cluster, that represent 64K of
> >>data.
> >
> >
> > How about we switch to ecc, which would help with bit rot not sector loss?
>
> Interesting aspect.
>
> Yes, we can implement ECC as a special crypto transform that inflates
> data. As I mentioned earlier, it is possible via translation of key
> offsets with scale factor > 1.
>
> Of course, it is better then nothing, but anyway meta-data remains
> ecc-unprotected, and, hence, robustness is not increased..
>
> Edward.
>
> >
> >>
> >> and how much and
> >>
> >>
> >>>what failure patterns can it correct? URL suffices.
> >>>
> >>
> >>Checksum is checked before unsafe decompression (when trying to
> >>decompress incorrect data can lead to fatal things). It can be
> >>broken because of many reasons. The main one is tree corruption
> >>(for example, when disk cluster became incomplete - ECC can not
> >>help here). Perhaps such checksumming is also useful for other
> >>things, I didnt classify the patterns..
> >>
> >>Edward.
> >>
> >>

Would the storage + plugin subsystem support storing >1 copies of the
metadata tree?


-- 
Greetz, Antonio Vargas aka winden of network

http://network.amigascne.org/
windNOenSPAMntw@gmail.com
thesameasabove@amigascne.org

Every day, every year
you have to work
you have to study
you have to scene.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Antonio Vargas wrote:

> On 8/4/06, Edward Shishkin <edward@namesys.com> wrote:
>
>> Hans Reiser wrote:
>> > Edward Shishkin wrote:
>> >
>> >
>> >>Matthias Andree wrote:
>> >>
>> >>
>> >>>On Tue, 01 Aug 2006, Hans Reiser wrote:
>> >>>
>> >>>
>> >>>
>> >>>>You will want to try our compression plugin, it has an ecc for every
>> >>>>64k....
>> >>>
>> >>>
>> >>>
>> >>>What kind of forward error correction would that be,
>> >>
>> >>
>> >>
>> >>Actually we use checksums, not ECC. If checksum is wrong, then run
>> >>fsck - it will remove the whole disk cluster, that represent 64K of
>> >>data.
>> >
>> >
>> > How about we switch to ecc, which would help with bit rot not
>> sector loss?
>>
>> Interesting aspect.
>>
>> Yes, we can implement ECC as a special crypto transform that inflates
>> data. As I mentioned earlier, it is possible via translation of key
>> offsets with scale factor > 1.
>>
>> Of course, it is better then nothing, but anyway meta-data remains
>> ecc-unprotected, and, hence, robustness is not increased..
>>
>> Edward.
>>
>> >
>> >>
>> >> and how much and
>> >>
>> >>
>> >>>what failure patterns can it correct? URL suffices.
>> >>>
>> >>
>> >>Checksum is checked before unsafe decompression (when trying to
>> >>decompress incorrect data can lead to fatal things). It can be
>> >>broken because of many reasons. The main one is tree corruption
>> >>(for example, when disk cluster became incomplete - ECC can not
>> >>help here). Perhaps such checksumming is also useful for other
>> >>things, I didnt classify the patterns..
>> >>
>> >>Edward.
>> >>
>> >>
>
>
> Would the storage + plugin subsystem support storing >1 copies of the
> metadata tree?
>
>
I suppose....

What would be nice would be to have a plugin that when a node fails its
checksum/ecc it knows to get it from another mirror, and which generally
handles faults with a graceful understanding of its ability to get
copies from a mirror (or RAID parity calculation).

I would happily accept such a patch (subject to usual reservation of
right to complain about implementation details).

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Edward Shishkin wrote:

>
>>
>>
>> How about we switch to ecc, which would help with bit rot not sector
>> loss?
>
>
> Interesting aspect.
>
> Yes, we can implement ECC as a special crypto transform that inflates
> data. As I mentioned earlier, it is possible via translation of key
> offsets with scale factor > 1.
>
> Of course, it is better then nothing, but anyway meta-data remains
> ecc-unprotected, and, hence, robustness is not increased..
>
> Edward.

Would you prefer to do it as a node layout plugin instead, so as to get
the metadata?

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Edward Shishkin]

Hans Reiser wrote:
> Edward Shishkin wrote:
> 
> 
>>>
>>>How about we switch to ecc, which would help with bit rot not sector
>>>loss?
>>
>>
>>Interesting aspect.
>>
>>Yes, we can implement ECC as a special crypto transform that inflates
>>data. As I mentioned earlier, it is possible via translation of key
>>offsets with scale factor > 1.
>>
>>Of course, it is better then nothing, but anyway meta-data remains
>>ecc-unprotected, and, hence, robustness is not increased..
>>
>>Edward.
> 
> 
> Would you prefer to do it as a node layout plugin instead, so as to get
> the metadata?
> 

Yes, it looks like a business of node plugin, but AFAIK, you
objected against such checks: currently only bitmap nodes have
a protection (checksum); supporting ecc-signatures is more
space/cpu expensive.

Edward.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Edward Shishkin wrote:

> Hans Reiser wrote:
>
>> Edward Shishkin wrote:
>>
>>
>>>>
>>>> How about we switch to ecc, which would help with bit rot not sector
>>>> loss?
>>>
>>>
>>>
>>> Interesting aspect.
>>>
>>> Yes, we can implement ECC as a special crypto transform that inflates
>>> data. As I mentioned earlier, it is possible via translation of key
>>> offsets with scale factor > 1.
>>>
>>> Of course, it is better then nothing, but anyway meta-data remains
>>> ecc-unprotected, and, hence, robustness is not increased..
>>>
>>> Edward.
>>
>>
>>
>> Would you prefer to do it as a node layout plugin instead, so as to get
>> the metadata?
>>
>
> Yes, it looks like a business of node plugin, but AFAIK, you
> objected against such checks:

Did I really?  Well, I think that allowing users to choose whether to
checksum or not is a reasonable thing to allow them.  I personally would
skip the checksum on my computer, but others....

It could be a useful mkfs option....

> currently only bitmap nodes have
> a protection (checksum); supporting ecc-signatures is more
> space/cpu expensive.
>
> Edward.
>
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan Engelhardt]

>> Yes, it looks like a business of node plugin, but AFAIK, you
>> objected against such checks:
>
>Did I really?  Well, I think that allowing users to choose whether to
>checksum or not is a reasonable thing to allow them.  I personally would
>skip the checksum on my computer, but others....
>
>It could be a useful mkfs option....

It should preferably a runtime tunable variable, at best even
per-superblock and (overriding the sb setting), per-file.


Jan Engelhardt
-- 

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Jan Engelhardt wrote:
>>> Yes, it looks like a business of node plugin, but AFAIK, you
>>> objected against such checks:
>> Did I really?  Well, I think that allowing users to choose whether to
>> checksum or not is a reasonable thing to allow them.  I personally would
>> skip the checksum on my computer, but others....
>>
>> It could be a useful mkfs option....
> 
> It should preferably a runtime tunable variable, at best even
> per-superblock and (overriding the sb setting), per-file.

Sounds almost exactly like a plugin.  And yes, that would be the way to 
do it, especially considering some files will already have internal 
consistency checking -- just as we should allow direct disk IO to some 
files (no journaling) when the files in question are databases that do 
their own journaling.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

[stripping Cc: list]

On Thu, 03 Aug 2006, Edward Shishkin wrote:

> >What kind of forward error correction would that be,
> 
> Actually we use checksums, not ECC. If checksum is wrong, then run
> fsck - it will remove the whole disk cluster, that represent 64K of
> data.

Well, that's quite a difference...

> Checksum is checked before unsafe decompression (when trying to
> decompress incorrect data can lead to fatal things).

Is this sufficient? How about corruptions that lead to the same checksum
and can then confuse the decompressor? Is the decompressor safe in that
it does not scribble over memory it has not allocated?

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Edward Shishkin]

Matthias Andree wrote:
> [stripping Cc: list]
> 
> On Thu, 03 Aug 2006, Edward Shishkin wrote:
> 
> 
>>>What kind of forward error correction would that be,
>>
>>Actually we use checksums, not ECC. If checksum is wrong, then run
>>fsck - it will remove the whole disk cluster, that represent 64K of
>>data.
> 
> 
> Well, that's quite a difference...
> 
> 
>>Checksum is checked before unsafe decompression (when trying to
>>decompress incorrect data can lead to fatal things).
> 
> 
> Is this sufficient? How about corruptions that lead to the same checksum
> and can then confuse the decompressor? 


It is a multiplication of two unlikely events: fs corruption
and 32-hash collision. Paranoid people can assign zlib-based
transform plugin: afaik everything is safe there.


Is the decompressor safe in that
> it does not scribble over memory it has not allocated?
> 

yes

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Ric Wheeler wrote:
> Alan Cox wrote:
>> Ar Maw, 2006-08-01 am 16:52 +0200, ysgrifennodd Adrian Ulrich:
>>
>>> WriteCache, Mirroring between 2 Datacenters, snapshotting.. etc..
>>> you don't need your filesystem beeing super-robust against bad sectors
>>> and such stuff because:
>>
>>
>> You do it turns out. Its becoming an issue more and more that the sheer
>> amount of storage means that the undetected error rate from disks,
>> hosts, memory, cables and everything else is rising.

> Most people use absolutely giant disks in laptops and desktop systems 
> (300GB & 500GB are common, 750GB on the way). File systems need to be as 
> robust as possible for users of these systems as people are commonly 
> storing personal "critical" data like photos mostly on these unprotected 
> drives.

Their loss.  Robust FS is good, but really, if you aren't doing backup, 
you are going to lose data.  End of story.

> Even for the high end users, array based mirroring and so on can only do 
> so much to protect you.
> 
> Mirroring a corrupt file system to a remote data center will mirror your 
> corruption.

Assuming it's undetected.  Why would it be undetected?

> Rolling back to a snapshot typically only happens when you notice a 
> corruption which can go undetected for quite a while, so even that will 
> benefit from having "reliability" baked into the file system (i.e., it 
> should grumble about corruption to let you know that you need to roll 
> back or fsck or whatever).

Yes, the filesystem should complain about corruption.  So should the 
block layer -- if you don't trust the FS, use a checksum at the block 
layer.  So should...

There are just so many other, better places to do this than the FS.  The 
FS should complain, yes, but if the disk is bad, there's going to be 
corruption.

> An even larger issue is that our tools, like fsck, which are used to 
> uncover these silent corruptions need to scale up to the point that they 
> can uncover issues in minutes instead of days.  A lot of the focus at 
> the file system workshop was around how to dramatically reduce the 
> repair time of file systems.

That would be interesting.  I know from experience that fsck.reiser4 is 
amazing.  Blew away my data with something akin to an rm -rf, and fsck 
fixed it.  Tons of crashing/instability in the early days, but only once 
-- before they even had a version instead of a date, I think -- did I 
ever have a case where fsck couldn't fix it.

So I guess the next step would be to make fsck faster.  Someone 
mentioned a fsck that repairs the FS in the background?

> In a way, having super reliable storage hardware is only as good as the 
> file system layer on top of it - reliability needs to be baked into the 
> entire IO system stack...

That bit makes no sense.  If you have super reliable storage failure 
(never dies), and your FS is also reliable (never dies unless hardware 
does, but may go bat-shit insane when hardware dies), then you've got a 
super reliable system.

You're right, running Linux's HFS+ or NTFS write support is generally a 
bad idea, no matter how reliable your hardware is.  But this discussion 
was not about whether an FS is stable, but how well an FS survives 
hardware corruption.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Tue, 01 Aug 2006, Ric Wheeler wrote:

> Mirroring a corrupt file system to a remote data center will mirror your 
> corruption.
> 
> Rolling back to a snapshot typically only happens when you notice a 
> corruption which can go undetected for quite a while, so even that will 
> benefit from having "reliability" baked into the file system (i.e., it 
> should grumble about corruption to let you know that you need to roll 
> back or fsck or whatever).
> 
> An even larger issue is that our tools, like fsck, which are used to 
> uncover these silent corruptions need to scale up to the point that they 
> can uncover issues in minutes instead of days.  A lot of the focus at 
> the file system workshop was around how to dramatically reduce the 
> repair time of file systems.

Which makes me wonder if backup systems shouldn't help with this. If
they are reading the whole file anyways, they can easily compute strong
checksums as they go, and record them for later use, and check so many
percent of unchanged files every day to complain about corruptions.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Theodore Tso]

On Thu, Aug 03, 2006 at 04:03:07PM +0200, Matthias Andree wrote:
> On Tue, 01 Aug 2006, Ric Wheeler wrote:
> 
> > Mirroring a corrupt file system to a remote data center will mirror your 
> > corruption.
> > 
> 
> Which makes me wonder if backup systems shouldn't help with this. If
> they are reading the whole file anyways, they can easily compute strong
> checksums as they go, and record them for later use, and check so many
> percent of unchanged files every day to complain about corruptions.

They absolutely should do this sort of thing.

Also sounds like yet another option that could be added to rsync.
(Only half-joking.  :-)

						- Ted

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Horst H. von Brand wrote:
> Bernd Schubert <bernd-schubert@gmx.de> wrote:

>> While filesystem speed is nice, it also would be great if reiser4.x would be 
>> very robust against any kind of hardware failures.
> 
> Can't have both.

Why not?  I mean, other than TANSTAAFL, is there a technical reason for 
them being mutually exclusive?  I suspect it's more "we haven't found a 
way yet..."

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Pavel Machek]

On Tue 01-08-06 11:57:10, David Masover wrote:
> Horst H. von Brand wrote:
> >Bernd Schubert <bernd-schubert@gmx.de> wrote:
> 
> >>While filesystem speed is nice, it also would be great 
> >>if reiser4.x would be very robust against any kind of 
> >>hardware failures.
> >
> >Can't have both.
> 
> Why not?  I mean, other than TANSTAAFL, is there a 
> technical reason for them being mutually exclusive?  I 
> suspect it's more "we haven't found a way yet..."

What does the acronym mean?

Yes, I'm afraid redundancy/checksums kill write speed, and you need
that for robustness...

You could have filesystem that can be tuned for reliability and tuned
for speed... but you can't have both in one filesystem instance.
-- 
Thanks for all the (sleeping) penguins.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Pavel Machek wrote:
> On Tue 01-08-06 11:57:10, David Masover wrote:
>> Horst H. von Brand wrote:
>>> Bernd Schubert <bernd-schubert@gmx.de> wrote:
>>>> While filesystem speed is nice, it also would be great 
>>>> if reiser4.x would be very robust against any kind of 
>>>> hardware failures.
>>> Can't have both.
>> Why not?  I mean, other than TANSTAAFL, is there a 
>> technical reason for them being mutually exclusive?  I 
>> suspect it's more "we haven't found a way yet..."
> 
> What does the acronym mean?

There Ain't No Such Thing As A Free Lunch.

> Yes, I'm afraid redundancy/checksums kill write speed, and you need
> that for robustness...

Not necessarily -- if you do it on flush, and store it near the data it 
relates to, you can expect a similar impact to compression, except that 
due to slow disks, the compression can actually speed things up 2x, 
whereas checksums should be some insignificant amount slower than 1x.

Redundancy, sure, but checksums should be easy, and I don't see what 
robustness (abilities of fsck) has to do with it.

> You could have filesystem that can be tuned for reliability and tuned
> for speed... but you can't have both in one filesystem instance.

That's an example of TANSTAAFL, if it's true.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Pavel Machek wrote:

>
>
>Yes, I'm afraid redundancy/checksums kill write speed,
>
they kill write speed to cache, but not to disk....  our compression
plugin is faster than the uncompressed plugin.....

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Pavel Machek]

On Wed 2006-08-09 02:37:45, Hans Reiser wrote:
> Pavel Machek wrote:
> 
> >
> >
> >Yes, I'm afraid redundancy/checksums kill write speed,
> >
> they kill write speed to cache, but not to disk....  our compression
> plugin is faster than the uncompressed plugin.....

Yes, you can get clever. But your compression plugin also means that
single bit error means whole block is lost, so there _is_ speed
vs. stability-against-hw-problems.

But you are right that compression will catch same class of errors
checksums will, so that it is probably good thing w.r.t. stability.

								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Pavel Machek wrote:

>On Wed 2006-08-09 02:37:45, Hans Reiser wrote:
>  
>
>>Pavel Machek wrote:
>>
>>    
>>
>>>Yes, I'm afraid redundancy/checksums kill write speed,
>>>
>>>      
>>>
>>they kill write speed to cache, but not to disk....  our compression
>>plugin is faster than the uncompressed plugin.....
>>    
>>
>
>Yes, you can get clever. But your compression plugin also means that
>single bit error means whole block is lost, so there _is_ speed
>vs. stability-against-hw-problems.
>
>But you are right that compression will catch same class of errors
>checksums will, so that it is probably good thing w.r.t. stability.
>
>								Pavel
>  
>
So we need to use ecc not checksums if we want to increase
reliability.   Edward, can you comment in more detail regarding your
views and the performance issues for ecc that you see?

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Hans Reiser wrote:
> Pavel Machek wrote:
> 
>>
>> Yes, I'm afraid redundancy/checksums kill write speed,
>>
> they kill write speed to cache, but not to disk....  our compression
> plugin is faster than the uncompressed plugin.....

Regarding cache, do we do any sort of consistency checking for RAM, or 
do we leave that to some of the stranger kernel patches -- or just an 
occasional memtest?

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Alan Cox]

Ar Llu, 2006-07-31 am 12:17 -0700, ysgrifennodd Clay Barnes:
> Of course, if ext3 were proven to be more robust against failures, I bet
> the reiser team would be very interested in all the forensic data you
> can offer, since, from what I've seen, they are always trying to make
> reiser as good as possible---in speed, flexability, *and* robustness.

Its well accepted that reiserfs3 has some robustness problems in the
face of physical media errors. The structure of the file system and the
tree basis make it very hard to avoid such problems. XFS appears to have
managed to achieve both robustness and better data structures. 

How reiser4 compares I've no idea. 

Alan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Clay Barnes]

On 20:42 Mon 31 Jul     , Alan Cox wrote:
> Ar Llu, 2006-07-31 am 12:17 -0700, ysgrifennodd Clay Barnes:
> > Of course, if ext3 were proven to be more robust against failures, I bet
> > the reiser team would be very interested in all the forensic data you
> > can offer, since, from what I've seen, they are always trying to make
> > reiser as good as possible---in speed, flexability, *and* robustness.
> 
> Its well accepted that reiserfs3 has some robustness problems in the
> face of physical media errors. The structure of the file system and the
> tree basis make it very hard to avoid such problems. XFS appears to have
> managed to achieve both robustness and better data structures. 

Yes, that is true, and I think that's a big motivator for the reiser
team to get reiser4 in a place where people can't say that.  I suspect
that they know that reiserfs's shortcomings in that respect are probably
the biggest deterrent to using that fs, and they'll do everything they
can to prevent such a problem in reiser4.  That's pure conjecture based
on the stuff I see on the list, so if I'm wrong, reiser people, please
correct me.

--Clay

> 
> How reiser4 compares I've no idea. 
> 
> Alan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Gregory Maxwell]

On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> Its well accepted that reiserfs3 has some robustness problems in the
> face of physical media errors. The structure of the file system and the
> tree basis make it very hard to avoid such problems. XFS appears to have
> managed to achieve both robustness and better data structures.
>
> How reiser4 compares I've no idea.

Citation?

I ask because your clam differs from the only detailed research that
I'm aware of on the subject[1]. In figure 2 of the iron filesystems
paper that Ext3 is show to ignore a great number of data-loss inducing
failure conditions that Reiser3 detects an panics under.

Are you sure that you aren't commenting on cases where Reiser3 alerts
the user to a critical data condition (via a panic) which leads to a
trouble report while ext3 ignores the problem which suppresses the
trouble report from the user?

*1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Alan Cox]

Ar Llu, 2006-07-31 am 17:00 -0400, ysgrifennodd Gregory Maxwell:
> On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > Its well accepted that reiserfs3 has some robustness problems in the
> > face of physical media errors. The structure of the file system and the
> > tree basis make it very hard to avoid such problems. XFS appears to have
> > managed to achieve both robustness and better data structures.
> >
> > How reiser4 compares I've no idea.
> 
> Citation?

Two sources, the cases I've looked at myself when IDE maintainer and
also comments Hans made when we met at UKUUG a few years ago. Generally
speaking on an IDE failure that lost chunks of disk the ext2/ext3 users
got most of their data back. The reiserfs ones sometimes got it back and
sometimes got catastrophic failure.

The ext3 fsck is extremely effective in the face of serious errors. Some
of that is clever code that knows about things like rewriting inode
blocks to force reallocation of failed metadata by the drive but the
majority of it is simply because you *know* where inode X is on the disk
rather than having to deal with data structures and walk them.

That's a tradeoff with the reiser performance with small files and until
recently the reiser large directory performance. Which is right is
another question altogether. Its also something I think XFS demonstrates
can be done better so doesn't invalidate the theory behind reiserfs just
the rev 3 implementation.

> Are you sure that you aren't commenting on cases where Reiser3 alerts
> the user to a critical data condition (via a panic) which leads to a
> trouble report while ext3 ignores the problem which suppresses the
> trouble report from the user?

man mount

Ext3 is configurable, and has been for years via the errors= option.

Alan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Alan Cox wrote:
> Ar Llu, 2006-07-31 am 17:00 -0400, ysgrifennodd Gregory Maxwell:

>> Are you sure that you aren't commenting on cases where Reiser3 alerts
>> the user to a critical data condition (via a panic) which leads to a
>> trouble report while ext3 ignores the problem which suppresses the
>> trouble report from the user?
> 
> man mount
> 
> Ext3 is configurable, and has been for years via the errors= option.

Sure, but I think the suggestion is that the reason we generally see 
more ReiserFS complaints than ext3 complaints might be because of the 
default level of errors logged.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff V. Merkey]

Gregory Maxwell wrote:

> On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>
>> Its well accepted that reiserfs3 has some robustness problems in the
>> face of physical media errors. The structure of the file system and the
>> tree basis make it very hard to avoid such problems. XFS appears to have
>> managed to achieve both robustness and better data structures.
>>
>> How reiser4 compares I've no idea.
>
>
> Citation?
>
> I ask because your clam differs from the only detailed research that
> I'm aware of on the subject[1]. In figure 2 of the iron filesystems
> paper that Ext3 is show to ignore a great number of data-loss inducing
> failure conditions that Reiser3 detects an panics under.
>
> Are you sure that you aren't commenting on cases where Reiser3 alerts
> the user to a critical data condition (via a panic) which leads to a
> trouble report while ext3 ignores the problem which suppresses the
> trouble report from the user?
>
> *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf

Hi Gregory, Wikimedia Foundation and LKML? 

How's Wikimania going. :-)

What he says is correct.  I have seen some serious issues with reiserfs 
in terms of stability and
data corruption.  Resier is however FASTER, but the statement is has 
robustness issues is accurate.
I was using reiserfs but we opted to make EXT3 the default for Solera 
appliances, even when using Suse 10
due to issues I have seen with data corruption and hard hangs on RAID 0 
read/write sector errors.  I have
stopped using it for local drives and based everything on EXT3.  Not to 
say it won't get there eventually, but
file systems have to endure a lot of time in the field and deployment 
befor they are ready for prime time.

The Wikimedia appliances use Wolf Mountain, and I've tested it for about 
4 months with few problems, but
I only use it for hosting the Cherokee Langauge Wikipedia.  It's 
performance is several magnitudes better
than either EXT3 or ReiserFS.  Despite this, for vertical wiki servers, 
its ok to go out with, folks can specifiy
whether they want appliances with EXT3, Reiser, or WMFS, but iit's a 
long way from being "cooked"
completely, though it does scale to 1 exabyte FS images. 

Reiser does have issues still, and I hestitate to standardize on it 
until I stop seeing reports from the field about
corruption and failover issues.

Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff V. Merkey]

Jeff V. Merkey wrote:

> Gregory Maxwell wrote:
>
>> On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>>
>>> Its well accepted that reiserfs3 has some robustness problems in the
>>> face of physical media errors. The structure of the file system and the
>>> tree basis make it very hard to avoid such problems. XFS appears to 
>>> have
>>> managed to achieve both robustness and better data structures.
>>>
>>> How reiser4 compares I've no idea.
>>
>>
>>
>> Citation?
>>
>> I ask because your clam differs from the only detailed research that
>> I'm aware of on the subject[1]. In figure 2 of the iron filesystems
>> paper that Ext3 is show to ignore a great number of data-loss inducing
>> failure conditions that Reiser3 detects an panics under.
>>
>> Are you sure that you aren't commenting on cases where Reiser3 alerts
>> the user to a critical data condition (via a panic) which leads to a
>> trouble report while ext3 ignores the problem which suppresses the
>> trouble report from the user?
>>
>> *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
>
>
> Hi Gregory, Wikimedia Foundation and LKML?
> How's Wikimania going. :-)
>
> What he says is correct.  I have seen some serious issues with 
> reiserfs in terms of stability and
> data corruption.  Resier is however FASTER, but the statement is has 
> robustness issues is accurate.
> I was using reiserfs but we opted to make EXT3 the default for Solera 
> appliances, even when using Suse 10
> due to issues I have seen with data corruption and hard hangs on RAID 
> 0 read/write sector errors.  I have
> stopped using it for local drives and based everything on EXT3.  Not 
> to say it won't get there eventually, but
> file systems have to endure a lot of time in the field and deployment 
> befor they are ready for prime time.
>

Correction,

That's "MediWiki" appliances.   Two many transposed acronyms...

www.wolfmountaingroup.com

:-)

Jeff

> The Wikimedia appliances use Wolf Mountain, and I've tested it for 
> about 4 months with few problems, but
> I only use it for hosting the Cherokee Langauge Wikipedia.  It's 
> performance is several magnitudes better
> than either EXT3 or ReiserFS.  Despite this, for vertical wiki 
> servers, its ok to go out with, folks can specifiy
> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a 
> long way from being "cooked"
> completely, though it does scale to 1 exabyte FS images.
> Reiser does have issues still, and I hestitate to standardize on it 
> until I stop seeing reports from the field about
> corruption and failover issues.
>
> Jeff
>
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff V. Merkey]

>
> Correction,
>
> That's "MediWiki" appliances.   Two many transposed acronyms...
>
> www.wolfmountaingroup.com
>
> :-)
>
> Jeff
>
And WMFS does not belong to the WolfMountainGroup any longer.  It has 
been acquired by another company, so you won't see any info
about it on the website.  It's has been rolled into another company.  I 
can bundle it with appliances but it is no longer the property of
Wolf Mountain Group.  WMG is a MediaWiki/Wikipedia Appliance company 
that does Machine translations of Wikipedia into several
Native American Languages.  The Translator is Linux and Windows based, 
but WMFS has a new home now. 

Just to clarify. 

Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Nate Diller]

On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote:
> Gregory Maxwell wrote:
>
> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> >
> >> Its well accepted that reiserfs3 has some robustness problems in the
> >> face of physical media errors. The structure of the file system and the
> >> tree basis make it very hard to avoid such problems. XFS appears to have
> >> managed to achieve both robustness and better data structures.
> >>
> >> How reiser4 compares I've no idea.
> >
> >
> > Citation?
> >
> > I ask because your clam differs from the only detailed research that
> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems
> > paper that Ext3 is show to ignore a great number of data-loss inducing
> > failure conditions that Reiser3 detects an panics under.
> >
> > Are you sure that you aren't commenting on cases where Reiser3 alerts
> > the user to a critical data condition (via a panic) which leads to a
> > trouble report while ext3 ignores the problem which suppresses the
> > trouble report from the user?
> >
> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
>
> Hi Gregory, Wikimedia Foundation and LKML?
>
> How's Wikimania going. :-)
>
> What he says is correct.  I have seen some serious issues with reiserfs
> in terms of stability and
> data corruption.  Resier is however FASTER, but the statement is has
> robustness issues is accurate.
> I was using reiserfs but we opted to make EXT3 the default for Solera
> appliances, even when using Suse 10
> due to issues I have seen with data corruption and hard hangs on RAID 0
> read/write sector errors.  I have
> stopped using it for local drives and based everything on EXT3.  Not to
> say it won't get there eventually, but
> file systems have to endure a lot of time in the field and deployment
> befor they are ready for prime time.
>
> The Wikimedia appliances use Wolf Mountain, and I've tested it for about
> 4 months with few problems, but
> I only use it for hosting the Cherokee Langauge Wikipedia.  It's
> performance is several magnitudes better
> than either EXT3 or ReiserFS.  Despite this, for vertical wiki servers,
> its ok to go out with, folks can specifiy
> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a
> long way from being "cooked"
> completely, though it does scale to 1 exabyte FS images.

i've seen you mention the Wolf Mountain FS in other emails, but google
isn't telling me a lot about it.  Do you have a whitepaper?  are there
any published benchmark results?  what sort of workloads do you
benchmark?

NATE

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff V. Merkey]

Nate Diller wrote:

> On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote:
>
>> Gregory Maxwell wrote:
>>
>> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> >
>> >> Its well accepted that reiserfs3 has some robustness problems in the
>> >> face of physical media errors. The structure of the file system 
>> and the
>> >> tree basis make it very hard to avoid such problems. XFS appears 
>> to have
>> >> managed to achieve both robustness and better data structures.
>> >>
>> >> How reiser4 compares I've no idea.
>> >
>> >
>> > Citation?
>> >
>> > I ask because your clam differs from the only detailed research that
>> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems
>> > paper that Ext3 is show to ignore a great number of data-loss inducing
>> > failure conditions that Reiser3 detects an panics under.
>> >
>> > Are you sure that you aren't commenting on cases where Reiser3 alerts
>> > the user to a critical data condition (via a panic) which leads to a
>> > trouble report while ext3 ignores the problem which suppresses the
>> > trouble report from the user?
>> >
>> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
>>
>> Hi Gregory, Wikimedia Foundation and LKML?
>>
>> How's Wikimania going. :-)
>>
>> What he says is correct.  I have seen some serious issues with reiserfs
>> in terms of stability and
>> data corruption.  Resier is however FASTER, but the statement is has
>> robustness issues is accurate.
>> I was using reiserfs but we opted to make EXT3 the default for Solera
>> appliances, even when using Suse 10
>> due to issues I have seen with data corruption and hard hangs on RAID 0
>> read/write sector errors.  I have
>> stopped using it for local drives and based everything on EXT3.  Not to
>> say it won't get there eventually, but
>> file systems have to endure a lot of time in the field and deployment
>> befor they are ready for prime time.
>>
>> The Wikimedia appliances use Wolf Mountain, and I've tested it for about
>> 4 months with few problems, but
>> I only use it for hosting the Cherokee Langauge Wikipedia.  It's
>> performance is several magnitudes better
>> than either EXT3 or ReiserFS.  Despite this, for vertical wiki servers,
>> its ok to go out with, folks can specifiy
>> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a
>> long way from being "cooked"
>> completely, though it does scale to 1 exabyte FS images.
>
>
> i've seen you mention the Wolf Mountain FS in other emails, but google
> isn't telling me a lot about it.  Do you have a whitepaper?  are there
> any published benchmark results?  what sort of workloads do you
> benchmark?
>
> NATE
>
Wikipedia is the app for now.  I have not done any benchmarks on the FS 
side, just the capture side, and its been transferred to
another entity.  I have no idea what they are naming it to, but I expect 
you may hear about it soon.  One of the incarnations
of it is Solera's DSFS which can be reviewed here:

www.soleranetworks.com

I can sustain 850 MB/S throughput from user space with it -- about 5 x 
any other FS.  On some hardware, I've broken
the 1.25 GB/S (gigabyte/second) windows with it.

Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Nate Diller]

On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote:
> Nate Diller wrote:
>
> > On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote:
> >
> >> Gregory Maxwell wrote:
> >>
> >> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> >> >
> >> >> Its well accepted that reiserfs3 has some robustness problems in the
> >> >> face of physical media errors. The structure of the file system
> >> and the
> >> >> tree basis make it very hard to avoid such problems. XFS appears
> >> to have
> >> >> managed to achieve both robustness and better data structures.
> >> >>
> >> >> How reiser4 compares I've no idea.
> >> >
> >> >
> >> > Citation?
> >> >
> >> > I ask because your clam differs from the only detailed research that
> >> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems
> >> > paper that Ext3 is show to ignore a great number of data-loss inducing
> >> > failure conditions that Reiser3 detects an panics under.
> >> >
> >> > Are you sure that you aren't commenting on cases where Reiser3 alerts
> >> > the user to a critical data condition (via a panic) which leads to a
> >> > trouble report while ext3 ignores the problem which suppresses the
> >> > trouble report from the user?
> >> >
> >> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
> >>
> >> Hi Gregory, Wikimedia Foundation and LKML?
> >>
> >> How's Wikimania going. :-)
> >>
> >> What he says is correct.  I have seen some serious issues with reiserfs
> >> in terms of stability and
> >> data corruption.  Resier is however FASTER, but the statement is has
> >> robustness issues is accurate.
> >> I was using reiserfs but we opted to make EXT3 the default for Solera
> >> appliances, even when using Suse 10
> >> due to issues I have seen with data corruption and hard hangs on RAID 0
> >> read/write sector errors.  I have
> >> stopped using it for local drives and based everything on EXT3.  Not to
> >> say it won't get there eventually, but
> >> file systems have to endure a lot of time in the field and deployment
> >> befor they are ready for prime time.
> >>
> >> The Wikimedia appliances use Wolf Mountain, and I've tested it for about
> >> 4 months with few problems, but
> >> I only use it for hosting the Cherokee Langauge Wikipedia.  It's
> >> performance is several magnitudes better
> >> than either EXT3 or ReiserFS.  Despite this, for vertical wiki servers,
> >> its ok to go out with, folks can specifiy
> >> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a
> >> long way from being "cooked"
> >> completely, though it does scale to 1 exabyte FS images.
> >
> >
> > i've seen you mention the Wolf Mountain FS in other emails, but google
> > isn't telling me a lot about it.  Do you have a whitepaper?  are there
> > any published benchmark results?  what sort of workloads do you
> > benchmark?
> >
> > NATE
> >
> Wikipedia is the app for now.  I have not done any benchmarks on the FS
> side, just the capture side, and its been transferred to
> another entity.  I have no idea what they are naming it to, but I expect
> you may hear about it soon.  One of the incarnations
> of it is Solera's DSFS which can be reviewed here:
>
> www.soleranetworks.com

so this is a single stream, write only? ...

> I can sustain 850 MB/S throughput from user space with it -- about 5 x
> any other FS.  On some hardware, I've broken
> the 1.25 GB/S (gigabyte/second) windows with it.

and you're saying it scales to much higher multi-spindle
single-machine throughput.  cool.

i'd love to see a whitepaper, or failing that, have an off-list
discussion of your approach and the various kernel limitations you ran
up against in testing.  i don't suppose they invited you to the Kernel
Summit to talk about it, heh.

NATE

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeffrey V. Merkey]

Nate Diller wrote:

> On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote:
>
>> Nate Diller wrote:
>>
>> > On 7/31/06, Jeff V. Merkey <jmerkey@wolfmountaingroup.com> wrote:
>> >
>> >> Gregory Maxwell wrote:
>> >>
>> >> > On 7/31/06, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> >> >
>> >> >> Its well accepted that reiserfs3 has some robustness problems 
>> in the
>> >> >> face of physical media errors. The structure of the file system
>> >> and the
>> >> >> tree basis make it very hard to avoid such problems. XFS appears
>> >> to have
>> >> >> managed to achieve both robustness and better data structures.
>> >> >>
>> >> >> How reiser4 compares I've no idea.
>> >> >
>> >> >
>> >> > Citation?
>> >> >
>> >> > I ask because your clam differs from the only detailed research 
>> that
>> >> > I'm aware of on the subject[1]. In figure 2 of the iron filesystems
>> >> > paper that Ext3 is show to ignore a great number of data-loss 
>> inducing
>> >> > failure conditions that Reiser3 detects an panics under.
>> >> >
>> >> > Are you sure that you aren't commenting on cases where Reiser3 
>> alerts
>> >> > the user to a critical data condition (via a panic) which leads 
>> to a
>> >> > trouble report while ext3 ignores the problem which suppresses the
>> >> > trouble report from the user?
>> >> >
>> >> > *1) http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
>> >>
>> >> Hi Gregory, Wikimedia Foundation and LKML?
>> >>
>> >> How's Wikimania going. :-)
>> >>
>> >> What he says is correct.  I have seen some serious issues with 
>> reiserfs
>> >> in terms of stability and
>> >> data corruption.  Resier is however FASTER, but the statement is has
>> >> robustness issues is accurate.
>> >> I was using reiserfs but we opted to make EXT3 the default for Solera
>> >> appliances, even when using Suse 10
>> >> due to issues I have seen with data corruption and hard hangs on 
>> RAID 0
>> >> read/write sector errors.  I have
>> >> stopped using it for local drives and based everything on EXT3.  
>> Not to
>> >> say it won't get there eventually, but
>> >> file systems have to endure a lot of time in the field and deployment
>> >> befor they are ready for prime time.
>> >>
>> >> The Wikimedia appliances use Wolf Mountain, and I've tested it for 
>> about
>> >> 4 months with few problems, but
>> >> I only use it for hosting the Cherokee Langauge Wikipedia.  It's
>> >> performance is several magnitudes better
>> >> than either EXT3 or ReiserFS.  Despite this, for vertical wiki 
>> servers,
>> >> its ok to go out with, folks can specifiy
>> >> whether they want appliances with EXT3, Reiser, or WMFS, but iit's a
>> >> long way from being "cooked"
>> >> completely, though it does scale to 1 exabyte FS images.
>> >
>> >
>> > i've seen you mention the Wolf Mountain FS in other emails, but google
>> > isn't telling me a lot about it.  Do you have a whitepaper?  are there
>> > any published benchmark results?  what sort of workloads do you
>> > benchmark?
>> >
>> > NATE
>> >
>> Wikipedia is the app for now.  I have not done any benchmarks on the FS
>> side, just the capture side, and its been transferred to
>> another entity.  I have no idea what they are naming it to, but I expect
>> you may hear about it soon.  One of the incarnations
>> of it is Solera's DSFS which can be reviewed here:
>>
>> www.soleranetworks.com
>
>
> so this is a single stream, write only? ...
>
>> I can sustain 850 MB/S throughput from user space with it -- about 5 x
>> any other FS.  On some hardware, I've broken
>> the 1.25 GB/S (gigabyte/second) windows with it.
>
>
> and you're saying it scales to much higher multi-spindle
> single-machine throughput.  cool.
>
> i'd love to see a whitepaper, or failing that, have an off-list
> discussion of your approach and the various kernel limitations you ran
> up against in testing.  i don't suppose they invited you to the Kernel
> Summit to talk about it, heh.
>
> NATE
>
The patents have been filed for over a year, and will publish in several 
weeks at uspto.gov -- that's the only acclaim I care for --
one that results in value for the industry and more patent protection 
for Linux and profits for folks.  No, I have not been invited
to the summit, probably because of the lawsuit I filed against some 
folks who were threatening my family -- Peter Anvin booted
me off Kernel.org after allowing folks to pinch my code and copy my bash 
history files all over the internet, and several folks
have stiffed me.  I could care less.  I keep creating cool technology, 
make tons of money off of it, and I have cultivated an
excellent relationship with the Wikimedia Foundation, and I am now the 
principal contributor on the Cherokee Wikipedia.  Wales
even deleted the article folks had used to smear me and made folks 
rewrite it.  Wales is a very nice man and good dude.

I am content to contribute to Linux from a business viewpoint, and if 
the treatment I received from Anvin is par for kernel.org accounts,
I don't care for one -- IP addresses are rather cheap on the 
internet.    I was and have remained loyal to Linux through it all.

I am appreciative of your interest.  Check uspto.gov in next few weeks 
for published applications, it's all described there, distributed
architecture and all. 

All my Wikilove.

Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Jan-Benedict Glaw schrieb am 2006-07-31:

> > Massive hardware problems don't count. ext2/ext3 doesn't look much better in
> > such cases. I had a machine with RAM gone bad (no ECC - I wonder what
> 
> They do! Very much, actually. These happen In Real Life, so I have to
> pay attention to them. Once you're in setups with > 10000 machines,
> everything counts. At some certain point, you can even use HDD's
> temperature sensors in old machines to diagnose dead fans.
> 
> Everything that eases recovery for whatever reason is something you
> have to pay attention to. The simplicity of ext{2,3} is something I
> really fail to find proper words for. As well as the really good fsck.
> Once seen a SIGSEGV'ing fsck, you really don't want to go there.

The point is: If you've written data with broken hardware (RAM, bus,
controllers - loads of them, CPU), what is on your disks is
untrustworthy anyways, and fsck isn't going to repair your gzip file
where every 64th bit has become a 1 or when the battery-backed write
cache threw 60 MB down the drain...

Of course, an fsck that crashes is unbearable, but that doesn't apply to
"broken hardware" failures. You need backups with a few generations to
avoid massively losing data.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Łukasz Mierzwa]

Dnia Mon, 31 Jul 2006 19:32:39 +0200, Jan-Benedict Glaw  
<jbglaw@lug-owl.de> napisał:

> On Mon, 2006-07-31 18:44:33 +0200, Rudy Zijlstra <rudy@edsons.demon.nl>  
> wrote:
>> On Mon, 31 Jul 2006, Jan-Benedict Glaw wrote:
>> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich
>> > <reiser4@blinkenlights.ch> wrote:
>> > > A colleague of mine happened to create a ~300gb filesystem and  
>> started
>> > > to migrate Mailboxes (Maildir-style format = many small files  
>> (1-3kb))
>> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
>> >
>> > So preparation work wasn't done.
>>
>> Of course you are right. Preparation work was not fully done. And using
>> ext1 would also have been possible. I suspect you are still using ext1,
>> cause with proper preparation it is perfectly usable.
>
> Oh, and before people start laughing at me, here are some personal or
> friend's experiences with different filesystems:
>
>   * reiser3: A HDD containing a reiser3 filesystem was tried to be
>     booted on a machine that fucked up DMA writes. Fortunately, it
>     crashed really soon (right after going for read-write.)  After
>     rebooting the HDD on a sane PeeCee, it refused to boot. Starting
>     off some rescue system showed an _empty_ root filesystem.
>
>   * A friend's XFS data partition (portable USB/FireWire HDD) once
>     crashed due to being hot-unplugged off the USB.  The in-kernel XFS
>     driver refused to mount that thing again, and the tools also
>     refused to fix any errors. (Don't ask, no details at my hands...)
>
>   * JFS just always worked for me. Though I've never ever had a broken
>     HDD where it (or it's tools) could have shown how well-done they
>     were, so from a crash-recovery point of view, it's untested.
>
>   * Being a regular ext3 user, I had lots of broken HDDs containing
>     ext3 filesystems. For every single case, it has been easy fixing
>     the filesystem after cloning. Just _once_, fsck wasn't able to fix
>     something, so I did it manually with some disk editor. This worked
>     well because the on-disk data structures are actually as simple as
>     they are.

Is this some kind of "who lost more files on what fs" competition? What's  
the prize?
Topic subject sugests that it should cover something else. Please, let's  
get back on track.
First of all, it's about reiser4 so can't we forget about other  
filesystem's unless it's got something to do with reiser4 merge?

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Dan Oglesby]

On Mon, 2006-07-31 at 18:22 +0200, Jan-Benedict Glaw wrote:
> On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote:
> > A colleague of mine happened to create a ~300gb filesystem and started
> > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> 
> So preparation work wasn't done.
> 
> MfG, JBG
> 

I'd agree with that statement.  The wrong filesystem was chosen at the
beginning of the project.

As someone who is currently planning to migrate ~100GB of stored mail to
the Maildirs format, it was pretty clear early on that EXT3 would not
cut it (from past and current experiences), and not just for the sake of
calculating inodes.

--Dan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1454 bytes --]

On Mon, 2006-07-31 11:47:00 -0500, Dan Oglesby <doglesby@teleformix.com> wrote:
> On Mon, 2006-07-31 at 18:22 +0200, Jan-Benedict Glaw wrote:
> > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote:
> > > A colleague of mine happened to create a ~300gb filesystem and started
> > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> > So preparation work wasn't done.
> 
> As someone who is currently planning to migrate ~100GB of stored mail to
> the Maildirs format, it was pretty clear early on that EXT3 would not
> cut it (from past and current experiences), and not just for the sake of
> calculating inodes.

Uh?  Where did you face a problem there?

With maildir, you shouldn't face any problems IMO. Even users with
zillions of mails should work properly with the dir_index stuff:

	tune2fs -O dir_index /dev/hdXX

or alternatively (to start that for already existing directories):

	e2fsck -fD /dev/hdXX


Of course, you'll always face a problem with lots of files in one
directory at getdents() time (eg. opendir()/readdir()/closedir()), but
this is a common limit for all filesystems.

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
Signature of:           Alles wird gut! ...und heute wirds schon ein bißchen besser.
the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Jan-Benedict Glaw schrieb am 2006-07-31:

> Uh?  Where did you face a problem there?
> 
> With maildir, you shouldn't face any problems IMO. Even users with
> zillions of mails should work properly with the dir_index stuff:
> 
> 	tune2fs -O dir_index /dev/hdXX
> 
> or alternatively (to start that for already existing directories):
> 
> 	e2fsck -fD /dev/hdXX

hat is not "alternatively", but "tune2fs first", then "e2fsck -fD"
(which can't happen on a RW-mounted FS and you should only try this on
your rootfs if you can reboot with magic sysrq or from a rescue system).

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Dan Oglesby]

On Mon, 2006-07-31 at 19:16 +0200, Jan-Benedict Glaw wrote:
> On Mon, 2006-07-31 11:47:00 -0500, Dan Oglesby <doglesby@teleformix.com> wrote:
> > On Mon, 2006-07-31 at 18:22 +0200, Jan-Benedict Glaw wrote:
> > > On Mon, 2006-07-31 17:59:58 +0200, Adrian Ulrich <reiser4@blinkenlights.ch> wrote:
> > > > A colleague of mine happened to create a ~300gb filesystem and started
> > > > to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> > > > to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> > > So preparation work wasn't done.
> > 
> > As someone who is currently planning to migrate ~100GB of stored mail to
> > the Maildirs format, it was pretty clear early on that EXT3 would not
> > cut it (from past and current experiences), and not just for the sake of
> > calculating inodes.
> 
> Uh?  Where did you face a problem there?
> 

Past experiences dealing with systems that generate several thousand to
tens of thousands of files a day, adding up to well in the millions over
the course of normal production (this is not for a mail server, BTW).
Once we got close to a million files, filesystem transactions started
bogging the system, driving the load over 50.  Simply switching to
ReiserFS v3 allowed us to go well past the number of files EXT3 could
handle reasonably and the max load stayed right around the number of
CPUs the system contained (typically 8).

There is a LOT going on with these systems, not just filesystem
transactions.  EXT3 does not do well in our environment at all.

> With maildir, you shouldn't face any problems IMO. Even users with
> zillions of mails should work properly with the dir_index stuff:
> 
> 	tune2fs -O dir_index /dev/hdXX
> 
> or alternatively (to start that for already existing directories):
> 
> 	e2fsck -fD /dev/hdXX
> 
> 

I've been tuning EXT3 to see what performance I can get for the mail
server, and it's just not there compared to ReiserFS with minimal
tuning.

> Of course, you'll always face a problem with lots of files in one
> directory at getdents() time (eg. opendir()/readdir()/closedir()), but
> this is a common limit for all filesystems.
> 
> MfG, JBG
> 

Of course, but the issue is EXT3 does this a whole lot worse than
ReiserFS v3 from my experiences.

At any rate, that's about all I have to say about this issue.  I'll be
patiently waiting to see ReiserFS v4 included in the main kernel, so I
have less hoops to jump through to implement the latest and greatest
from Namesys.

--Dan

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

(resending complete message to the list).

Adrian Ulrich schrieb am 2006-07-31:

> Hello Matthias,
> 
> > This looks rather like an education issue rather than a technical limit.
> 
> We aren't talking about the same issue: I was asking to do it
> on-the-fly. Umounting the filesystem, running e2fsck and resize2fs
> is something different ;-)

There was stuff by Andreas Dilger, to support "online" resizing of
mounted ext2 file systems. I never cared to look for this (does it
support ext3, does it work with current kernels, merge status) since
offline resizing was always sufficient for me.

> A colleague of mine happened to create a ~300gb filesystem and started
> to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> to the new LUN. At about 70% the filesystem ran out of inodes;

Well - easy to fix, newfs again with proper inode density (perhaps 1 per
2 kB) and redo the migration. Of course you're free to pay for a new
file system if your fellow admin can't be bothered to remember newfs's
-i option.

> > Well, such "silly limitations"... looks like they are mostly hot air
> > spewn by marketroids that need to justify people spending money on their
> > new filesystem.
> 
> Have you ever seen VxFS or WAFL in action?

No I haven't. As long as they are commercial, it's not likely that I
will.

> Great to see that Sun ships a state-of-the-art Filesystem with
> Solaris... I think linux should do the same...

I think reallocating inodes for UFS and/or ext2/ext3 is possible, even
online, but someone needs to write, debug and field-test the code to do
that - possibly based on Andreas Dilger's earlier ext2 online resizing
work.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

> Well - easy to fix, newfs again with proper inode density (perhaps 1 per
> 2 kB) and redo the migration.

Ehr: Such a migration (on a very busy system) takes *some* time (weeks).
Re-Doing (migrate users back / recreate the FS / start again) the whole
thing isn't really an option..


> Of course you're free to pay for a new
> file system if your fellow admin can't be bothered to remember newfs's
> -i option.

Let's face it: Shit happens and nobody is perfect. A filesystem should
be flexible (modern..) and support Admin/User-needs.

We wouldn't need ECC / Raid / UPS's in a perfect world.

 
> > Have you ever seen VxFS or WAFL in action?
> 
> No I haven't. As long as they are commercial, it's not likely that I
> will.

Why?

NetApp WAFL-Blurb:
 http://www.netapp.com/library/tr/3002.pdf
 

Maybe we should crop the Cc: list .. this is getting OT.

 -- Adrian

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Adrian Ulrich schrieb am 2006-07-31:

> Ehr: Such a migration (on a very busy system) takes *some* time (weeks).
> Re-Doing (migrate users back / recreate the FS / start again) the whole
> thing isn't really an option..

All the more important to think about FS requirements *before*
newfs-ing if a quick "one day for rsync/star/dump+restore" isn't
available. If you're hitting, for instance, the hash collision problem
in reiser3, you're as dead as with a FS without inodes.

> > > Have you ever seen VxFS or WAFL in action?
> > 
> > No I haven't. As long as they are commercial, it's not likely that I
> > will.
> 
> Why?

I'm trying to shift my focus away from computer administration and
better file systems than old-style non-journalling, non-softupdates UFS
are available today and more will follow.

Cc: list weeded out.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Ulrich]

> All the more important to think about FS requirements *before*
> newfs-ing if a quick "one day for rsync/star/dump+restore" isn't
> available. If you're hitting, for instance, the hash collision problem
> in reiser3, you're as dead as with a FS without inodes.

Quoting myself:
>> Let's face it: Shit happens and nobody is perfect. A filesystem should
>> be flexible (modern..) and support Admin/User-needs.

Of COURSE you should think BEFORE creating the filesystem.
But if you somehow failed to do it 'the right thing' your filesystem
shouldn't let you down.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Theodore Tso]

On Mon, Jul 31, 2006 at 06:54:06PM +0200, Matthias Andree wrote:
> > > This looks rather like an education issue rather than a technical limit.
> > 
> > We aren't talking about the same issue: I was asking to do it
> > on-the-fly. Umounting the filesystem, running e2fsck and resize2fs
> > is something different ;-)
> 
> There was stuff by Andreas Dilger, to support "online" resizing of
> mounted ext2 file systems. I never cared to look for this (does it
> support ext3, does it work with current kernels, merge status) since
> offline resizing was always sufficient for me.

With the latest e2fsprogs and 2.6 kernels, the online resizing support
has been merged in, and as long as the filesystem was created with
space reserved for growing the filesystem (which is now the default,
or if the filesystem has the off-line prepration step ext2prepare run
on it), you can run resize2fs on a mounted filesystem and grow an
ext2/3 filesystem on-line.  And yes, you get more inodes as you add
more disk blocks, using the original inode ratio that was established
when the filesystem was created.

						- Ted

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

Theodore Tso schrieb am 2006-07-31:

> With the latest e2fsprogs and 2.6 kernels, the online resizing support
> has been merged in, and as long as the filesystem was created with
> space reserved for growing the filesystem (which is now the default,
> or if the filesystem has the off-line prepration step ext2prepare run
> on it), you can run resize2fs on a mounted filesystem and grow an
> ext2/3 filesystem on-line.  And yes, you get more inodes as you add
> more disk blocks, using the original inode ratio that was established
> when the filesystem was created.

That's cool.

The interesting part for some people would be, if I read past postings
correctly, to change the inode ratio in an existing (perhaps even
mounted) file system without losing data.

(I'm not sure how many blocks have to be moved and/or changed for that
purpose, because I know too little about the on-disk ext2 layout, but
since block relocating is already in place for shrink support in the
offline resizer, some of the work appears to be done already.)

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Matthias Andree wrote:

>
>>Have you ever seen VxFS or WAFL in action?
>>    
>>
>
>No I haven't. As long as they are commercial, it's not likely that I
>will.
>  
>
WAFL was well done.   It has several innovations that I admire,
including quota trees, non-support of fragments for performance reasons,
and the basic WAFL notion applied to an NFS RAID special (though
important) case.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Helge Hafting]

On Mon, Jul 31, 2006 at 05:59:58PM +0200, Adrian Ulrich wrote:
> Hello Matthias,
> 
> > This looks rather like an education issue rather than a technical limit.
> 
> We aren't talking about the same issue: I was asking to do it
> on-the-fly. Umounting the filesystem, running e2fsck and resize2fs
> is something different ;-)
> 
> > Which is untrue at least for Solaris, which allows resizing a life file
> > system. FreeBSD and Linux require an unmount.
> 
> Correct: You can add more inodes to a Solaris UFS on-the-fly if you are
> lucky enough to have some free space available.
> 
> A colleague of mine happened to create a ~300gb filesystem and started
> to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
> to the new LUN. At about 70% the filesystem ran out of inodes; Not a
> big deal with VxFS because such a problem is fixable within seconds.
> What would have happened if he had used UFS? mkfs -G wouldn't work
> because he had no additional Diskspace left... *ouch*..
> 
This case is solvable by planning.  When you know that the new fs
must be created with all inodes from the start, simply count
how many you need before migration.  (And add a decent safety margin.)
That's what I do with my home machine ask disks wear out every third 
year or so. The tools for ext2/3 tells how many inodes are in use,
and the new fs can be made accordingly.  The approach works for bigger
machines too of course.

Helge Hafting

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[venom]

planning sometimes is not possible, exspecially in certain highly stressed 
environment.

Just think. I had 3 years ago a database that was 2 TB, we were supposing 
it could grow in three years of 6 TB, but now it is 40 TB because
the market situation is changed, and with this the number of the users and 
their needs.

Please you have to suppose that when you 
have to deal with filesystems use for some kind of services, it is 
impossible to predict the grown rate, and this is true also about the 
numeber of used i-nodes.



On Tue, 1 Aug 2006, Helge Hafting wrote:

> On Mon, Jul 31, 2006 at 05:59:58PM +0200, Adrian Ulrich wrote:
>> Hello Matthias,
>>
>>> This looks rather like an education issue rather than a technical limit.
>>
>> We aren't talking about the same issue: I was asking to do it
>> on-the-fly. Umounting the filesystem, running e2fsck and resize2fs
>> is something different ;-)
>>
>>> Which is untrue at least for Solaris, which allows resizing a life file
>>> system. FreeBSD and Linux require an unmount.
>>
>> Correct: You can add more inodes to a Solaris UFS on-the-fly if you are
>> lucky enough to have some free space available.
>>
>> A colleague of mine happened to create a ~300gb filesystem and started
>> to migrate Mailboxes (Maildir-style format = many small files (1-3kb))
>> to the new LUN. At about 70% the filesystem ran out of inodes; Not a
>> big deal with VxFS because such a problem is fixable within seconds.
>> What would have happened if he had used UFS? mkfs -G wouldn't work
>> because he had no additional Diskspace left... *ouch*..
>>
> This case is solvable by planning.  When you know that the new fs
> must be created with all inodes from the start, simply count
> how many you need before migration.  (And add a decent safety margin.)
> That's what I do with my home machine ask disks wear out every third
> year or so. The tools for ext2/3 tells how many inodes are in use,
> and the new fs can be made accordingly.  The approach works for bigger
> machines too of course.
>
> Helge Hafting
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Matthias Andree wrote:
> Adrian Ulrich schrieb am 2006-07-31:

>> Why are a lot of Solaris-people using (buying) VxFS? Maybe because UFS
>> also has such silly limitations? (..and performs awkward with trillions
>> of files..?..)
> 
> Well, such "silly limitations"... looks like they are mostly hot air
> spewn by marketroids that need to justify people spending money on their
> new filesystem.

I think the limitations are silly, and I'm not paid to say this. 
Besides, we're talking about a filesystem that will be free (and libre), 
so I don't see the point of marketroids, certainly not in this context.

But let's not stoop to name-calling.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan Engelhardt]

>
>> A filesystem with a fixed number of inodes (= not readjustable while
>> mounted) is ehr.. somewhat unuseable for a lot of people with
>> big and *flexible* storage needs (Talking about NetApp/EMC owners)
>
>Which is untrue at least for Solaris, which allows resizing a life file
>system. FreeBSD and


>Linux require an unmount.

Only for shrinking.


Jan Engelhardt
-- 

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Pavel Machek]

Hi!

> >of the story for me. There's nothing wrong about focusing on newer code,
> >but the old code needs to be cared for, too, to fix remaining issues
> >such as the "can only have N files with the same hash value". 
> >
> Requires a disk format change, in a filesystem without plugins, to fix it.

Well, too bad, if reiser3 is so broken it needs on-disk-format-change,
then I guess doing that change is the right thing to do...
							Pavel
-- 
Thanks for all the (sleeping) penguins.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Grzegorz Kulewski]

On Wed, 26 Jul 2006, Pavel Machek wrote:

> Hi!
>
>>> of the story for me. There's nothing wrong about focusing on newer code,
>>> but the old code needs to be cared for, too, to fix remaining issues
>>> such as the "can only have N files with the same hash value".
>>>
>> Requires a disk format change, in a filesystem without plugins, to fix it.
>
> Well, too bad, if reiser3 is so broken it needs on-disk-format-change,
> then I guess doing that change is the right thing to do...

Sorry for my stupid question, but could you tell me why starting to make 
incompatible changes to reiserfs3 now (when reiserfs3 "technology" is 
rather old) and making reiserfs3 unstable (again), possibly for several 
months or even years is better than fixing big issues with reiser4 (if 
there are any really big left) merging it and trying to stabilize it?

For end user both ways will result in mkfs so...


Thanks,

Grzegorz Kulewski

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Thu, 27 Jul 2006, Grzegorz Kulewski wrote:

> Sorry for my stupid question, but could you tell me why starting to make 
> incompatible changes to reiserfs3 now (when reiserfs3 "technology" is 
> rather old) and making reiserfs3 unstable (again), possibly for several 
> months or even years is better than fixing big issues with reiser4 (if 
> there are any really big left) merging it and trying to stabilize it?
> 
> For end user both ways will result in mkfs so...

ext2fs and ext3fs, without "plugins", added dir_index as a compatible
upgrade, with an e2fsck option (that implies optional) to build indices
for directories without them.

ext3fs is a compatible upgrade from ext2fs, it's as simple as unmount,
tune2fs -j, mount.

reiserfs 3.6 could deal with 3.5 file systems, and "mount -o conv" with
a 3.6 driver would convert a 3.5 file system to 3.6 level
(ISTR it had to do with large file support and perhaps NFS
exportability, but don't quote me on that).

I wonder what makes the hash overflow issue so complicated (other than
differing business plans, that is) that upgrading in place isn't
possible. Changes introduce instability, but namesys were proud of their
regression testing - so how sustainable is their internal test suite?

Instead, we're told reiser4 would fix this (quite likely) and we should
wait until it's ready (OK, we shouldn't be using experimental stuff for
production but rather for /tmp, but the file system will take many
months to mature after integration) and it will be "mkfs" time - so
reiser4 better be mature before we go that way if there's no way back
short of "amrecover", "restore" or "tar -x".

Smashing out most of the Cc:s in order not to bore people.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Matthias Andree wrote:

>I wonder what makes the hash overflow issue so complicated (other than
>differing business plans, that is) that upgrading in place isn't
>possible. Changes introduce instability, but namesys were proud of their
>regression testing - so how sustainable is their internal test suite?
>  
>
>
Never met a test suite the equal of a few million users.....

People have this image of Namesys as some large corporation that has
large resources.  We just barely are going to be able to ship reiser4,
at the cost of a LOT of financial pain.  We can't afford to go in two
directions at once.  We can add bugfixes to V3, but adding features, I
have to tell you that we ain't got the staff for both that and shipping
V4.  Our whole corporation has a budget about what most corporations
spend on two programmers.   We have 5 developers, including me, and
making little bits of money is a constant distraction from the main work.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff Garzik]

Pavel Machek wrote:
> Hi!
> 
>>> of the story for me. There's nothing wrong about focusing on newer code,
>>> but the old code needs to be cared for, too, to fix remaining issues
>>> such as the "can only have N files with the same hash value". 
>>>
>> Requires a disk format change, in a filesystem without plugins, to fix it.
> 
> Well, too bad, if reiser3 is so broken it needs on-disk-format-change,
> then I guess doing that change is the right thing to do...

Actually, there is reiser4 brokenness lurking in Hans' statement, too:

A filesystem WITH plugins must still handle the standard Linux 
compatibility stuff that other filesystems handle.

Plugins --do not-- mean that you can just change the filesystem format 
willy-nilly, with zero impact.

	Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Jeff Garzik wrote:
> Pavel Machek wrote:
>> Hi!
>>
>>>> of the story for me. There's nothing wrong about focusing on newer 
>>>> code,
>>>> but the old code needs to be cared for, too, to fix remaining issues
>>>> such as the "can only have N files with the same hash value".
>>> Requires a disk format change, in a filesystem without plugins, to 
>>> fix it.

> A filesystem WITH plugins must still handle the standard Linux 
> compatibility stuff that other filesystems handle.
> 
> Plugins --do not-- mean that you can just change the filesystem format 
> willy-nilly, with zero impact.

They --do-- mean that you can change much of the filesystem behavior 
without requiring massive on-disk changes or massive interface changes.

After all, this is how many FUSE plugins work -- standard FS interface, 
usually uses another standard FS as storage, but does crazy things like 
compression, encryption, and other transformations in between.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Jeff Garzik wrote:

>
> Actually, there is reiser4 brokenness lurking in Hans' statement, too:

Where!  Someone tell me!;-)

>
> A filesystem WITH plugins must still handle the standard Linux
> compatibility stuff that other filesystems handle.

Hmm, you mean we should first implement regular unix file plugins before
implementing enhanced functionality ones?  Are you aware that reiser4
plugins are per file, and thus if a user selects a plugin that is not
the default, and which has user visible semantic differences, it means
they said they want non-standard behavior?

>
> Plugins --do not-- mean that you can just change the filesystem format
> willy-nilly, with zero impact.

Yes they do.....

>
>     Jeff
>
>
>
>
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1378 bytes --]

On Sun, 2006-07-23 01:20:40 -0600, Hans Reiser <reiser@namesys.com> wrote:
> There is nothing about small patches that makes them better code.  There

Erm, a small patch is something which should _obviously_ fix one
issue. A small patch, containing at max some 100 lines, can easily be
read and understood.

A complete filesystem (I'm co-maintaining one for an ancient on-disk
format, too) isn't really easy to understand or to verify from looking
at it for 5min.

> is no reason we should favor them, if the developers are willing to work
> on something for 5 years to escape a local optimum, that is often the
> RIGHT thing to do.

I give a shit of nothing to some 5 year work if I cannot verify that
it won't hurt me at some point.

> It is importand that we embrace our diversity, and be happy for the
> strength it gives us.  Some of us are good at small patches that evolve,
> and some are good at escaping local optimums.  We all have value, both
> trees and grass have their place in the world.

Just put reiser4 in some GIT tree and publish it. Maybe you can place
it on git.kernel.org .

MfG, JBG

-- 
       Jan-Benedict Glaw       jbglaw@lug-owl.de                +49-172-7608481
 Signature of:                     ...und wenn Du denkst, es geht nicht mehr,
 the second  :                            kommt irgendwo ein Lichtlein her.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hans Reiser wrote:
> I just got an email from the programmer who wrote the MythTV bug saying
> that he is just too busy to bother fixing the bug in his code.....  so
> my response is that a Namesys programmer is going to fix it on Monday.


Hans -

I'll accept blame when it's my bug, but the MythTV one isn't. I've been
working with the bitmap code and did the analysis to track down what was
happening, but that doesn't make it a bug in my code.

That particular bug isn't in the bitmap scanning code, it's a side
effect of the write batching higher up. It's looking for a window of 32
blocks, and there's just no window that large available. It ends up
scanning all the bitmaps looking for the window, and then backs off to
single block allocations. The scanning code works fine, and it does skip
where there aren't enough free blocks available in a particular bitmap.

It's a pathological case when the file system is seriously fragmented. A
quick fix would be to set a flag indicating that future writes shouldn't
bother trying to find a window that large, but that's a hack. A better
allocation algorithm would keep track of free space extents in memory,
subject to getting dropped by memory pressure. Since that information
would be separate from the bitmaps themselves, we could get rid of that
nasty "is this block free, but in the journal?" check that we need to do
as well. It's invasive, and a quicker fix would just be to track the
largest window, and rescan when it gets used or a block in that bitmap
gets freed.

That said, I actually did start work on a fix for this one, but I really
just don't have the time right now.

- -Jeff

- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFEw+BBLPWxlyuTD7IRAtG8AKCOWW/AH3NAen6gd6BToJGVfzdnNACfYkVS
j2/6yAAeWKAhs4ng9fdGW0Y=
=gB+v
-----END PGP SIGNATURE-----

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Jeff Mahoney wrote:

>
>
> That particular bug isn't in the bitmap scanning code, it's a side
> effect of the write batching higher up.

Did you write the code that looks for a window of 32
blocks?  If not, and if this code has been around for a long time, I
apologize.   I thought you did write it and added it in recent months.

>
>
> It's a pathological case when the file system is seriously fragmented.

Most bugs are pathological cases.;-)

> A
> quick fix would be to set a flag indicating that future writes shouldn't
> bother trying to find a window that large,

There are lots of quick fixes.  1) The quickest is to not scan for the
window at all.  2) The second quickest is to limit the number of bitmaps
that will be scanned to some number like 3.  3) The not at all quickest
is to track free extents like XFS does, which is not a hack, but it
belongs in a development branch.  I am not sure it is worth the
complexity, but my mind is not closed.

On monday we will do 1) or 2), probably 1).   After the repacker is
done, we should review all our block allocation algorithms.  I have an
idea for how to do things more optimally for streaming media that will
avoid fragmentation over time, and when combined with the repacker may
make 3 not worthwhile.

I am grateful that you and Chris do bug fixes, but when you guys are too
busy, (and that can and will happen to any of us), the baton needs to
get passed.  V3 needs to be a zero defect product, and once we know it
is a bug I don't want bugs in V3 to remain unfixed for more than a day
plus the time it takes to fix it.    If you do add code, I want any bugs
that show up in the aftermath of mainstream merging to get jumped on.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hans Reiser wrote:
> Jeff Mahoney wrote:
> 
>>
>> That particular bug isn't in the bitmap scanning code, it's a side
>> effect of the write batching higher up.
> 
> Did you write the code that looks for a window of 32
> blocks?  If not, and if this code has been around for a long time, I
> apologize.   I thought you did write it and added it in recent months.

Nope. The scan_bitmap_block() code that looks for windows was added in
a changeset from a bk merge against 2.5.33 in September 2002. The change
to want a minimum size window was added in June 2004 to 2.6.8-rc2. That
patch is actually credited to both Mason and I, but I don't recall who
wrote that bit. It may well have been my code, after all, but it's
certainly not a new bug. *shrug* I guess MythTV might just be generating
an i/o pattern that hadn't been seen before.

>> A
>> quick fix would be to set a flag indicating that future writes shouldn't
>> bother trying to find a window that large,
> 
> There are lots of quick fixes.  1) The quickest is to not scan for the
> window at all.  2) The second quickest is to limit the number of bitmaps
> that will be scanned to some number like 3.  3) The not at all quickest
> is to track free extents like XFS does, which is not a hack, but it
> belongs in a development branch.  I am not sure it is worth the
> complexity, but my mind is not closed.
> 
> On monday we will do 1) or 2), probably 1).   After the repacker is
> done, we should review all our block allocation algorithms.  I have an
> idea for how to do things more optimally for streaming media that will
> avoid fragmentation over time, and when combined with the repacker may
> make 3 not worthwhile.

If you want to go the 1) route, it's trivial. See patch below. It will
restore the one-block-at-a-time behavior.

> I am grateful that you and Chris do bug fixes, but when you guys are too
> busy, (and that can and will happen to any of us), the baton needs to
> get passed.  V3 needs to be a zero defect product, and once we know it
> is a bug I don't want bugs in V3 to remain unfixed for more than a day
> plus the time it takes to fix it.    If you do add code, I want any bugs
> that show up in the aftermath of mainstream merging to get jumped on.

Anyone up for it? :) There are changes I'd like to see in reiser3,
particularly ones that address the severe problems observed in David
Chinner's high bandwidth file system talk this year at OLS. Specifically,
it ended up making very little progress and spending the majority of the
time in the journal when the workload is streaming data at the disk at a
very high rate on a very large file system. Yes, that is certainly XFS's
sweet spot, but barely making progress at all is a bit more severe than
"poor performance." Perhaps mkreiserfs should be a bit saner about choosing
journal sizes, since a 32 MB journal is not a good fit for all cases. Also,
I'd like to see the usage of the BKL gone as it severely limits performance
when more than one thread is writing to the file system, or even another
reiserfs file system. It's not entirely low hanging fruit since the nested
cases need to be audited, but it shouldn't be too hard to eliminate the
inter-filesystem lock contention by replacing the BKL with a per-sb mutex.
I have some more things, but I have nowhere near the time to do them,
and other file systems will perform fine.

- -Jeff

Patch:

- --- linux-2.6.17.orig/fs/reiserfs/bitmap.c	2006-01-02 22:21:10.000000000 -0500
+++ linux-2.6.17.orig.devel/fs/reiserfs/bitmap.c	2006-07-23 19:10:57.000000000 -0400
@@ -1020,7 +1020,6 @@
 	b_blocknr_t finish = SB_BLOCK_COUNT(s) - 1;
 	int passno = 0;
 	int nr_allocated = 0;
- -	int bigalloc = 0;
 
 	determine_prealloc_size(hint);
 	if (!hint->formatted_node) {
@@ -1047,28 +1046,9 @@
 				hint->preallocate = hint->prealloc_size = 0;
 		}
 		/* for unformatted nodes, force large allocations */
- -		bigalloc = amount_needed;
 	}
 
 	do {
- -		/* in bigalloc mode, nr_allocated should stay zero until
- -		 * the entire allocation is filled
- -		 */
- -		if (unlikely(bigalloc && nr_allocated)) {
- -			reiserfs_warning(s, "bigalloc is %d, nr_allocated %d\n",
- -					 bigalloc, nr_allocated);
- -			/* reset things to a sane value */
- -			bigalloc = amount_needed - nr_allocated;
- -		}
- -		/*
- -		 * try pass 0 and pass 1 looking for a nice big
- -		 * contiguous allocation.  Then reset and look
- -		 * for anything you can find.
- -		 */
- -		if (passno == 2 && bigalloc) {
- -			passno = 0;
- -			bigalloc = 0;
- -		}
 		switch (passno++) {
 		case 0:	/* Search from hint->search_start to end of disk */
 			start = hint->search_start;
@@ -1106,8 +1086,7 @@
 								 new_blocknrs +
 								 nr_allocated,
 								 start, finish,
- -								 bigalloc ?
- -								 bigalloc : 1,
+								 1,
 								 amount_needed -
 								 nr_allocated,
 								 hint->


- -- 
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFExATJLPWxlyuTD7IRAmeKAJsFI/awPPAXpB2DI+kO19EZtr3tRwCfWduO
Re+5kXNtj6St/LuUy9lbNm4=
=anQd
-----END PGP SIGNATURE-----

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Jeff Mahoney wrote:

>
>
> Anyone up for it? :) There are changes I'd like to see in reiser3,
> particularly ones that address the severe problems observed in David
> Chinner's high bandwidth file system talk this year at OLS. Specifically,
> it ended up making very little progress and spending the majority of the
> time in the journal when the workload is streaming data at the disk at a
> very high rate on a very large file system. Yes, that is certainly XFS's
> sweet spot, but barely making progress at all is a bit more severe than
> "poor performance." Perhaps mkreiserfs should be a bit saner about
> choosing
> journal sizes, since a 32 MB journal is not a good fit for all cases.
> Also,
> I'd like to see the usage of the BKL gone as it severely limits
> performance
> when more than one thread is writing to the file system, or even another
> reiserfs file system. It's not entirely low hanging fruit since the nested
> cases need to be audited, but it shouldn't be too hard to eliminate the
> inter-filesystem lock contention by replacing the BKL with a per-sb mutex.

Getting rid of the BKL is a huge task that was done in V4 for a reason. 
You are talking about 6+ man-months, and years of shake-out to fully
debug.  Actually, it is a tribute to Zam's skill that V4's locking got
debugged so fast: I gave him the task knowing it was going to be the
hardest code to debug, and he did it very well.

These things you discuss, except for the journal size, are not things to
fix in a stable branch.

My apologies that I thought this was a new bug.  Let us be glad that a
user gave us enough detail we saw it.

> I have some more things, but I have nowhere near the time to do them,
> and other file systems will perform fine.
>
>
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Russell Cattelan]

On Sun, 2006-07-23 at 01:20 -0600, Hans Reiser wrote:
> Jeff, I think that a large part of what is going on is that any patch
> that can be read in 15 minutes gets reviewed immediately, and any patch
> that is worked on for 5 years and then takes a week to read gets
> neglected.  This is true even if line for line the 1 week to read patch
> is more valuable.    What is more is that people know this is
> irrational, but aren't able to cure it in themselves.  Even I have a
> problem of paying too much attention to endless 5 minute emails when I
> know I should instead, say, read the compression plugin from beginning
> to end.
> 
> There is nothing about small patches that makes them better code.  There
> is no reason we should favor them, if the developers are willing to work
> on something for 5 years to escape a local optimum, that is often the
> RIGHT thing to do.
> 
> It is importand that we embrace our diversity, and be happy for the
> strength it gives us.  Some of us are good at small patches that evolve,
> and some are good at escaping local optimums.  We all have value, both
> trees and grass have their place in the world.
> 
Which is summed up quite well by:
http://en.wikipedia.org/wiki/Color_of_the_bikeshed

It seem to be a well know tendency for people to want to
be involved in some way, thus keeping to much of the development
cycle internal tends to generate friction.


-Russell Cattelan
cattelan@xfs.org

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

[-- Attachment #1: Type: text/plain, Size: 3367 bytes --]

Russell Cattelan wrote:
> On Sun, 2006-07-23 at 01:20 -0600, Hans Reiser wrote:
>> Jeff, I think that a large part of what is going on is that any patch
>> that can be read in 15 minutes gets reviewed immediately, and any patch
>> that is worked on for 5 years and then takes a week to read gets
[...]
>> It is importand that we embrace our diversity, and be happy for the
>> strength it gives us.  Some of us are good at small patches that evolve,
>> and some are good at escaping local optimums.  We all have value, both
>> trees and grass have their place in the world.
>>
> Which is summed up quite well by:
> http://en.wikipedia.org/wiki/Color_of_the_bikeshed
> 
> It seem to be a well know tendency for people to want to
> be involved in some way, thus keeping to much of the development
> cycle internal tends to generate friction.

No, I think Hans is right.

Although I should mention, Hans, that there is a really good reason to
prefer the 15 minute patches.  The patches that take a week are much
harder to read during that week than any number of 15 minute incremental
patches, because the incremental patches are already broken down into
nice, small, readable, ordered chunks.  And since development follows
some sort of logical, orderly pattern, it can be much easier to read it
that way than to try to consider the whole.

Think of it this way -- why are debuggers useful?  One of the nicest
thing about a debugger, especially for newbies, is the ability to step
through a program a line at a time.  It's the same principle -- you can
understand the program state at one point in time, and the impact of one
line of code, much more easily than the overall model of the program
state (and all of its edge cases), or the impact of several hundred
(thousand? tens of thousands?) lines of code.

So while I don't blame the Namesys team for putting off inclusion till
it's done, I also can't really blame the kernel guys for not wanting to
read it, especially if it's revolutionary.  Revolutionary ideas are hard
to grasp, and it's not their fault.

I mean, if revolutionary ideas were easy, why didn't you write Reiser4
for a system like, say, Tunes? (http://tunes.org/)  Say what you will,
but there are ways of doing fast filesystems which don't require that
said filesystems be written in kernel C.  Consider this:

http://www.cs.utah.edu/flux/oskit/

If I understand that right, it's a mechanism for writing kernel code in
languages like "Java, Lisp, Scheme, or ML"...

If we could all grasp every single good (best) idea from every corner of
software engineering, and write completely new software (including the
OS) using those ideas, we could potentially replace all existing
software in something like 3-5 years with software which has none of the
problems ours does now.  We'd never have inflexibility, insecurity,
instability, user interface issues...  Never have to worry about getting
software out the door (it'd be so fast to develop), but always have it
designed the right way the first time, yet be able to rearrange it
completely with only 5-10 line patches.

So it's not always the computer hardware that's the limitation.  Often
it's our hardware as well.  Human beings usually aren't equipped to be
able to grok the whole universe all at once.  If we were...  see above.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 890 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

David Masover wrote:

>
>
>Although I should mention, Hans, that there is a really good reason to
>prefer the 15 minute patches.  The patches that take a week are much
>harder to read during that week than any number of 15 minute incremental
>patches, because the incremental patches are already broken down into
>nice, small, readable, ordered chunks.  And since development follows
>some sort of logical, orderly pattern, it can be much easier to read it
>that way than to try to consider the whole.
>  
>
No, I disagree, if the code is well commented, it is easier to read the
whole thing at the end when it has its greatest coherence and
refinement.  A problem with Reiser4 is that its core algorithms are
simply complex.  We pushed the envelope in multiple areas all at once.
Benchmarks don't always suggest simple algorithms are the ones that will
be highest performance.  Tree algorithms are notorious in the database
industry for being simple on web pages but complex as code.

Some people program in small increments, some program things that
require big increments of change, both kinds of people are needed.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Russell Cattelan]

On Wed, 2006-07-26 at 08:26 -0600, Hans Reiser wrote:
> David Masover wrote:
> 
> >
> >
> >Although I should mention, Hans, that there is a really good reason to
> >prefer the 15 minute patches.  The patches that take a week are much
> >harder to read during that week than any number of 15 minute incremental
> >patches, because the incremental patches are already broken down into
> >nice, small, readable, ordered chunks.  And since development follows
> >some sort of logical, orderly pattern, it can be much easier to read it
> >that way than to try to consider the whole.
> >  
> >
> No, I disagree, if the code is well commented, it is easier to read the
> whole thing at the end when it has its greatest coherence and
> refinement.  A problem with Reiser4 is that its core algorithms are
> simply complex.  We pushed the envelope in multiple areas all at once.
> Benchmarks don't always suggest simple algorithms are the ones that will
> be highest performance.  Tree algorithms are notorious in the database
> industry for being simple on web pages but complex as code.
> 
> Some people program in small increments, some program things that
> require big increments of change, both kinds of people are needed.
> 
FWIW I think both points are valid.
 
Personally I find it difficult to completely stop what I'm doing 
and spend several days studying a large patch/complete new subsystem.

But on the other hand it's important that code that goes in the kernel 
gets a proper review, I think we all come to rely on the gatekeepers job
of making sure any given part of the kernel does not destabilize to the 
point were it interferes with our individual areas of development.

So exactly what level of granularity is appropriate  for changes? Well
that should probably be left of to the gatekeeper for each particular
area. In the case of filesystems generic vfs changes obviously need to 
be small and easy to digest, and more importantly easy to bisect
regressions. The core of the file system can be left to the person who
can best manage the code, but hopefully that person applies a reasonable
of granularity to the changes, thus allowing non familiar developers to
at least keep up and possibly make helpful suggestions.

If we look at the current "beliefs" surrounding XFS you can see the
affects of a code base that did not have an incremental development with
regards to linux anyways. Ok so ya XFS was a close sourced IRIX FS for
the first 8 years of it's life, and even once the Linux project started
there was another year or so encumbrance investigation and cleaning
before legal would sign off on its release.
So to this day the major hang up with XFS seems to be "it's to complex
because it has x thousands lines of code".  Hell by that argument Linux
has way more lines of code than IRIX could ever hope to have and is
therefore Linux is more complex than IRIX and to hard to understand. :-)
Fortunately many developers (especially ones that have worked on other
OS's) do not use "wc -l" as a tool to measure code
quality/readability/complexity. 
XFS unfortunately will probably always suffer from skepticism since
it did not grow up in in Linux.

Guess it's sort of like adopting a 8 year old child vs a new born, hard
to tell what happened in first 8 years.

-Russell Cattelan
cattelan@xfs.org

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

Russell Cattelan wrote:

> Guess it's sort of like adopting a 8 year old child vs a new born, hard
> to tell what happened in first 8 years.

And, by that token, the newborn is sometimes preferable.  It hasn't had 
time to develop severe emotional problems, it's physically harmless, and 
you get to help it form its ideas and beliefs.

On the other hand, the 8 year old is potty trained, you won't have to 
change a single diaper, it's intelligent and makes you question things, 
it understands what you want it to do...

So, it depends what kind of developer you are, what kind of gatekeeper, 
and what kind of project it is.  I still believe in releasing early and 
often, but I can see many reasons not to.  Some are financial -- Namesys 
is trying to operate a business, so any features they open up too early 
could be that much harder to sell as a commercial product.  The 
repacker, for instance.

I'm not arguing for closed source, I'm just saying that once you open, 
there's no going back.  Many times it's a good thing, but sometimes you 
want to wait and see.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Masover]

[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]

Hans Reiser wrote:

> to use as his default.  Now that we paid the 5 year development price
> tag to get everything as plugins, we can now upgrade in littler pieces
> than any other FS.  Hmm, I need a buzz phrase, its not extreme
> programming, maybe "moderate programming".  Does that sound exciting to

Hah!  No, it doesn't sound exciting.

Plugins don't work well either, not as a marketing concept.  People have
had so many bad experiences with plugins, and they're only ever visible
when you have a bad experience.  Think about it -- missing plugin (so
you have to download it),

On the other hand, it works for WordPress.  My day job is work on a
plugin for WordPress.  Not including a link because I feel dirty for
having to work with PHP...

Fluid programming?  If you build a solution from the bottom up with
gravel or large rocks, you leave gaps that are hard to fill without
ripping off the top layer and redoing it.  But if you can do fluid
programming, your program just flows around any obstacle, and into every
crack / between every space (metaphor for new customer requirements)...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 890 bytes --]

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Luigi Genoni]

On Wed, July 26, 2006 02:44, David Masover wrote:
> Hans Reiser wrote:
>
>
>> to use as his default.  Now that we paid the 5 year development price tag
>> to get everything as plugins, we can now upgrade in littler pieces than
>> any other FS.  Hmm, I need a buzz phrase, its not extreme programming,
>> maybe "moderate programming".  Does that sound exciting to
>
> Hah!  No, it doesn't sound exciting.
>
>
> Plugins don't work well either, not as a marketing concept.  People have
> had so many bad experiences with plugins, and they're only ever visible when
> you have a bad experience.  Think about it -- missing plugin (so you have to
> download it),
>
marketing?
</joke mode on>
if you do not like the word "plugin" why don't you suggest some alternative?
like "modules"?
</joke mode off>

Seriously, please leave out this kind of marketing. The plugin concept in
reiser4 is probably the most interessant feature I filesystem some users
could need.

>
> Fluid programming?  If you build a solution from the bottom up with
> gravel or large rocks, you leave gaps that are hard to fill without ripping
> off the top layer and redoing it.  But if you can do fluid programming, your
> program just flows around any obstacle, and into every crack / between every
> space (metaphor for new customer requirements)...

probably I do not know english enought well to appraciate this metaphor and
to understand what it means.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

David Masover wrote:

>Hans Reiser wrote:
>
>  
>
>>to use as his default.  Now that we paid the 5 year development price
>>tag to get everything as plugins, we can now upgrade in littler pieces
>>than any other FS.  Hmm, I need a buzz phrase, its not extreme
>>programming, maybe "moderate programming".
>>
This phrase was a bit tongue-in-cheek.

>>  Does that sound exciting to
>>    
>>
>
>  
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[David Weinehall]

On Sun, Jul 23, 2006 at 01:20:40AM -0600, Hans Reiser wrote:
> Jeff, I think that a large part of what is going on is that any patch
> that can be read in 15 minutes gets reviewed immediately, and any patch
> that is worked on for 5 years and then takes a week to read gets
> neglected.  This is true even if line for line the 1 week to read patch
> is more valuable.    What is more is that people know this is
> irrational, but aren't able to cure it in themselves.  Even I have a
> problem of paying too much attention to endless 5 minute emails when I
> know I should instead, say, read the compression plugin from beginning
> to end.

Well, the problem is that someone actually went through the
effort of doing the week-long reviews of your code, you flamed him
every time he suggested that there was need for improvement, instead
of saying thanks for all the hard work and implementing the needed
fixes.

Not exactly the right way to make people fond of reviewing your code.
Getting flamed by someone having a hard time taking criticism
after spending 5 minutes to review a patch is bearable, after all,
what's 5 minutes, right?  Flaming someone that has put down a week
or two of hard work on review is just disrespectful.

[snip]


Regards: David Weinehall
-- 
 /) David Weinehall <tao@acc.umu.se> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Sat, 22 Jul 2006, Jeff Garzik wrote:

> Anyone who fails to respect the kernel development process, the process 
> of building consensus, is turn not respected, flamed, and/or ignored.

That reminds me of the old "layer 8 and 9" extensions to the OSI/ISO
layering model. Layer 8: financial, Layer 9: policital.

SCNR.

> With you in particular, you demonstrated NO interest in maintaining 
> reiser3, once reiser4 began to make a splash.  Linux kernel code exists 
> for DECADES, and as such, long term maintenance is a CRITICAL aspect of 
> development.
> 
> Regardless of whatever new whiz-bang technology exists in reiser4, there 
> is a very real worry that you will abandon reiser4 once its in the tree 
> for a few years, just like what happened with reiser3.

The most worrying point was that reiser3 maintenance was given up at the
point where it was just about to transition from usable to mature.

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Matthias Andree wrote:

>The most worrying point was that reiser3 maintenance was given up at the
>point where it was just about to transition from usable to mature.
>
>  
>
Name a bug (not a feature) that has not been fixed by us, and which is
not also so deep that it reasonably required waiting for a major release
to fix it (for example, bug fixes that require disk format changes
belong in v4 not v3 even if a case can be made that they are bug fixes).

I mean, god, sometimes I think users are like little children waiting
for the pie that is in the oven and who want to take it out now before
it finishes cooking so they can eat it, and they are very angry about
it, and I should just understand that and not try to reason with it (but
also not give them the pie before it finishes cooking either).   Someone
please tell me I don't understand the users and it all makes more sense
than that, please....

No code before its time.  No features in stable branches.  Wait for it. 
Stop complaining about how you are abandoned, we are working hard.  It's
going to be the best pie ever.  Wait for it.

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Matthias Andree]

On Mon, 24 Jul 2006, Hans Reiser wrote:

> I mean, god, sometimes I think users are like little children waiting
> for the pie that is in the oven and who want to take it out now before
> it finishes cooking so they can eat it, and they are very angry about
> it, and I should just understand that and not try to reason with it (but
> also not give them the pie before it finishes cooking either).   Someone
> please tell me I don't understand the users and it all makes more sense
> than that, please....

namesys.com doesn't list reiserfs 3.6 hash collision limits in an easy
to find place (which would be the same place that boasts about 100,000
files per directory if it wants to be honest).

> No code before its time.  No features in stable branches.  Wait for it. 
> Stop complaining about how you are abandoned, we are working hard.  It's
> going to be the best pie ever.  Wait for it.

I'm not going to eat it while it's still steaming and fogging my glasses.

You're now making the same noise about how good reiser4 is that was made
when reiserfs 3.5 was a patch for Linux 2.2 and that wedged local access
when NFS exported, and 3.6 didn't fix the hash collision issue (but
required a format change, too, right).

-- 
Matthias Andree

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Matthias Andree <matthias.andree@gmx.de> wrote:
> On Sat, 22 Jul 2006, Jeff Garzik wrote:
> 
> > Anyone who fails to respect the kernel development process, the process 
> > of building consensus, is turn not respected, flamed, and/or ignored.
> 
> That reminds me of the old "layer 8 and 9" extensions to the OSI/ISO
> layering model. Layer 8: financial, Layer 9: policital.

Got that one wrong.

Layer  8: User
Layer  9: Political
Layer 10: Financial
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Rene Rebe]

Hi,

On Saturday 22 July 2006 15:02, Theodore Tso wrote:
> > The code isn't even written, benchmarked, or tested yet,
>
> Actually, the first bits that we plan to merge have already been
> written and in use by hundreds of clusterfs customers, posted to LKML
> for comments (and we don't attack our reviewers, we thank them for
> their comments), and in fact they were written about at last year's
> OLS complete with benchmarks and graphs.
> (http://ext2.sourceforge.net/2005-ols/2005-ols-ext3.html)

However I would estimate that Reiser4 is used by more people than yet
aother ext2 patchups.

Yours,

-- 
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            http://exactcode.de | http://t2-project.org | http://rebe.name
            +49 (0)30 / 255 897 45

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Nikita Danilov]

Rene Rebe writes:
 > Hi,
 > 
 > On Saturday 22 July 2006 15:02, Theodore Tso wrote:
 > > > The code isn't even written, benchmarked, or tested yet,
 > >
 > > Actually, the first bits that we plan to merge have already been
 > > written and in use by hundreds of clusterfs customers, posted to LKML
 > > for comments (and we don't attack our reviewers, we thank them for
 > > their comments), and in fact they were written about at last year's
 > > OLS complete with benchmarks and graphs.
 > > (http://ext2.sourceforge.net/2005-ols/2005-ols-ext3.html)
 > 
 > However I would estimate that Reiser4 is used by more people than yet
 > aother ext2 patchups.

Any data backing up that estimation? Just to give an example, that
"patchup" is used by (tens of) thousands of computers in governmental
laboratories the US "national security" depends upon.

 > 
 > Yours,

Nikita.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Andrea Arcangeli]

I'm no reiserfs user (nor v3 nor v4), but I respect views of people
that needs the fs to handle zillons of tiny files, for that reiser3
did a good job, and my humble opinion is that the only chance for
reiser4 to stabilize is to grow the userbase and including it in
mainline or in some distro would obviously help (like it happened to
reiser3 in the past with SUSE inclusion first and mainline inclusion
later, nothing has changed dramatically since then, reiser3 was very
buggy at the time too, it was probably my backport of lfs that made us
notice it couldn't even handle lfs on-disk so they had to change the
fs format later on with some backwards compatibility mess with older
kernels, hopefully this time the fixage of v4 for production usage
won't require a change of the on-disk format).

On Mon, Jul 24, 2006 at 11:49:43AM +0400, Nikita Danilov wrote:
> Any data backing up that estimation? Just to give an example, that

Probably KLive is the only tool that can provide an estimate of the
part of the reiser4 userbase compared to the other fs. reiser4 had so
far 1926 days of total uptime and 4409 users using it at least once
for root or anyway mounted nearly at boot. Avg uptime is 16 hours, max
uptime is 28 days.

I can also provide the exact storage scsi/SATA/ATA chipsets used in
combination with it, but it doesn't seem necessary.

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where f = 'reiser4' group by f order by sum(uptime) desc;
                f                 | count |             sum              |       avg       |       max        
----------------------------------+-------+------------------------------+-----------------+------------------
 reiser4                          |  4409 | 1926 days 27479:22:27.459534 | 16:42:59.666015 | 28 days 16:16:15
(1 row)

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where f = 'ext4' group by f order by sum(uptime) desc;
 f | count | sum | avg | max 
---+-------+-----+-----+-----
(0 rows)

klive=> 

Full data of all fs (all kernels not just mainline ones) follows. In
total uptime and avg uptime terms it beats many other common fs like
isofs, autofs and ramfs. It'd be cool if I'd be able to receive a
packet during sigterm so I could then differentiate sessions that
rebooted cleanly from the ones that didn't. Receiving oopses
(optionally encrypted) is also a long term planned feature. At the
moment I can't (and I've no much time to work on klive), but this is
still good to quantify the current userbase compared to the other more
common fs.

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive group by f order by sum(uptime) desc;
                f                 | count |              sum               |           avg           |           max           
----------------------------------+-------+--------------------------------+-------------------------+-------------------------
 sysfs                            | 40998 | 35138 days 259797:16:29.93576  | 26:54:23.100394         | 173 days 16:16:15
 tmpfs                            | 40507 | 35025 days 257298:57:48.690973 | 27:06:14.154311         | 173 days 16:16:15
 usbfs                            | 38803 | 26681 days 241169:00:36.268676 | 22:43:03.543444         | 173 days 16:16:15
 reiserfs                         | 19934 | 19257 days 133294:48:27.256508 | 29:52:18.181361         | 155 days 16:16:15
 ext3                             | 22077 | 16346 days 132190:42:38.862752 | 23:45:27.062502         | 173 days 16:16:15
 ext2                             |  9396 | 5629 days 63099:50:27.500173   | 21:05:37.103821         | 139 days 14:54:11
 nfs                              |  2824 | 6373 days 23368:45:22.667638   | 2 days 14:26:11.502361  | 86 days 16:16:15
 binfmt_misc                      |  4396 | 5421 days 33069:20:47.871638   | 1 day 13:07:06.944466   | 173 days 16:16:15
 nfsd                             |  2231 | 4727 days 19824:00:00.354481   | 2 days 11:44:11.187967  | 89 days 16:16:15
 xfs                              |  4326 | 4067 days 28000:38:11.762818   | 29:02:08.685105         | 139 days 14:54:11
 vfat                             |  8684 | 2470 days 46585:39:49.859957   | 12:11:27.193673         | 68 days 16:16:15
 ntfs                             |  8482 | 2382 days 47344:00:52.902741   | 12:19:17.846369         | 68 days 16:16:15
 rpc_pipefs                       |  1188 | 3078 days 11297:55:12.600343   | 2 days 23:41:30.667172  | 173 days 16:16:15
 reiser4                          |  4409 | 1926 days 27479:22:27.459534   | 16:42:59.666015         | 28 days 16:16:15
^^^^^^^^^^^^^
 autofs                           |  2030 | 2374 days 14512:14:16.617646   | 1 day 11:12:57.170748   | 54 days 16:16:15
 ramfs                            |  2847 | 817 days 14809:30:42.122875    | 12:05:20.562741         | 32 days 16:16:15
 selinuxfs                        |   600 | 1006 days 3466:41:58.91087     | 1 day 22:01:04.198185   | 173 days 16:16:15
 smbfs                            |   222 | 1053 days 2155:11:48.374153    | 4 days 27:32:45.353037  | 122 days 16:20:01
 jfs                              |   370 | 971 days 2435:07:17.805929     | 2 days 21:33:54.696773  | 160 days 16:16:15
 iso9660                          |   651 | 792 days 4698:41:27.153665     | 1 day 12:24:56.90807    | 84 days 16:06:15
 subfs                            |   611 | 736 days 3482:38:06.342656     | 1 day 10:36:35.558662   | 122 days 16:20:01
 devfs                            |    51 | 713 days 543:13:20.51142       | 13 days 34:10:50.99042  | 155 days 16:16:15
 supermount                       |   260 | 437 days 1281:43:13.500606     | 1 day 21:16:05.359618   | 68 days 16:16:15
 cifs                             |   506 | 222 days 2997:17:59.130448     | 16:27:11.381681         | 35 days 16:16:15
 openpromfs                       |    53 | 296 days 631:17:42.10027       | 5 days 25:56:56.266043  | 43 days 16:16:15
 fuse                             |   390 | 163 days 2443:31:37.846769     | 16:17:46.404735         | 17 days 16:16:15
 squashfs                         |    62 | 198 days 622:55:15.099784      | 3 days 14:41:32.179029  | 34 days 16:16:15
 configfs                         |    45 | 160 days 504:05:40.811143      | 3 days 24:32:07.573581  | 23 days 16:16:15
 capifs                           |    67 | 135 days 569:04:11.286495      | 2 days 08:51:06.437112  | 39 days 16:16:15
 minix                            |    35 | 128 days 428:57:59             | 3 days 28:01:39.40      | 16 days 16:16:15
 cramfs                           |     6 | 140 days 69:58:03.269142       | 23 days 19:39:40.544857 | 58 days 16:16:15
 udf                              |    15 | 102 days 161:35:54.111551      | 6 days 29:58:23.607437  | 51 days 16:16:15
 debugfs                          |   314 | 39 days 1144:50:09.253475      | 06:37:36.717368         | 11 days 16:14:23.049312
 ufs                              |   146 | 24 days 884:40:35.839055       | 10:00:16.683829         | 10 days 16:16:11.062917
 usbdevfs                         |   139 | 1076:04:34.054012              | 07:44:29.597511         | 19:39:58
 shm                              |   139 | 1076:04:34.054012              | 07:44:29.597511         | 19:39:58
 hfsplus                          |    80 | 14 days 480:38:46.186027       | 10:12:29.077325         | 2 days 23:36:48
 unionfs                          |    23 | 24 days 129:38:06              | 1 day 06:40:47.217391   | 22 days 16:16:15
 ocfs2_dlmfs                      |     1 | 23 days 16:16:15               | 23 days 16:16:15        | 23 days 16:16:15
 oprofilefs                       |     1 | 23 days 16:16:08.088967        | 23 days 16:16:08.088967 | 23 days 16:16:08.088967
 nfs4                             |     5 | 20 days 71:36:00               | 4 days 14:19:12         | 12 days 16:16:15
 nnpfs                            |     9 | 18 days 78:16:50               | 2 days 08:41:52.222222  | 8 days 16:16:15
 afs                              |     1 | 19 days 16:16:15               | 19 days 16:16:15        | 19 days 16:16:15
 securityfs                       |     9 | 4 days 22:13:22                | 13:08:09.111111         | 4 days 16:16:15
 ufsd                             |    15 | 75:38:57                       | 05:02:35.80             | 23:01:05
 vmware-hgfs                      |     1 | 1 day 04:56:21                 | 1 day 04:56:21          | 1 day 04:56:21
 shfs                             |     2 | 14:46:00                       | 07:23:00                | 11:27:35
(47 rows)

Now let's eliminate from the screen all sessions that rebooted (or
crashed) in less than one day:

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where uptime > '1 day' group by f order by sum(uptime) desc;
                f                 | count |              sum              |           avg           |           max           
----------------------------------+-------+-------------------------------+-------------------------+-------------------------
 sysfs                            |  5385 | 35138 days 80491:02:11.763381 | 6 days 27:33:04.202742  | 173 days 16:16:15
 tmpfs                            |  5344 | 35025 days 79890:55:31.804764 | 6 days 28:14:51.192329  | 173 days 16:16:15
 usbfs                            |  4681 | 26681 days 69552:08:46.865054 | 5 days 31:39:17.301189  | 173 days 16:16:15
 reiserfs                         |  2959 | 19257 days 44003:36:16.956207 | 6 days 27:03:42.161864  | 155 days 16:16:15
 ext3                             |  2520 | 16346 days 37501:47:54.627011 | 6 days 26:33:28.283582  | 173 days 16:16:15
 nfs                              |   645 | 6373 days 10050:19:24.173861  | 9 days 36:43:00.409572  | 86 days 16:16:15
 ext2                             |  1012 | 5629 days 15173:13:30.06032   | 5 days 28:29:14.555396  | 139 days 14:54:11
 binfmt_misc                      |   806 | 5421 days 12168:47:27.060755  | 6 days 32:31:01.59685   | 173 days 16:16:15
 nfsd                             |   634 | 4727 days 9800:42:01.958542   | 7 days 26:23:54.892679  | 89 days 16:16:15
 xfs                              |   607 | 4067 days 9311:21:11.733095   | 6 days 32:08:38.075343  | 139 days 14:54:11
 rpc_pipefs                       |   445 | 3078 days 6789:18:13.704195   | 6 days 37:15:40.884729  | 173 days 16:16:15
 vfat                             |   669 | 2470 days 9424:11:09.850314   | 3 days 30:41:48.624589  | 68 days 16:16:15
 ntfs                             |   636 | 2382 days 9081:10:26.573867   | 3 days 32:09:55.324802  | 68 days 16:16:15
 autofs                           |   405 | 2374 days 6335:32:22.311385   | 5 days 36:19:29.240275  | 54 days 16:16:15
 reiser4                          |   509 | 1926 days 7617:09:58.618041   | 3 days 33:46:41.961921  | 28 days 16:16:15
^^^^^^^^^^
 smbfs                            |    96 | 1053 days 1476:55:37.466481   | 10 days 38:38:04.765276 | 122 days 16:20:01
 selinuxfs                        |    97 | 1006 days 1584:22:47.021437   | 10 days 25:14:27.701252 | 173 days 16:16:15
 jfs                              |    71 | 971 days 1070:04:09.186991    | 13 days 31:17:48.298408 | 160 days 16:16:15
 ramfs                            |   241 | 817 days 3394:18:10.874752    | 3 days 23:26:42.8667    | 32 days 16:16:15
 iso9660                          |    85 | 792 days 1321:54:28.689609    | 9 days 23:10:31.396348  | 84 days 16:06:15
 subfs                            |    89 | 736 days 1346:31:28.394797    | 8 days 21:36:05.038144  | 122 days 16:20:01
 devfs                            |    24 | 713 days 399:55:43.321687     | 29 days 33:39:49.30507  | 155 days 16:16:15
 supermount                       |    34 | 437 days 466:58:54.119926     | 12 days 34:12:19.238821 | 68 days 16:16:15
 openpromfs                       |    29 | 296 days 515:10:26.10027      | 10 days 22:43:48.486216 | 43 days 16:16:15
 cifs                             |    39 | 222 days 622:53:46.261055     | 5 days 32:35:13.493873  | 35 days 16:16:15
 squashfs                         |    23 | 198 days 410:38:26            | 8 days 32:27:45.478261  | 34 days 16:16:15
 fuse                             |    36 | 163 days 534:27:25.006209     | 4 days 27:30:45.694617  | 17 days 16:16:15
 configfs                         |    23 | 160 days 344:26:41.378742     | 6 days 37:55:56.581684  | 23 days 16:16:15
 minix                            |    24 | 128 days 393:11:05            | 5 days 24:22:57.708333  | 16 days 16:16:15
 cramfs                           |     4 | 140 days 66:29:40.119926      | 35 days 16:37:25.029981 | 58 days 16:16:15
 capifs                           |    11 | 135 days 170:13:33            | 12 days 22:01:13.909091 | 39 days 16:16:15
 udf                              |     4 | 102 days 64:26:53             | 25 days 28:06:43.25     | 51 days 16:16:15
 debugfs                          |    11 | 39 days 182:23:03.211977      | 3 days 29:40:16.655634  | 11 days 16:14:23.049312
 ufs                              |    11 | 24 days 151:36:11.289322      | 2 days 18:08:44.662666  | 10 days 16:16:11.062917
 unionfs                          |     2 | 24 days 39:53:03              | 12 days 19:56:31.50     | 22 days 16:16:15
 ocfs2_dlmfs                      |     1 | 23 days 16:16:15              | 23 days 16:16:15        | 23 days 16:16:15
 oprofilefs                       |     1 | 23 days 16:16:08.088967       | 23 days 16:16:08.088967 | 23 days 16:16:08.088967
 nfs4                             |     4 | 20 days 62:33:56              | 5 days 15:38:29         | 12 days 16:16:15
 nnpfs                            |     4 | 18 days 73:50:18              | 4 days 30:27:34.50      | 8 days 16:16:15
 afs                              |     1 | 19 days 16:16:15              | 19 days 16:16:15        | 19 days 16:16:15
 hfsplus                          |    11 | 14 days 109:35:09             | 1 day 16:30:28.090909   | 2 days 23:36:48
 securityfs                       |     1 | 4 days 16:16:15               | 4 days 16:16:15         | 4 days 16:16:15
 vmware-hgfs                      |     1 | 1 day 04:56:21                | 1 day 04:56:21          | 1 day 04:56:21
(43 rows)

Now let's eliminate everything that rebooted or crashed in less than 10 days:

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where uptime > '10 day' group by f order by sum(uptime) desc;
                f                 | count |              sum              |           avg           |           max           
----------------------------------+-------+-------------------------------+-------------------------+-------------------------
 sysfs                            |   948 | 22969 days 15224:50:33.717548 | 24 days 21:33:13.073542 | 173 days 16:16:15
 tmpfs                            |   949 | 22960 days 15240:45:20.729486 | 24 days 20:42:47.250505 | 173 days 16:16:15
 usbfs                            |   689 | 16015 days 11048:34:25.54762  | 23 days 21:53:15.450722 | 173 days 16:16:15
 reiserfs                         |   516 | 12330 days 8297:16:33.102261  | 23 days 37:34:06.110663 | 155 days 16:16:15
 ext3                             |   453 | 10925 days 7258:04:40.240741  | 24 days 18:49:48.698103 | 173 days 16:16:15
 nfs                              |   193 | 4902 days 3138:54:10.459336   | 25 days 25:50:19.950567 | 86 days 16:16:15
 binfmt_misc                      |   154 | 3581 days 2484:52:11.438999   | 23 days 22:12:48.385968 | 173 days 16:16:15
 ext2                             |   139 | 3359 days 2218:31:49.225427   | 24 days 19:55:54.742629 | 139 days 14:54:11
 nfsd                             |   135 | 3226 days 2186:35:50.40609    | 23 days 37:42:29.262267 | 89 days 16:16:15
 xfs                              |   116 | 2666 days 1848:31:20.130065   | 22 days 39:31:18.276983 | 139 days 14:54:11
 rpc_pipefs                       |    84 | 2034 days 1356:31:00.312955   | 24 days 21:17:30.718011 | 173 days 16:16:15
 autofs                           |    69 | 1366 days 1095:10:03.052435   | 19 days 35:00:08.73989  | 54 days 16:16:15
 ntfs                             |    56 | 1058 days 905:01:16.41019     | 18 days 37:35:22.793039 | 68 days 16:16:15
 vfat                             |    52 | 1060 days 846:41:12.168038    | 20 days 25:30:47.541693 | 68 days 16:16:15
 reiser4                          |    53 | 820 days 859:44:03.65926      | 15 days 27:32:31.767156 | 28 days 16:16:15
^^^^^^^^^^^^
 jfs                              |    21 | 837 days 336:58:57            | 39 days 36:37:05.571429 | 160 days 16:16:15
 smbfs                            |    30 | 815 days 475:06:57.099919     | 27 days 19:50:13.903331 | 122 days 16:20:01
 selinuxfs                        |    28 | 767 days 454:42:21.021437     | 27 days 25:40:05.03648  | 173 days 16:16:15
 devfs                            |    14 | 686 days 233:05:03.154639     | 49 days 16:38:55.939617 | 155 days 16:16:15
 iso9660                          |    24 | 596 days 386:04:44.125293     | 24 days 36:05:11.838554 | 84 days 16:06:15
 subfs                            |    20 | 542 days 306:50:12.359485     | 27 days 17:44:30.617974 | 122 days 16:20:01
 supermount                       |    13 | 383 days 210:20:37.087554     | 29 days 27:15:25.929812 | 68 days 16:16:15
 ramfs                            |    17 | 327 days 261:34:11.310952     | 19 days 21:02:00.66535  | 32 days 16:16:15
 openpromfs                       |    13 | 238 days 211:24:12.08448      | 18 days 23:38:47.083422 | 43 days 16:16:15
 squashfs                         |     8 | 142 days 130:10:00            | 17 days 34:16:15        | 34 days 16:16:15
 cramfs                           |     3 | 137 days 48:48:44.087554      | 45 days 32:16:14.695851 | 58 days 16:16:15
 cifs                             |     6 | 115 days 97:37:30             | 19 days 20:16:15        | 35 days 16:16:15
 capifs                           |     4 | 116 days 65:05:00             | 29 days 16:16:15        | 39 days 16:16:15
 configfs                         |     5 | 95 days 80:52:24.077093       | 19 days 16:10:28.815419 | 23 days 16:16:15
 udf                              |     3 | 96 days 48:10:38              | 32 days 16:03:32.666667 | 51 days 16:16:15
 fuse                             |     6 | 85 days 93:19:29              | 14 days 19:33:14.833333 | 17 days 16:16:15
 minix                            |     3 | 42 days 48:48:45              | 14 days 16:16:15        | 16 days 16:16:15
 ocfs2_dlmfs                      |     1 | 23 days 16:16:15              | 23 days 16:16:15        | 23 days 16:16:15
 oprofilefs                       |     1 | 23 days 16:16:08.088967       | 23 days 16:16:08.088967 | 23 days 16:16:08.088967
 unionfs                          |     1 | 22 days 16:16:15              | 22 days 16:16:15        | 22 days 16:16:15
 afs                              |     1 | 19 days 16:16:15              | 19 days 16:16:15        | 19 days 16:16:15
 nfs4                             |     1 | 12 days 16:16:15              | 12 days 16:16:15        | 12 days 16:16:15
 debugfs                          |     1 | 11 days 16:14:23.049312       | 11 days 16:14:23.049312 | 11 days 16:14:23.049312
 ufs                              |     1 | 10 days 16:16:11.062917       | 10 days 16:16:11.062917 | 10 days 16:16:11.062917
(39 rows)

klive=> 

Now let's eliminate everything that rebooted or crashed in less than 20 days:

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where uptime > '20 day' group by f order by sum(uptime) desc;
                f                 | count |             sum              |           avg           |           max           
----------------------------------+-------+------------------------------+-------------------------+-------------------------
 sysfs                            |   425 | 15940 days 6844:38:13.264145 | 37 days 28:14:46.337092 | 173 days 16:16:15
 tmpfs                            |   425 | 15908 days 6844:19:50.221708 | 37 days 26:26:18.329933 | 173 days 16:16:15
 usbfs                            |   266 | 10341 days 4258:02:54.770525 | 38 days 37:01:48.927709 | 173 days 16:16:15
 reiserfs                         |   236 | 8553 days 3809:00:57.232965  | 36 days 21:56:11.428953 | 155 days 16:16:15
 ext3                             |   197 | 7465 days 3163:26:12.784865  | 37 days 37:29:58.846624 | 173 days 16:16:15
 nfs                              |   127 | 4029 days 2066:09:04.104158  | 31 days 33:39:17.04019  | 86 days 16:16:15
 binfmt_misc                      |    63 | 2318 days 1020:40:19.272961  | 36 days 35:14:55.544015 | 173 days 16:16:15
 nfsd                             |    63 | 2243 days 1023:44:21.136625  | 35 days 30:43:33.668835 | 89 days 16:16:15
 ext2                             |    53 | 2220 days 827:52:02.580825   | 41 days 36:54:11.36945  | 139 days 14:54:11
 xfs                              |    42 | 1665 days 662:12:10.297862   | 39 days 31:11:43.10233  | 139 days 14:54:11
 rpc_pipefs                       |    30 | 1286 days 486:09:06.068985   | 42 days 37:00:18.202299 | 173 days 16:16:15
 autofs                           |    23 | 756 days 374:13:45           | 32 days 37:08:25.434783 | 54 days 16:16:15
 jfs                              |    10 | 677 days 158:20:12           | 67 days 32:38:01.20     | 160 days 16:16:15
 vfat                             |    20 | 636 days 326:41:41.13894     | 31 days 35:32:05.056947 | 68 days 16:16:15
 smbfs                            |    15 | 620 days 231:13:12.099919    | 41 days 23:24:52.806661 | 122 days 16:20:01
 devfs                            |     9 | 609 days 151:46:53.100264    | 67 days 32:51:52.566696 | 155 days 16:16:15
 ntfs                             |    20 | 584 days 325:04:00.023183    | 29 days 21:03:12.001159 | 68 days 16:16:15
 selinuxfs                        |     9 | 485 days 145:33:36.021437    | 53 days 37:30:24.002382 | 173 days 16:16:15
 iso9660                          |    13 | 463 days 207:52:24           | 35 days 30:45:34.153846 | 84 days 16:06:15
 subfs                            |    10 | 403 days 144:54:16.30619     | 40 days 21:41:25.630619 | 122 days 16:20:01
 supermount                       |     9 | 325 days 145:38:07.087554    | 36 days 18:50:54.120839 | 68 days 16:16:15
 ramfs                            |    10 | 249 days 147:43:34.143084    | 24 days 36:22:21.414308 | 32 days 16:16:15
 reiser4                          |    10 | 240 days 161:33:44.187484    | 24 days 16:09:22.418748 | 28 days 16:16:15
^^^^^^^^^^^^^
 cramfs                           |     3 | 137 days 48:48:44.087554     | 45 days 32:16:14.695851 | 58 days 16:16:15
 openpromfs                       |     5 | 130 days 81:21:15            | 26 days 16:16:15        | 43 days 16:16:15
 capifs                           |     3 | 102 days 48:48:45            | 34 days 16:16:15        | 39 days 16:16:15
 udf                              |     3 | 96 days 48:10:38             | 32 days 16:03:32.666667 | 51 days 16:16:15
 cifs                             |     3 | 81 days 48:48:45             | 27 days 16:16:15        | 35 days 16:16:15
 squashfs                         |     3 | 79 days 48:48:45             | 26 days 24:16:15        | 34 days 16:16:15
 configfs                         |     3 | 64 days 48:28:45             | 21 days 24:09:35        | 23 days 16:16:15
 ocfs2_dlmfs                      |     1 | 23 days 16:16:15             | 23 days 16:16:15        | 23 days 16:16:15
 oprofilefs                       |     1 | 23 days 16:16:08.088967      | 23 days 16:16:08.088967 | 23 days 16:16:08.088967
 unionfs                          |     1 | 22 days 16:16:15             | 22 days 16:16:15        | 22 days 16:16:15
(33 rows)

klive=> 

Currently the number of reiser4 users is 35, this lists only the
kernels running in real time as I write this that are running reiser4:

klive=> select f, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where last + '1 day' > 'now' group by f order by sum(uptime) desc;
                f                 | count |             sum             |           avg           |           max           
----------------------------------+-------+-----------------------------+-------------------------+-------------------------
 sysfs                            |   484 | 3928 days 4530:39:11.52799  | 8 days 12:08:15.76762   | 173 days 16:16:15
 tmpfs                            |   480 | 3927 days 4501:01:21.406313 | 8 days 13:43:37.669596  | 173 days 16:16:15
 usbfs                            |   437 | 2903 days 3824:39:39.983599 | 6 days 24:11:04.485088  | 173 days 16:16:15
 ext3                             |   279 | 2053 days 2355:05:46.852916 | 7 days 17:02:36.08191   | 173 days 16:16:15
 reiserfs                         |   230 | 1949 days 2245:30:53.639863 | 8 days 21:08:13.276695  | 155 days 16:16:15
 xfs                              |    58 | 624 days 587:32:09.569396   | 10 days 28:20:12.578783 | 139 days 14:54:11
 ext2                             |    91 | 608 days 933:47:49.912624   | 6 days 26:36:47.361677  | 139 days 14:54:11
 binfmt_misc                      |    58 | 600 days 619:39:09.233036   | 10 days 18:57:34.297121 | 173 days 16:16:15
 nfsd                             |    32 | 546 days 420:58:29.048813   | 17 days 14:39:19.657775 | 89 days 16:16:15
 jfs                              |    10 | 546 days 143:40:39          | 54 days 28:46:03.90     | 160 days 16:16:15
 nfs                              |    28 | 518 days 380:48:49.210675   | 18 days 25:36:01.757524 | 64 days 16:16:15
 rpc_pipefs                       |    25 | 473 days 347:10:34.048813   | 18 days 35:58:01.361953 | 173 days 16:16:15
 autofs                           |    17 | 211 days 218:50:13.090969   | 12 days 22:45:18.417116 | 54 days 16:16:15
 selinuxfs                        |    10 | 204 days 105:21:47          | 20 days 20:08:10.70     | 173 days 16:16:15
 devfs                            |     3 | 193 days 48:35:34.022351    | 64 days 24:11:51.340784 | 155 days 16:16:15
 ntfs                             |    74 | 93 days 459:41:24.880971    | 1 day 12:22:27.092986   | 30 days 16:16:15
 vfat                             |    64 | 83 days 437:49:49.197282    | 1 day 13:57:57.956208   | 37 days 16:16:15
 iso9660                          |    16 | 90 days 126:40:31.377898    | 5 days 22:55:01.961119  | 75 days 13:45:31
 smbfs                            |     7 | 72 days 122:03:20           | 10 days 24:17:37.142857 | 27 days 16:16:15
 reiser4                          |    35 | 56 days 314:41:54.194701    | 1 day 23:23:28.976991   | 24 days 16:16:15
^^^^^^^^^^^^^^^
 udf                              |     1 | 51 days 16:16:15            | 51 days 16:16:15        | 51 days 16:16:15
 ramfs                            |    11 | 45 days 82:10:59.027323     | 4 days 09:39:10.820666  | 30 days 16:16:15
 configfs                         |     2 | 24 days 25:15:41            | 12 days 12:37:50.50     | 22 days 16:06:15
 squashfs                         |     1 | 21 days 16:16:15            | 21 days 16:16:15        | 21 days 16:16:15
 subfs                            |     2 | 20 days 18:08:18.063223     | 10 days 09:04:09.031611 | 20 days 16:15:43.063223
 cifs                             |     6 | 15 days 66:54:31            | 2 days 23:09:05.166667  | 11 days 16:16:15
 openpromfs                       |     3 | 14 days 61:21:33.08448      | 4 days 36:27:11.02816   | 11 days 16:09:12.08448
 fuse                             |     4 | 10 days 34:01:09            | 2 days 20:30:17.25      | 8 days 16:16:15
 debugfs                          |     6 | 9 days 46:25:01             | 1 day 19:44:10.166667   | 8 days 16:16:15
 minix                            |     1 | 8 days 16:16:15             | 8 days 16:16:15         | 8 days 16:16:15
 hfsplus                          |     1 | 1 day 12:20:26              | 1 day 12:20:26          | 1 day 12:20:26
 ufs                              |     7 | 10:55:17                    | 01:33:36.714286         | 04:18:01
 supermount                       |     2 | 03:14:39                    | 01:37:19.50             | 01:52:35
 securityfs                       |     1 | 00:38:07                    | 00:38:07                | 00:38:07
(34 rows)

klive=> 

And these are the respective kernel versions:

klive=> select kernel_release, count(*), sum(uptime), avg(uptime), max(uptime) from fs natural inner join klive where last + '1 day' > 'now' and f = 'reiser4' group by kernel_release order by sum(uptime) desc;
     kernel_release      | count |          sum           |          avg           |          max           
-------------------------+-------+------------------------+------------------------+------------------------
 2.6.17-ckpp1            |     1 | 24 days 16:16:15       | 24 days 16:16:15       | 24 days 16:16:15
 2.6.17-beyond2          |     3 | 9 days 61:26:13.021287 | 3 days 20:28:44.340429 | 6 days 16:13:52.021287
 2.6.15-gentoo-r1-blip-1 |     1 | 10 days 16:16:15       | 10 days 16:16:15       | 10 days 16:16:15
 2.6.16-gentoo-r12       |     1 | 6 days 16:06:15        | 6 days 16:06:15        | 6 days 16:06:15
 2.6.17-beyond1.1        |     1 | 3 days 17:41:00        | 3 days 17:41:00        | 3 days 17:41:00
 2.6.16-reiser4-r7       |     1 | 2 days 08:58:55.041969 | 2 days 08:58:55.041969 | 2 days 08:58:55.041969
 2.6.18-rc1-mm2-non14    |     1 | 1 day 21:35:33         | 1 day 21:35:33         | 1 day 21:35:33
 2.6.17-no4              |     1 | 1 day 04:56:21         | 1 day 04:56:21         | 1 day 04:56:21
 2.6.17-beyond2.1        |     2 | 24:53:40               | 12:26:50               | 23:01:05
 2.6.16-beyond4.1        |     1 | 22:51:05               | 22:51:05               | 22:51:05
 2.6.17-PRX5.1           |     2 | 18:16:52               | 09:08:26               | 18:16:52
 2.6.18-rc1-mm2          |     1 | 18:16:52               | 18:16:52               | 18:16:52
 2.6.17-gentoo-r3        |     3 | 11:02:11               | 03:40:43.666667        | 07:05:39
 2.6.17-rc1-no2          |     2 | 10:24:03.045926        | 05:12:01.522963        | 07:05:39
 2.6.17-beyond1          |     1 | 09:02:04               | 09:02:04               | 09:02:04
 2.6.17-beyond-git2      |     1 | 09:02:04               | 09:02:04               | 09:02:04
 2.6.16-beyond3          |     1 | 07:05:39               | 07:05:39               | 07:05:39
 2.6.17-no1              |     3 | 06:33:04               | 02:11:01.333333        | 03:18:25
 2.6.17-beyond2-r2       |     1 | 05:22:31               | 05:22:31               | 05:22:31
 2.6.17-gentoo-r4        |     1 | 04:18:01               | 04:18:01               | 04:18:01
 2.6.16-beyond4          |     3 | 03:19:24.068344        | 01:06:28.022781        | 03:09:24.068344
 2.6.17-reiser4-r4       |     2 | 00:57:37.017176        | 00:28:48.508588        | 00:57:37.017176
 2.6.16-reiser4-r11      |     1 | 00:00:00               | 00:00:00               | 00:00:00
(23 rows)

klive=> 

This is the whole list of kernel_releases that ever run reiser4:
           kernel_release            | count |              sum              |           avg            |           max
-------------------------------------+-------+-------------------------------+--------------------------+--------------------------
 2.6.15-gentoo-r1                    | 15726 | 12498 days 94186:27:26.881778 | 25:03:46.27794           | 139 days 14:54:11
 2.6.15.6                            |   262 | 6192 days 4014:29:48.915128   | 23 days 30:31:43.011126  | 114 days 16:16:15
 2.6.16-gentoo-r7                    |  5893 | 3852 days 34120:33:11.729484  | 21:28:40.005384          | 35 days 16:11:20.052466
 2.6.16-gentoo-r9                    |  7349 | 3505 days 39841:34:06.868008  | 16:52:04.132109          | 45 days 16:16:15
 2.6.14-gentoo-r5                    |  3363 | 3774 days 25871:49:28.023204  | 1 day 10:37:34.22778     | 85 days 16:16:15
 2.6.16-rc5                          |  1309 | 3909 days 10877:31:17.420649  | 2 days 31:58:47.179084   | 42 days 16:16:15
 2.6.15-gentoo-r7                    |  4183 | 2741 days 25823:28:50.761387  | 21:53:59.811322          | 48 days 16:10:16.072702
 2.6.16-gentoo-r8                    |  3300 | 2737 days 21674:06:24.528491  | 26:28:24.116524          | 55 days 16:16:15
 2.6.16-gentoo-r3                    |  4780 | 2369 days 26106:57:10.720939  | 17:21:22.558728          | 91 days 16:16:15
 2.6.16-gentoo-r2                    |  2303 | 2600 days 17412:40:17.134521  | 1 day 10:39:21.449038    | 89 days 16:16:15
 2.6.15.1                            |  1463 | 2818 days 10447:34:09.563652  | 1 day 29:22:10.177419    | 136 days 16:16:15
 2.6.16-gentoo-r1                    |  3608 | 2233 days 21934:44:07.471825  | 20:55:59.270364          | 64 days 16:03:18.045239
 2.6.16                              |  1165 | 2508 days 9354:44:26.049247   | 2 days 11:41:48.382875   | 53 days 16:06:15
 2.6.16-gentoo-r6                    |  2075 | 2300 days 13980:21:11.200159  | 1 day 09:20:23.745157    | 58 days 16:16:15
 2.6.15-gentoo-r5                    |  3321 | 1909 days 17474:22:48.504908  | 19:03:27.458147          | 104 days 16:15:50.041517
 2.6.17-gentoo                       |  3291 | 1753 days 19508:59:15.100422  | 18:42:42.97633           | 30 days 16:16:15
 2.6.17-rc1                          |   785 | 2053 days 8234:01:41.533576   | 2 days 25:15:21.912782   | 69 days 16:16:15
 2.6.15                              |  2206 | 1673 days 14112:53:10.800042  | 24:35:55.571532          | 45 days 16:16:15
 2.6.15-gentoo                       |  3284 | 1260 days 20322:09:18.149661  | 15:23:47.453761          | 26 days 16:16:15
 2.6.16-beyond4                      |  2123 | 1268 days 15252:17:49.21025   | 21:31:07.484319          | 21 days 16:16:15
 2.6.16-archck1                      |  1330 | 1388 days 10490:30:49.827973  | 1 day 08:56:03.195359    | 36 days 16:16:15
 2.6.15.4                            |   302 | 1683 days 2958:42:00.940999   | 5 days 23:32:43.314374   | 80 days 16:16:15
 2.6.12-12mdk                        |   360 | 1630 days 3696:44:27.785247   | 4 days 22:56:07.410515   | 84 days 16:06:15
 2.6.15-ck3-r1                       |   769 | 1364 days 6171:30:41.606372   | 1 day 26:35:41.796627    | 43 days 16:16:15
 2.6.15-nitro3                       |  1709 | 1142 days 10979:07:40.254166  | 22:27:42.293888          | 31 days 16:16:15
 2.6.16-beyond3                      |  1806 | 1030 days 13334:56:02.827243  | 21:04:17.011532          | 22 days 16:14:05.037859
 2.6.15-ck7                          |   637 | 1357 days 5372:37:06.322867   | 2 days 11:33:41.07743    | 31 days 16:16:15
 2.6.9-22.0.1.ELsmp                  |    13 | 1571 days 211:31:15           | 120 days 36:34:42.692308 | 173 days 16:16:15
 2.6.15-archck                       |  2579 | 865 days 15948:06:33.240877   | 14:14:00.478186          | 28 days 16:16:15
 2.6.15-1-k7                         |   164 | 1376 days 2179:27:47.995174   | 8 days 22:39:18.95119    | 35 days 16:16:15
 2.6.16-ck11                         |  1402 | 1016 days 10372:57:34.350942  | 24:47:27.542333          | 39 days 16:14:38.079596
 2.6.16-gentoo                       |  1927 | 942 days 11516:24:54.920025   | 17:42:30.853617          | 47 days 16:16:15
 2.6.14.5                            |   169 | 1318 days 1681:33:11          | 7 days 29:07:17.816568   | 86 days 16:16:15
 2.6.16-ck10                         |   814 | 1130 days 5987:58:35.725072   | 1 day 16:40:23.483692    | 40 days 16:16:15
 2.6.14-hardened-r5                  |   636 | 1194 days 4394:03:05.986415   | 1 day 27:57:55.76413     | 52 days 16:16:15
 2.6.12-gentoo-r10                   |   300 | 1213 days 3756:26:28.092538   | 4 days 13:33:41.293642   | 89 days 16:16:15
 2.6.17                              |  1038 | 918 days 10628:20:44.642394   | 31:27:52.875378          | 32 days 16:04:49.04156
 2.6.16-ck3                          |  1043 | 1053 days 6467:18:14.426646   | 1 day 06:25:50.809613    | 54 days 16:16:15
 2.6.16.1                            |   314 | 1161 days 3632:43:26.24459    | 3 days 28:18:28.937085   | 67 days 16:16:15
 2.6.15-archck2                      |  1738 | 727 days 12587:25:00.214091   | 17:16:53.751562          | 14 days 16:16:15
 2.6.13-gentoo-r5                    |   880 | 833 days 9688:10:28.922616    | 33:43:38.896503          | 34 days 16:16:15
 2.6.15-ck2pwr                       |    92 | 1127 days 1357:25:27.655134   | 12 days 20:45:16.604947  | 52 days 16:16:15
[..]

too long to list them all.

> "patchup" is used by (tens of) thousands of computers in governmental
> laboratories the US "national security" depends upon.

I don't see any ext4 in the klive logs, it'd be nice if I would know
how to track your ext4 patchset too to make a more accurate
comparison. We can't compare the above numbers to the klive ones, the
above is the interpolation of the reiser4 and klive userbase
only. While if I could detect the ext4 "patchup" it would be feasible
to attempt a comparison.

Disclaimer: the data should not be trusted, use it at your
risk. Furthermore it may not represent a reliable sample, because the
fs are only submitted at the first connection with the server after
reboot, so if reiser4 or fuse are mounted later on (normally after an
average of 5 minutes) they may not get logged in KLive. OTOH this also
means that all computers showing reiser4 in the logs had it mounted
since about the boot time (so if reiser4 would corrupt memory badly by
just mounting nobody could reasonably hope to reach 20 days of
uptime).

Hope this helps.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Rene Rebe]

Hi,

On Tuesday 25 July 2006 14:35, Andrea Arcangeli wrote:
...
> average of 5 minutes) they may not get logged in KLive. OTOH this also
> means that all computers showing reiser4 in the logs had it mounted
> since about the boot time (so if reiser4 would corrupt memory badly by
> just mounting nobody could reasonably hope to reach 20 days of
> uptime).

Well I tested Reiser4 on systems building our Linux flavour (based on the T2 
framework) thus with excessive software compilation, a lot of tarball 
extraction, compilation of fat packages such as linux, mozilla, openoffice 
et.al. and huge per-package ccache directories -> zilions of small files and 
did not encounter any abnormal behaviour aside the yet-to-be-fixed commit 
storms. Performance was superiour over ext3, XFS*, JFS or Reiser3 in this 
workload (that is less time "wasted" for FS housekeeping and the build 
finished noticable earlier). Patchset was tested around 2.6.1{4,5]. But I do 
not have the numbers around anymore and was basically waiting for Reiser4 to 
make it into the kernel to schedule further testing.

You can see I'm not a blind Reiser4 follower but would rather like to see a 
fair chance for it.

*) XFS oopsed so often in this test that I'll never touch it again ...

Yours,

-- 
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            http://exactcode.de | http://t2-project.org | http://rebe.name
            +49 (0)30 / 255 897 45

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Wow, I would never have guessed our market share was that high as 1/5th
of ext3.  I mean, you can't even get a distro which allows you to
install onto reiser4 without hours of work so far as I know.  I guess
there are people who really do care about twice as fast.

Hans

Andrea Arcangeli wrote:

> reiserfs                         | 19934 | 19257 days 133294:48:27.256508 | 29:52:18.181361         | 155 days 16:16:15
> ext3                             | 22077 | 16346 days 132190:42:38.862752 | 23:45:27.062502         | 173 days 16:16:15
>
> reiser4                          |  4409 | 1926 days 27479:22:27.459534   | 16:42:59.666015         | 28 days 16:16:15
>^^^^^^^^^^^^^
>  
>

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Francesco Biscani]

On Tuesday 25 July 2006 19:47, Hans Reiser wrote:
> Wow, I would never have guessed our market share was that high as 1/5th
> of ext3.  I mean, you can't even get a distro which allows you to
> install onto reiser4 without hours of work so far as I know.  I guess
> there are people who really do care about twice as fast.
>
> Hans
>

Yes, this was a pleasant surprise indeed :)

This useless mail from a mere user is just to testify that I've used reiser4 
on my laptop since June 2004, and that, barring occasional hiccups, I never 
had serious problems with it. In fact I haven't lost a single bit of data 
despite crashes and cold shutdowns due to power outage.

I don't know if this means something or if I've just been lucky, but it seems 
to me that when reiser4 crashes (and sometimes it does, given its young age), 
it behaves very very well. That's the thing that impressed me the most (and 
it is really something, given for example the debacle of a mature filesystem 
like XFS in 2.6.17). Oh, and performance, of course ;)

I really hope that the technical difficulties that are preventing reiser4 from 
entering mainline will be sorted out soon.

Thanks,

  Francesco (crawling back in the shadow)

-- 
Dr. Francesco Biscani
Dipartimento di Astronomia
Università di Padova
biscani@pd.astro.it

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Hans Reiser <reiser@namesys.com> wrote:
> Wow, I would never have guessed our market share was that high as 1/5th
> of ext3.  I mean, you can't even get a distro which allows you to
> install onto reiser4 without hours of work so far as I know.  I guess
> there are people who really do care about twice as fast.

That makes the data /higly/ suspect: Most of the Linux users I know
wouldn't know where to start to compile a kernel, forget about patching
around.  And of the minority who can, they mostly run machines in
production, and won't dream of using non-distribution kernels, let alone
custom patched ones.

BTW, where do you get the "twice as fast" number from?
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jim Crilly]

On 07/25/06 05:17:48PM -0400, Horst H. von Brand wrote:
> Hans Reiser <reiser@namesys.com> wrote:
> > Wow, I would never have guessed our market share was that high as 1/5th
> > of ext3.  I mean, you can't even get a distro which allows you to
> > install onto reiser4 without hours of work so far as I know.  I guess
> > there are people who really do care about twice as fast.
> 
> That makes the data /higly/ suspect: Most of the Linux users I know
> wouldn't know where to start to compile a kernel, forget about patching
> around.  And of the minority who can, they mostly run machines in
> production, and won't dream of using non-distribution kernels, let alone
> custom patched ones.
 
Well if you take a look at the KLive webpage you'll see that there's only
485 computers providing data. I'm not trying to knock Andrea's project, but
the sample size is quite small and only really representative of people who
would be inclined to read lkml, patch their kernels, etc.

Jim.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Tue, Jul 25, 2006 at 06:48:35PM -0400, Jim Crilly wrote:
> Well if you take a look at the KLive webpage you'll see that there's only
> 485 computers providing data. I'm not trying to knock Andrea's project, but
> the sample size is quite small and only really representative of people who
> would be inclined to read lkml, patch their kernels, etc.

There are quite a few using the official distro kernels but the point
remains that the KLive users are probably the ones most interested to
try all new stuff and to live on the edge. They're not afraid to put
the new technologies to work for them. That's probably why they're
using reiser4 so much. OTOH they're probably the ones putting the fs
under the most stress, a regular desktop user not capable of running
klive.sh --install from the shell, would probably leave his CPU and
harddisk idle most of the time too.

Being this only a sample I can't come up with absolute figures, but
obviously there are more users than what is being recorded by KLive.

However the sample is not so small, in less then a year KLive logged
76740 days of uptime and 61355 reboots (or klive session restarts),
and like you noticed an average of about 400-500 systems are always
connected.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Luigi Genoni]

I suppose from the ones, lime me,  who has production servers, but also some 
test system to play with.

Luigi
On Tuesday 25 July 2006 23:17, Horst H. von Brand wrote:
> Hans Reiser <reiser@namesys.com> wrote:
> > Wow, I would never have guessed our market share was that high as 1/5th
> > of ext3.  I mean, you can't even get a distro which allows you to
> > install onto reiser4 without hours of work so far as I know.  I guess
> > there are people who really do care about twice as fast.
>
> That makes the data /higly/ suspect: Most of the Linux users I know
> wouldn't know where to start to compile a kernel, forget about patching
> around.  And of the minority who can, they mostly run machines in
> production, and won't dream of using non-distribution kernels, let alone
> custom patched ones.
>
> BTW, where do you get the "twice as fast" number from?

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Tue, Jul 25, 2006 at 11:47:29AM -0600, Hans Reiser wrote:

> Wow, I would never have guessed our market share was that high as 1/5th
> of ext3.  I mean, you can't even get a distro which allows you to
> install onto reiser4 without hours of work so far as I know.  I guess
> there are people who really do care about twice as fast.

I doubt Andreas values have much value.

According to the numbers on the klive website, Gentoo has 47 times as 
many users as SuSE (sic).

Considering that klive is offered by someone working for SuSE, the 
Gentoo project could make a good news article ("Numbers by a SuSE 
developer confirm Gentoo has 47 times the market share of Suse.")
out of it.  ;-)

> Hans

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Wed, Jul 26, 2006 at 02:45:57PM +0200, Adrian Bunk wrote:
> On Tue, Jul 25, 2006 at 11:47:29AM -0600, Hans Reiser wrote:
> 
> > Wow, I would never have guessed our market share was that high as 1/5th
> > of ext3.  I mean, you can't even get a distro which allows you to
> > install onto reiser4 without hours of work so far as I know.  I guess
> > there are people who really do care about twice as fast.
> 
> I doubt Andreas values have much value.

I think nobody ever pretended them to have much value, they only need
to have some minor value to be useful. And as far as I can tell no
better source of data about the kernel testing userbase exists as of
today.

> According to the numbers on the klive website, Gentoo has 47 times as 
> many users as SuSE (sic).
> 
> Considering that klive is offered by someone working for SuSE, the 
> Gentoo project could make a good news article ("Numbers by a SuSE 
> developer confirm Gentoo has 47 times the market share of Suse.")
> out of it.  ;-)

Oh well, you misunderstood KLive completely. Please read again the top
of the page: "this project aims to provide kernel live feedback about
the usage of every different Linux Kernel version". Where did I ever
mention anything about distributions?

There's absolutely no way to tell which distribution is running. I
only can tell which _kernel_ is running. It could be 100% of the
mainline users are running mainline on top of SUSE distro, or on top
of Gentoo, or on top of Ubuntuu, there's absolutely no way to know
about the distro. Nobody could ever make a claim like the above by
using the KLive data.

To make an example I run mainline myself on top of opensuse 10.1, and
I get rightfully accounted as a "mainline" and not as a "SUSE".

All you can tell is that there are many more people running KLive in
combination with Gentoo kernels than with SUSE kernels. But you can't
tell anything about what kind of distribution is running. One could
guess the Gentoo kernels run on top of a Gentoo userland, but for the
mainline ones you really can't tell, they could be slackware but they
could be any other distro too.

Also note that the cpushare.com domain is not related to any distro,
so the fact I also do consulting for SUSE is irrelevant to KLive or
any other dealings at the cpushare.com domain. As long there is people
running KLive and browsing the website like now, it means somebody
thinks it's useful and so I keep delivering the service.

As of now, the Gentoo and mainline kernel users are the ones providing
most of the feedback.

Perhaps I should have filed a patent on KLive too just to make you
happy, right?

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Wed, Jul 26, 2006 at 03:29:57PM +0200, andrea@cpushare.com wrote:
> On Wed, Jul 26, 2006 at 02:45:57PM +0200, Adrian Bunk wrote:
> > On Tue, Jul 25, 2006 at 11:47:29AM -0600, Hans Reiser wrote:
> > 
> > > Wow, I would never have guessed our market share was that high as 1/5th
> > > of ext3.  I mean, you can't even get a distro which allows you to
> > > install onto reiser4 without hours of work so far as I know.  I guess
> > > there are people who really do care about twice as fast.
> > 
> > I doubt Andreas values have much value.
> 
> I think nobody ever pretended them to have much value, they only need
> to have some minor value to be useful. And as far as I can tell no
> better source of data about the kernel testing userbase exists as of
> today.
>...

Hans said "Wow, I would never have guessed our market share was that 
high as 1/5th of ext3."

And estimate of the marked share based on klive data is simply wrong. 
That was my point.

My distribution example was just one example of what happens when you 
wrongly try to estimate a marked share based on klive data (and it seems 
you missed the smiley after the last paragraph of my email).

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Wed, Jul 26, 2006 at 03:43:26PM +0200, Adrian Bunk wrote:
> My distribution example was just one example of what happens when you 
> wrongly try to estimate a marked share based on klive data (and it seems 

You're again wrong generalizing. With KLive I can attempt to estimate
market share of _kernel_ code (what Hans did), but you can't attempt
to estimate market share of userland (what you did). It won't be a
smile at the end making your statement right.

The estimate market share of kernel code may not be accurate but it
has some minor significance.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Wed, Jul 26, 2006 at 04:28:54PM +0200, andrea@cpushare.com wrote:
> On Wed, Jul 26, 2006 at 03:43:26PM +0200, Adrian Bunk wrote:
> > My distribution example was just one example of what happens when you 
> > wrongly try to estimate a marked share based on klive data (and it seems 
> 
> You're again wrong generalizing. With KLive I can attempt to estimate
> market share of _kernel_ code (what Hans did), but you can't attempt
> to estimate market share of userland (what you did). It won't be a
> smile at the end making your statement right.

No, you can't estimate the market share of kernel code based on klive 
data.

Someone said in this thread 'is used by (tens of) thousands of computers 
in governmental laboratories the US "national security" depends upon.'

klive will never get any feedback from such users.

But the computer freak using Gentoo and always trying the latest things 
like reiser4 is relatively likely to also use klive.

> The estimate market share of kernel code may not be accurate but it
> has some minor significance.

You are able to say that based on klive data, reiser4 has at least 
35 users in the world.

But you can not tell based on klive data whether the ratio of 
reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Wed, Jul 26, 2006 at 04:50:19PM +0200, Adrian Bunk wrote:
> Someone said in this thread 'is used by (tens of) thousands of computers 
> in governmental laboratories the US "national security" depends upon.'

There can be people using reiser4 behind the firewall too, what's the
point? IIRC US .gov even sponsored part of reiser4 development, how do
you know they're not testing it too?

You don't believe KLive has any relation to reality, but you have no
way to proof your claim. JFYI: all statistics only take a sample of
the larger space, the whole point of having a statistic is because you
can't measure the total. The smaller the sample compared to the total,
the less the stats are accurate, but they still have some statistical
significance. And one can always hope that KLive will grow larger.

Last but not the least defining gentoo users as freaks isn't very nice
from your part. For all new startups they're the ideal userbase to
have, and they do a great deal of good work by testing all new stuff
and they help speeding up innovation. Infact I think having a sample
of what those brave users run is more important than the rest, the
rest usually follows the ones living on the edge eventually.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote:
> On Wed, Jul 26, 2006 at 04:50:19PM +0200, Adrian Bunk wrote:
> > Someone said in this thread 'is used by (tens of) thousands of computers 
> > in governmental laboratories the US "national security" depends upon.'
> 
> There can be people using reiser4 behind the firewall too, what's the
> point? IIRC US .gov even sponsored part of reiser4 development, how do
> you know they're not testing it too?

I don't know.

The klive data might be inaccurate in any direction.

In this case I'd suspect an inaccuraty in one specific direction.

> You don't believe KLive has any relation to reality, but you have no
> way to proof your claim. JFYI: all statistics only take a sample of
> the larger space, the whole point of having a statistic is because you
> can't measure the total. The smaller the sample compared to the total,
> the less the stats are accurate, but they still have some statistical
> significance. And one can always hope that KLive will grow larger.

Size isn't everything.
The problem is getting an accurate sample of data.


Let me try to make the problem clearer by making an example:

Consider you want to know the ratio between housewifes and women with a 
job for women aged between 30 and 40.

Consider you get your data by asking women in shopping malls between
10 and 11 o'clock in the morning on workdays.

Do you understand why the result might be quite different from the 
actual ratio?

Do you understand why asking a million women in shopping malls between
10 and 11 o'clock in the morning on workdays wouldn't make the data
better?


And do you know the Linux Counter at [1]?

I remember several years ago Debian had some default setting in 
some package to send reports there.

smail was at that times the default Debian MTA, and therefore also the 
most popular MTA at the Linux counter...

> Last but not the least defining gentoo users as freaks isn't very nice
> from your part. For all new startups they're the ideal userbase to
> have, and they do a great deal of good work by testing all new stuff

Please read carefully what I said.

I did NOT say:
"All Gentoo users are freaks."

It was more (implicitely) in the direction:
"Many freaks use Gentoo."

The latter is IMHO not that far from reality.

And "freak" wasn't meant negative.

> and they help speeding up innovation. Infact I think having a sample
> of what those brave users run is more important than the rest, the
> rest usually follows the ones living on the edge eventually.

The majority of user will use the filesystem their distribution offers 
as default, no matter what people living on the edge today will use...

cu
Adrian

[1] http://counter.li.org/

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[J. Bruce Fields]

On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote:
> JFYI: all statistics only take a sample of the larger space, the whole
> point of having a statistic is because you can't measure the total.
> The smaller the sample compared to the total, the less the stats are
> accurate

Definitely not true in general.  If I wanted to know the gender ratio at
the latest OLS I'd take the results from a sample of a dozen chosen
randomly over the results from a sample of hundreds all taken from the
men's room.

For exactly the same quality of sampling, yes, the larger the better,
but the point of diminishing returns comes pretty quickly.  So given
limited resources it's probably more important to work on the quality of
the sample rather than on its size....

--b.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Wed, Jul 26, 2006 at 01:02:36PM -0400, J. Bruce Fields wrote:
> On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote:
> > JFYI: all statistics only take a sample of the larger space, the whole
> > point of having a statistic is because you can't measure the total.
> > The smaller the sample compared to the total, the less the stats are
> > accurate
> 
> Definitely not true in general.  If I wanted to know the gender ratio at
> the latest OLS I'd take the results from a sample of a dozen chosen
> randomly over the results from a sample of hundreds all taken from the
> men's room.

Well, your example is perhaps the worst one since you wouldn't be
decreasing the quality of your stats very much by only doing the
sample in the men's room ;). I guess you meant the woman's room.

> For exactly the same quality of sampling, yes, the larger the better,
> but the point of diminishing returns comes pretty quickly.  So given
> limited resources it's probably more important to work on the quality of
> the sample rather than on its size....

No matter how you see it, the larger the better (in the worst case it
won't make a difference). Certainly if I could work on the quality,
that would be more important than adding 1 more user. But I can't work
on the quality.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jerome Pinot]

On 7/27/06, andrea@cpushare.com <andrea@cpushare.com> wrote:
> On Wed, Jul 26, 2006 at 01:02:36PM -0400, J. Bruce Fields wrote:
> > On Wed, Jul 26, 2006 at 06:06:04PM +0200, andrea@cpushare.com wrote:
> > > JFYI: all statistics only take a sample of the larger space, the whole
> > > point of having a statistic is because you can't measure the total.
> > > The smaller the sample compared to the total, the less the stats are
> > > accurate
> >
> > Definitely not true in general.  If I wanted to know the gender ratio at
> > the latest OLS I'd take the results from a sample of a dozen chosen
> > randomly over the results from a sample of hundreds all taken from the
> > men's room.
>
> Well, your example is perhaps the worst one since you wouldn't be
> decreasing the quality of your stats very much by only doing the
> sample in the men's room ;). I guess you meant the woman's room.
>
> > For exactly the same quality of sampling, yes, the larger the better,
> > but the point of diminishing returns comes pretty quickly.  So given
> > limited resources it's probably more important to work on the quality of
> > the sample rather than on its size....
>
> No matter how you see it, the larger the better (in the worst case it
> won't make a difference). Certainly if I could work on the quality,
> that would be more important than adding 1 more user. But I can't work
> on the quality.

Maybe you could try pushing the use of klive by some distros arguing
it's "a way of getting usage statistics in order to improve kernel
quality and hardware support". I mean, not just an extra package but a
service launch at the first boot so anyone can use it.

klive is a nice project, it just needs more _different_ users and
maybe, a support of some distros.

Maybe, you could try add a page in the klive wiki for putting packages
and RPMs of klive for several distros ? What do you think ? It's a
first step to get it merge into distros.

In my case, I got some .tgz that feel lonely...

Maybe be in 2 years, we will know EXACTLY of much people use
EXT4/Reiser5/CoincoinFS/CrashFS...

-- 
Jerome Pinot
http://ngc891.blogdns.net/

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

Removed some people from CC since we're quickly changing topic and I
doubt they're interested.

On Thu, Jul 27, 2006 at 04:01:02AM +0900, Jerome Pinot wrote:
> Maybe you could try pushing the use of klive by some distros arguing
> it's "a way of getting usage statistics in order to improve kernel
> quality and hardware support". I mean, not just an extra package but a
> service launch at the first boot so anyone can use it.
> 
> klive is a nice project, it just needs more _different_ users and
> maybe, a support of some distros.

That would be nice yes. KLive is already supported by Gentoo that
created a proper emerge package, that's why there are so many gentoo
kernels being tracked. Others are of course welcome. I'm open to
suggestions on extending it as well.

At the moment the todo list is like this:

1) create an optional http transport to pass through most firewalls
2) send a packet when system reboots so I can track which sessions
   didn't cleanly reboot (it's not reliable if you disconnect
   from the internet first and you shutdown later, but it's better
   than nothing)
3) send the oops from kernel to klive server, so no oopses will be
   lost and I can use the pciids details to track down bad hardware
   as well
4) possibly send info on usb buses too and not only about the pci buses

> Maybe, you could try add a page in the klive wiki for putting packages
> and RPMs of klive for several distros ? What do you think ? It's a
> first step to get it merge into distros.

If more distros will pickup KLive that will help like demonstrated by
the emerge package from gentoo.

It's already very simple to install klive, I wrote an universal
installer that runs on any flavour of any weird distro out there:

	wget klive.cpushare.com/klive.sh
	sh klive.sh --install

If you don't want it anymore:

       sh klive.sh --uninstall

that's it.

The rpm/deb packages would be just to make life easier for distro to
pick it up but I hoped they had the resources to do the packaging work
themself...

Perhaps once I'll create the CPUShare rpm/deb packages I'll be easy to
share the work to create the KLive ones too. As far as spare time work
goes, CPUShare currently has a much higher prio than KLive, and I'm
not yet at the point of creating rpm/deb packages, now that CPUShare
transaction works and you can already compute remotely, I'm currently
fighting with the paypal sendbox so I can start accepting cash into
the system ASAP. But the hardest part of the ebusiness side of
CPUShare is the handling of the invoicing and receipts generation
which is a non computer related problem. I almost solved it though.

> In my case, I got some .tgz that feel lonely...

The .tgz is only for the ones interested hacking or studying it (both
client and server). You should be using the .sh script above.

Also note, the export of the whole database isn't available on the
site, but I've no problem to publish it after filtering out all the ip
addresses (which are collected only for purely paranoid security
purposes, and tor doesn't route udp traffic).

> Maybe be in 2 years, we will know EXACTLY of much people use
> EXT4/Reiser5/CoincoinFS/CrashFS...

BTW, CPUShare records a lot more than the fs already. 2 years or more,
but I'm not in a hurry. The server will also soon require some caching
trick to avoid recomputing the queries every time you click on
it. Current KLive is the best I could do in a very little time I spent
on it, it works well, but it still has an huge room for
improvement. Also note, so far KLive logged 76989 cumulative days of
uptime. When I see a number like that my mind thinks at the energy
that can be recycled if all that time recorded on KLive would be
routed inside CPUShare. That's why for me it's more urgent to give a
chance to those brave 500 users to cash in with CPUShare than to push
KLive beyond its current status ;). In due time KLive will get more
attention too.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Wed, Jul 26, 2006 at 07:20:29PM +0200, andrea@cpushare.com wrote:
> On Wed, Jul 26, 2006 at 01:02:36PM -0400, J. Bruce Fields wrote:
>...
> > For exactly the same quality of sampling, yes, the larger the better,
> > but the point of diminishing returns comes pretty quickly.  So given
> > limited resources it's probably more important to work on the quality of
> > the sample rather than on its size....
> 
> No matter how you see it, the larger the better (in the worst case it
> won't make a difference). Certainly if I could work on the quality,
> that would be more important than adding 1 more user. But I can't work
> on the quality.

But depending on the nature of the error, the worst case might be the 
common case (as I've already explained in another email).

If you can't ensure the quality of your data, please don't use this data 
to wrongly draw any conclusions from them [1].

cu
Adrian

[1] the conclusion itself might or might not be true
    e.g. there _could_ be an 1:5 ratio between reiser4 and ext3 users
    but your data is not in any way able to support or reject this 
    statement

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Wed, Jul 26, 2006 at 10:50:22PM +0200, Adrian Bunk wrote:
> But depending on the nature of the error, the worst case might be the 
> common case (as I've already explained in another email).
> 
> If you can't ensure the quality of your data, please don't use this data 
> to wrongly draw any conclusions from them [1].

Please read the footer of the KLive pages:

"The use of the information and of the software in this website is at
 your own risk.  KLive probably doesn't represent a reliable sample of
 the real usage of the Linux Kernel."

Said that pretending that KLive data has absolutely no significance at
all and that you can't draw any conclusion at all from it, to me seems
as wrong as pretending it to perfect.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[J. Bruce Fields]

On Wed, Jul 26, 2006 at 11:17:41PM +0200, andrea@cpushare.com wrote:
> Said that pretending that KLive data has absolutely no significance at
> all and that you can't draw any conclusion at all from it, to me seems
> as wrong as pretending it to perfect.

This is where we disagree.  It may be wrong, but it's certainly not
nearly *as* wrong....

--b.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Wed, Jul 26, 2006 at 05:37:57PM -0400, J. Bruce Fields wrote:
> This is where we disagree.  It may be wrong, but it's certainly not
> nearly *as* wrong....

;)

My point of view in saying you can't dismiss the current KLive stats
completely, is simply that I didn't expect reiser4 numbers to be so
high even in the small sample of the brave KLive project that is full
of people willing to test new stuff (also note that I compared it to
isofs, I didn't attempt a comparison with ext3 myself). That makes it
a positive in my view.

If you think KLive numbers aren't a positive point for reiser4, then
it can only mean you expected them to be even higher than they were...

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Wed, Jul 26, 2006 at 11:17:41PM +0200, andrea@cpushare.com wrote:
> On Wed, Jul 26, 2006 at 10:50:22PM +0200, Adrian Bunk wrote:
> > But depending on the nature of the error, the worst case might be the 
> > common case (as I've already explained in another email).
> > 
> > If you can't ensure the quality of your data, please don't use this data 
> > to wrongly draw any conclusions from them [1].
> 
> Please read the footer of the KLive pages:
> 
> "The use of the information and of the software in this website is at
>  your own risk.  KLive probably doesn't represent a reliable sample of
>  the real usage of the Linux Kernel."

It was you who wrongly said:
"With KLive I can attempt to estimate market share of _kernel_ code"

Hadn't you read your own disclaimer?

> Said that pretending that KLive data has absolutely no significance at
> all and that you can't draw any conclusion at all from it, to me seems
> as wrong as pretending it to perfect.

Possibly wrong conclusions about the general market share based on data 
not having the quality for being the basis of such statements are worse 
than having no data.

Every time someone will repeat the "1:5 ratio for reiser4:ext3 users", 
this will be an additional proof it's really worse than no data.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Luigi Genoni]

On Thu, July 27, 2006 08:56, Adrian Bunk wrote:
>
>> Said that pretending that KLive data has absolutely no significance at
>> all and that you can't draw any conclusion at all from it, to me seems as
>> wrong as pretending it to perfect.
>
> Possibly wrong conclusions about the general market share based on data
> not having the quality for being the basis of such statements are worse than
> having no data.

Am I missing something, or it is not marketing what we are talking about?
please I do not understand your point.

For what I understand... I was not supposing reiser4 users to be so mutch,
but this is very usefull because this means that there are more
possibilities to test the code well and to fix bugs...

This also means that there are people who feels the need of this filesystem
for their work because they are not satisfied with the other filesystems, or
because anyway with reiser4 they see that they get better results about the
work they need to be done.
And this is probably the most important point.

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 10:33:12AM +0200, Luigi Genoni wrote:
> 
> 
> 
> On Thu, July 27, 2006 08:56, Adrian Bunk wrote:
> >
> >> Said that pretending that KLive data has absolutely no significance at
> >> all and that you can't draw any conclusion at all from it, to me seems as
> >> wrong as pretending it to perfect.
> >
> > Possibly wrong conclusions about the general market share based on data
> > not having the quality for being the basis of such statements are worse than
> > having no data.
> 
> Am I missing something, or it is not marketing what we are talking about?
> please I do not understand your point.
> 
> For what I understand... I was not supposing reiser4 users to be so mutch,
> but this is very usefull because this means that there are more
> possibilities to test the code well and to fix bugs...
>...

I'm sorry for saying it that directly, and this is not meant against you 
personally:

Your statement is a good example why this data is worse than having no 
data.

As I already said in this thread:

<--  snip  -->

You are able to say that based on klive data, reiser4 has at least 
35 users in the world.

But you can not tell based on klive data whether the ratio of 
reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000.

<--  snip  -->

I can't prove that the 1:5 ratio is wrong, but the point is that 
claiming a 1:5 ratio was true based on the klive data is not better than 
claiming it based on no data. But claiming it based on the klive data is 
worse since people like you are getting the wrong impression it was 
based on data that would have the quality for supporting such a 
statement.

The data simply has not the quality for such a statement.
Please read my two examples in [1] if you want to get an impression why 
such problems can occur.

cu
Adrian

[1] http://lkml.org/lkml/2006/7/26/203

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Luigi Genoni]

Still I am missing something.

I am not interested in discussing about 1:5 ora 5:1 or so on.
I am not even interested if reiser4 users are 35 or 35000.
But if some people felt they need reiser4 to get their work done, that means
something. The numbers and the datum per se are meaningless.

Anyway you have a datum.
Some people need reiser4, period.

Why they need reiser4?
all of them are just playing with him to have a new game for their spare time?
I don't think so.
Some of them are using reiser4 because it is the best solution for their work?
this is a concrete possibility.

again... why that?
Because the other FSs are not suitable for certain work?
Because the other FSs are suitable, but reiser4 offers better results?
Because even reiser4 is not complitelly suitable for some kind of work, but
the other FSs are a worser solution?

I don't think most reiser4 users decided to give it a try just to enjoy
their time making tests just because they are curious.

that does not mean reiser4 should be merged into linux kernel because of
this. On the other side you can agree reiser4 has its place, because there
are users who need it. What we would need to understand, sine ira nec
studio, is where this place is.

On Thu, July 27, 2006 12:04, Adrian Bunk wrote:
> On Thu, Jul 27, 2006 at 10:33:12AM +0200, Luigi Genoni wrote:
>
>>
>>
>>
>> On Thu, July 27, 2006 08:56, Adrian Bunk wrote:
>>
>>>
>>>> Said that pretending that KLive data has absolutely no significance
>>>> at all and that you can't draw any conclusion at all from it, to me
>>>> seems as wrong as pretending it to perfect.
>>>
>>> Possibly wrong conclusions about the general market share based on data
>>>  not having the quality for being the basis of such statements are
>>> worse than having no data.
>>
>> Am I missing something, or it is not marketing what we are talking about?
>>  please I do not understand your point.
>>
>> For what I understand... I was not supposing reiser4 users to be so
>> mutch, but this is very usefull because this means that there are more
>> possibilities to test the code well and to fix bugs... ...
>>
>
> I'm sorry for saying it that directly, and this is not meant against you
> personally:
>
>
> Your statement is a good example why this data is worse than having no
> data.
>
> As I already said in this thread:
>
>
> <--  snip  -->
>
>
> You are able to say that based on klive data, reiser4 has at least
> 35 users in the world.
>
>
> But you can not tell based on klive data whether the ratio of
> reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000.
>
>
> <--  snip  -->
>
>
> I can't prove that the 1:5 ratio is wrong, but the point is that
> claiming a 1:5 ratio was true based on the klive data is not better than
> claiming it based on no data. But claiming it based on the klive data is
> worse since people like you are getting the wrong impression it was based on
> data that would have the quality for supporting such a statement.
>
> The data simply has not the quality for such a statement.
> Please read my two examples in [1] if you want to get an impression why
> such problems can occur.
>
> cu Adrian
>
>
> [1] http://lkml.org/lkml/2006/7/26/203
>
>
> --
>
>
> "Is there not promise of rain?" Ling Tan asked suddenly out
> of the darkness. There had been need of rain for many days. "Only a promise,"
> Lao Er said.
> Pearl S. Buck - Dragon Seed
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>
>

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 01:07:27PM +0200, Luigi Genoni wrote:

> Still I am missing something.
> 
> I am not interested in discussing about 1:5 ora 5:1 or so on.
> I am not even interested if reiser4 users are 35 or 35000.
> But if some people felt they need reiser4 to get their work done, that means
> something. The numbers and the datum per se are meaningless.
>...

You answered to an email where I talked about the wrong assumption the 
klive data could support the 1:5 market share claim.

This is completely independent from discussions about why users might 
need reiser4, or which technical or social problems prevent its merging
(unless of course someone wants to justify its merging wrongly with the 
1:5 claim), and therefore your answer does not apply to this part of 
the thread.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Luigi Genoni]

I answered a mail about how klive data should not be took in account, and
could even be dangerous...


On Thu, July 27, 2006 13:35, Adrian Bunk wrote:
> On Thu, Jul 27, 2006 at 01:07:27PM +0200, Luigi Genoni wrote:
>
>
>> Still I am missing something.
>>
>>
>> I am not interested in discussing about 1:5 ora 5:1 or so on.
>> I am not even interested if reiser4 users are 35 or 35000.
>> But if some people felt they need reiser4 to get their work done, that
>> means something. The numbers and the datum per se are meaningless. ...
>>
>
> You answered to an email where I talked about the wrong assumption the
> klive data could support the 1:5 market share claim.
>
> This is completely independent from discussions about why users might
> need reiser4, or which technical or social problems prevent its merging
> (unless of course someone wants to justify its merging wrongly with the
> 1:5 claim), and therefore your answer does not apply to this part of
> the thread.
>
> cu Adrian
>
>
> --
>
>
> "Is there not promise of rain?" Ling Tan asked suddenly out
> of the darkness. There had been need of rain for many days. "Only a promise,"
> Lao Er said.
> Pearl S. Buck - Dragon Seed
>
>
>

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 01:43:53PM +0200, Luigi Genoni wrote:

> I answered a mail about how klive data should not be took in account, and
> could even be dangerous...

... and you talking about people who might need reiser4 was therefore 
a wrong answer.

I never said reiser4 shouldn't be merged, but this wasn't the topic of 
this subthread.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Luigi Genoni <genoni@sns.it> wrote:

[...]

> Anyway you have a datum.
> Some people need reiser4, period.

Nope. Some people run kernels that include reiser4. That is all you can
infer, and that I knew beforehand. They are at least 35, and that I'd have
guessed in any case.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[gmu 2k6]

On 7/27/06, Horst H. von Brand <vonbrand@inf.utfsm.cl> wrote:
> Luigi Genoni <genoni@sns.it> wrote:
>
> [...]
>
> > Anyway you have a datum.
> > Some people need reiser4, period.
>
> Nope. Some people run kernels that include reiser4. That is all you can
> infer, and that I knew beforehand. They are at least 35, and that I'd have
> guessed in any case.

35.5 as I'm testing it here on my workstation and it seems to be
faster when you test some things involving many copies of large
multi-level sourcetree directories each 3 to 6GiB big in size.
2.6.18-rc2-mm1 with Reiser4 looks ok so far and I had no sync() OOPS
like the last time with one -mm revision.

speed tells us nothing about reliability of course, but compared to
ext3 with dir_index,sparse_super Reiser4 seems to handle "du -sh" and
"rm -r" much faster and without eating all of the CPU cycles as it
finishes quicker although Reiser4 was meant to be CPU-heavy compared
to ext*/reiserfs3.

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 09:30:36AM -0400, Horst H. von Brand wrote:
> Nope. Some people run kernels that include reiser4. That is all you can
> infer, and that I knew beforehand. They are at least 35, and that I'd have

Well, if they were sure they could never get any benefit by reiser4
they wouldn't be testing it in the first place...

But certainly the fact they're testing it, doesn't mean they're
actually going to use it forever. You can monitor the 35 users live
with this link and to see if they increase or decrease over time ;)

	http://klive.cpushare.com/?order_by=kernel_group&where_machine=all&branch=all&scheduler=all&smp=all&live=live&ip=all

They're ordered by cumulative uptime and not by the number of
users. For example there are only 10 jfs users but those 10 jfs users
have a much larger average uptime (59 days) so they score much higher
in the cumulative uptime too. Why the average reiser4 uptime is much
lower is unknown, it could be because they shutdown the system by
night, or because they upgrade kernel so frequently, or because the
system crashes.

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Luigi Genoni]

Since reiser4 is not something enabled by default into a default kernel
distribution, I assume they enabled it knowing what they where doing because
they wanted to use it.



On Thu, July 27, 2006 15:30, Horst H. von Brand wrote:
> Luigi Genoni <genoni@sns.it> wrote:
>
>
> [...]
>
>
>> Anyway you have a datum.
>> Some people need reiser4, period.
>>
>
> Nope. Some people run kernels that include reiser4. That is all you can
> infer, and that I knew beforehand. They are at least 35, and that I'd have
> guessed in any case. --
> Dr. Horst H. von Brand                   User #22616 counter.li.org
> Departamento de Informatica                     Fono: +56 32 654431
> Universidad Tecnica Federico Santa Maria              +56 32 654239
> Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>
>

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Jim Crilly]

On 07/27/06 04:31:07PM +0200, Luigi Genoni wrote:
> Since reiser4 is not something enabled by default into a default kernel
> distribution, I assume they enabled it knowing what they where doing because
> they wanted to use it.
> 
> 

Or because they did 'allmodconfig' or 'allyesconfig'. Whenever I build
a kernel I enabled everything possible as a module in case I ever need
it. For instance, a few weeks ago I had the reiserfs module loaded because
I was testing something, if I had klive running it would have said that I
use reiserfs when in fact I don't.

Jim.

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 02:37:33PM -0400, Jim Crilly wrote:
> Or because they did 'allmodconfig' or 'allyesconfig'. Whenever I build
> a kernel I enabled everything possible as a module in case I ever need
> it. For instance, a few weeks ago I had the reiserfs module loaded because
> I was testing something, if I had klive running it would have said that I
> use reiserfs when in fact I don't.

reiserfs would showup in the module list in such case, but _not_ in
the fs list. KLive records both the modules loaded _and_ the mounted
fs.

This is the code that records the FS:

	if CONFIG_FS:
		fs = {}
		for _fs in [ re.search(r'^[^\s]+\s[^\s]+\s([^\s]+)', x).group(1)
			     for x in file('/proc/mounts').readlines() ]:
			if _fs not in ['rootfs', 'proc', 'devpts']:
				fs[_fs] = None
		fs = ' '.join(fs.keys())
		KERNEL += str_append(fs)

/proc/mounts only lists the _mounted_ fs, not the fs loaded into the
kernel statically or with insmod.

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Luigi Genoni <genoni@sns.it> wrote:

[...]

> Since reiser4 is not something enabled by default into a default kernel
> distribution, I assume they enabled it knowing what they where doing because
> they wanted to use it.

You know, I compile ReiserFS, JFS and XFS (and also CRAMFS and ROMFS) into
my self-compiled kernels too. Haven't used any of them in ages (if ever),
but one more go at compile-testing won't hurt...
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 02:43:48PM -0400, Horst H. von Brand wrote:
> You know, I compile ReiserFS, JFS and XFS (and also CRAMFS and ROMFS) into
> my self-compiled kernels too. Haven't used any of them in ages (if ever),

If you never mounted them shortly after boot time, they would have
never showed up in KLive fs list if that's what you mean.

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[CaT]

On Thu, Jul 27, 2006 at 12:04:46PM +0200, Adrian Bunk wrote:
> I can't prove that the 1:5 ratio is wrong, but the point is that 
> claiming a 1:5 ratio was true based on the klive data is not better than 
> claiming it based on no data. But claiming it based on the klive data is 
> worse since people like you are getting the wrong impression it was 
> based on data that would have the quality for supporting such a 
> statement.
> 
> The data simply has not the quality for such a statement.
> Please read my two examples in [1] if you want to get an impression why 
> such problems can occur.
> 
> [1] http://lkml.org/lkml/2006/7/26/203

Also, one may wish to invest in the purchase of 'How to Lie with
Statistics'[1] by Darrel Huff (ISBN 0-393-31072-8). An easy yet
powerful little read.

[1] this is not meant to imply that anyone here is lieing, attempting to
lie, thinking of lieing, wishing they were lieing or having anything to
do with lieing, purposeful or otherwise.

-- 
    "To the extent that we overreact, we proffer the terrorists the
    greatest tribute."
    	- High Court Judge Michael Kirby

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Adrian Bunk wrote:

>
>
>But you can not tell based on klive data whether the ratio of 
>reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000.
>
><--  snip  -->
>
>I can't prove that the 1:5 ratio is wrong, but the point is that 
>claiming a 1:5 ratio was true based on the klive data is not better than 
>claiming it based on no data. 
>
Yes, but I have been surprised that linux conference attendees all know
about reiser4, so I think it is consistent with the notion that reiser4
usage may well be much much higher than one would expect of a filesystem
that is not in the main tree, and which requires a lot of hassle to
install.  It is a datapoint.  I think the level of hobbyist enthusiasm
seen suggests that distros may gain market advantage by adding reiser4
support.....

Hans

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 08:25:23PM -0600, Hans Reiser wrote:
> Adrian Bunk wrote:
> 
> >
> >
> >But you can not tell based on klive data whether the ratio of 
> >reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000.
> >
> ><--  snip  -->
> >
> >I can't prove that the 1:5 ratio is wrong, but the point is that 
> >claiming a 1:5 ratio was true based on the klive data is not better than 
> >claiming it based on no data. 
> >
> Yes, but I have been surprised that linux conference attendees all know
> about reiser4, so I think it is consistent with the notion that reiser4
> usage may well be much much higher than one would expect of a filesystem
> that is not in the main tree,
>...

There were already several long flamewars^Wdiscussions about reiser4 on 
linux-kernel.

There has already been some amount of press articles covering reiser4.

But how much usage is the result of this?

There seem to be the the following possibilities how users get reiser4 
into their kernel today:
1. -mm kernel
2. non -mm kernel
   a) patch downloaded from namesys.com
   b) through their distribution
      - in a kernel image
      - as patch
   c) patch forwarded through other channels

2b) seems to be the only number that is easily measurable.
Do you have some download statistics for namesys.com?

Multiplying the downloads for the 2.6.16 patches with 10 seems to be the 
best estimate for the order of magnitude of current reiser4 users that 
seems to be available.

> Hans

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the ' 'official' point of view' expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Hans Reiser <reiser@namesys.com> wrote:
> Adrian Bunk wrote:

> >But you can not tell based on klive data whether the ratio of 
> >reiser4:ext3 users in the world is more like 1:5, 1:500 or 1:50000.
> >
> ><--  snip  -->
> >
> >I can't prove that the 1:5 ratio is wrong, but the point is that 
> >claiming a 1:5 ratio was true based on the klive data is not better than 
> >claiming it based on no data. 

> Yes, but I have been surprised that linux conference attendees all know
> about reiser4, so I think it is consistent with the notion that reiser4
> usage may well be much much higher than one would expect of a filesystem
> that is not in the main tree, and which requires a lot of hassle to
> install.  It is a datapoint.  I think the level of hobbyist enthusiasm
> seen suggests that distros may gain market advantage by adding reiser4
> support.....

Great! Then /sit down and do the work to get it into the vanilla
kernel/. You know then it is not a wasted effort... and you /do/ know by
now (or should) that playing the "politics"/"whining" card goes nowhere.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 08:56:03AM +0200, Adrian Bunk wrote:
> On Wed, Jul 26, 2006 at 11:17:41PM +0200, andrea@cpushare.com wrote:
> > On Wed, Jul 26, 2006 at 10:50:22PM +0200, Adrian Bunk wrote:
> > > But depending on the nature of the error, the worst case might be the 
> > > common case (as I've already explained in another email).
> > > 
> > > If you can't ensure the quality of your data, please don't use this data 
> > > to wrongly draw any conclusions from them [1].
> > 
> > Please read the footer of the KLive pages:
> > 
> > "The use of the information and of the software in this website is at
> >  your own risk.  KLive probably doesn't represent a reliable sample of
> >  the real usage of the Linux Kernel."
> 
> It was you who wrongly said:
> "With KLive I can attempt to estimate market share of _kernel_ code"
> 
> Hadn't you read your own disclaimer?

There is no contradiction in the two statements. To attempt to
estimate something I don't need a reliable sample of the whole
population. Estimation is still a statistical thing. Also I said
attempt to estimate, it doesn't mean I will make it.

If you don't consider those results a positive for reiser4, it can
only mean you expected reiser4 to have a much higher share among the
KLive users. This is obvious.

> Every time someone will repeat the "1:5 ratio for reiser4:ext3 users", 
> this will be an additional proof it's really worse than no data.

If they say "1:5 ratio for reiser4:ext3 KLive users" everything will
be correct and nobody can object because it's a fact.

I said myself that I'm no reiserfs user, and I don't plan to become
one any time soon (especially on my production systems), I'm only
reporting plain numbers as KLive measured the stuff. I'm surprised as
much as you are, but then I've to report facts, and not my own
opinions.

As far as I'm concerned the thing I like less of reiser4 is the plugin
thing, I'd be less concerned if that was a microkernel (fuse-like)
userland plugin system. Anyway with time perhaps things can change and
become userland based, and the stuff can be moved into vfs if that
code really belongs there as some kernel developer says. That doesn't
mean reiser4 can't be merged first and the stuff moved into vfs
later. xfs when was merged also pratically rewrote a vfs internally
that was meant to work with irix, if Christoph didn't complain about
xfs being merged, I don't see what's the problem of reiser4 being
merged even if it rewrites some part of vfs. xfs is also still having
various special features like the pinhole one that only belongs to the
vfs instead but nobody complains.

And if reiser4 is really so bad as they say, once people starts losing
data they will spread the word of not using it. As long as it's marked
experimental I don't see a big issue, the wireless driver for broadcom
chip will eat your filesystem too if the reverse engineered dma
operations writes into a buffer header instead of an skb.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 01:52:29PM +0200, andrea@cpushare.com wrote:
> On Thu, Jul 27, 2006 at 08:56:03AM +0200, Adrian Bunk wrote:
>...
> > It was you who wrongly said:
> > "With KLive I can attempt to estimate market share of _kernel_ code"
> >
> > Hadn't you read your own disclaimer?
>
> There is no contradiction in the two statements. To attempt to
> estimate something I don't need a reliable sample of the whole
> population. Estimation is still a statistical thing. Also I said

Sure, you don't need any data to estimate anything.

But if your estimate is based on bad data it sounds better than if it 
was based on no data although the value of the estimate is the same.

> attempt to estimate, it doesn't mean I will make it.

Giving low quality raw data and then saying "it wasn't me who did the 
estimate" is silly.

> If you don't consider those results a positive for reiser4, it can
> only mean you expected reiser4 to have a much higher share among the
> KLive users. This is obvious.

That's completely wrong.
I consider those results neither positive nor negative for reiser4.

They could only be considered positive if someone expected less than
35 reiser4 users worldwide.

> > Every time someone will repeat the "1:5 ratio for reiser4:ext3 users", 
> > this will be an additional proof it's really worse than no data.
> 
> If they say "1:5 ratio for reiser4:ext3 KLive users" everything will
> be correct and nobody can object because it's a fact.

And 90% of all people will misread it.

And many people will spread it wrong.

That's exactly where such data is worse than no data.

You could claim you know your the suboptimal quality of the data and 
that you had a disclaimer.

But did you read Hans' reaction on your email?

If you are intelligent (which I assume), you should have learned by this 
how to not present your data.

> I said myself that I'm no reiserfs user, and I don't plan to become
> one any time soon (especially on my production systems), I'm only
> reporting plain numbers as KLive measured the stuff. I'm surprised as
> much as you are, but then I've to report facts, and not my own
> opinions.
>...

I have no plans to use reiser4 myself (I'm a happy ext2 user).
But I'm not against mergng reiser4, either.

I'll leave this technical discussion to the people who know more about 
this.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 02:18:11PM +0200, Adrian Bunk wrote:
> They could only be considered positive if someone expected less than
> 35 reiser4 users worldwide.

The only thing we know for sure is that 35 out of 500 KLive user are
running reiser4, worldwide we have no clue.

> If you are intelligent (which I assume), you should have learned by this 
> how to not present your data.

I think readers are intelligent enough too to interpret the KLive data
properly for themself, without you having to prevent their eyes to see
the raw KLive data.

When you tell me how I should not present my data, you're asking me to
censor part or all of the very output of the KLive project. I'd rather
wipe out KLive completely, than to censor it. The way I presented it
was absolutely not biased, if you can make more transparent and
unbiased sql queries than the ones I did, please post them and I'll be
glad to run them.

> [..] (I'm a happy ext2 user).

Oh my, I hope you're only choosing the fs for your own workstation.

Even though I don't pretend to fully understand someone who claims to
be happy with ext2, I've no idea why you hate so much the stuff
running at cpushare.com domain. But if it helps KLive is actually one
of the non commercial projects I'm hosting there, and the only reason
I keep it there, is to be sure not to find it filled by ads.

Now let's try to get some work done instead of only sending emails ;)

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 03:10:32PM +0200, andrea@cpushare.com wrote:
> On Thu, Jul 27, 2006 at 02:18:11PM +0200, Adrian Bunk wrote:
> > They could only be considered positive if someone expected less than
> > 35 reiser4 users worldwide.
> 
> The only thing we know for sure is that 35 out of 500 KLive user are
> running reiser4, worldwide we have no clue.
> 
> > If you are intelligent (which I assume), you should have learned by this 
> > how to not present your data.
> 
> I think readers are intelligent enough too to interpret the KLive data
> properly for themself, without you having to prevent their eyes to see
> the raw KLive data.
> 
> When you tell me how I should not present my data, you're asking me to
> censor part or all of the very output of the KLive project. I'd rather
> wipe out KLive completely, than to censor it. The way I presented it
> was absolutely not biased, if you can make more transparent and
> unbiased sql queries than the ones I did, please post them and I'll be
> glad to run them.

What about the following statement:

"Gentoo is 47 times as popular as SuSE among KLive users (a service
 offered by a SuSE employee gathering data from many users worldwide)."

Is any part of this information wrong?

Is it therefore OK to put this information to /. ?

Surely not, but you might get the point.

> > [..] (I'm a happy ext2 user).
> 
> Oh my, I hope you're only choosing the fs for your own workstation.
> 
> Even though I don't pretend to fully understand someone who claims to
> be happy with ext2,

It's stable, and if the machine crashes the fsck is really great (Linus' 
tree contains at least a dozen patches that had their temporary home in 
my lost+found). (I don't care that the fsck takes 20 minutes on a 250 GB 
partition.)

Other people have other preferences and do therefore prefer other 
filesystems.

> I've no idea why you hate so much the stuff
> running at cpushare.com domain. But if it helps KLive is actually one
> of the non commercial projects I'm hosting there, and the only reason
> I keep it there, is to be sure not to find it filled by ads.

This isn't against cpushare.com, it's against publishing things people 
will easily misinterpret.

The Linux Counter [1] is another non commercial project gathering 
similar data (including live data about running kernels and uptimes) 
from far more users, and I've already given an example in this thread 
how someone could have misinterpreted the data offered there. And if 
someone would argue with data from there in a similar easy to 
misinterpret way on this list, I'd react similarly.

And another example would be that you could say "one third of all Linux 
Counter machines are still running kernels < 2.6". This would be based 
on current information submitted automatically from nearly 5000 
machines. But people would read this is "one third of all machines are 
still running kernels < 2.6". But the data doesn't have the quality for 
this generalization. And this is exactly the problem.

> Now let's try to get some work done instead of only sending emails ;)

Agreed.  ;-)

cu
Adrian

[1] http://counter.li.org/

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 03:58:23PM +0200, Adrian Bunk wrote:
> What about the following statement:
> 
> "Gentoo is 47 times as popular as SuSE among KLive users (a service
>  offered by a SuSE employee gathering data from many users worldwide)."
> 
> Is any part of this information wrong?
> 
> Is it therefore OK to put this information to /. ?
> 
> Surely not, but you might get the point.

I already told you once that such a claim is totally bogus because
KLive _doesn't_ record anything about the _userland_, it only records
kernel stuff.

It's not Dlive (distributon Live), it's KLive and K stands for
_kernel_ not distributions.

Not sure how many times I'll have to remind you about this.


Also note currently the current cumulative uptime ratio is 1:18 not
1:47 and even the ratio of the number of users is 1:42. The only claim
based on actual facts you can try to publish on the press is this:

"The Gentoo kernels are 18 times more popular than the SUSE kernels
among KLive users (a service offered in his spare time by a SUSE
contractor that tries to gather data from many users worldwide)."

You can also add:

"Another data point is that among the KLive users the Fedora kernels
are 24 times less popular then Gentoo ones."

Free to advertise the above if you find it interesting.

> This isn't against cpushare.com, it's against publishing things people 
> will easily misinterpret.

I think people deserve to think for themself without you acting as
censorship filter for them.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Adrian Bunk]

On Thu, Jul 27, 2006 at 04:45:29PM +0200, andrea@cpushare.com wrote:
> On Thu, Jul 27, 2006 at 03:58:23PM +0200, Adrian Bunk wrote:
> > What about the following statement:
> > 
> > "Gentoo is 47 times as popular as SuSE among KLive users (a service
> >  offered by a SuSE employee gathering data from many users worldwide)."
> > 
> > Is any part of this information wrong?
> > 
> > Is it therefore OK to put this information to /. ?
> > 
> > Surely not, but you might get the point.
> 
> I already told you once that such a claim is totally bogus because
> KLive _doesn't_ record anything about the _userland_, it only records
> kernel stuff.
> 
> It's not Dlive (distributon Live), it's KLive and K stands for
> _kernel_ not distributions.
> 
> Not sure how many times I'll have to remind you about this.
> 
> 
> Also note currently the current cumulative uptime ratio is 1:18 not
> 1:47 and even the ratio of the number of users is 1:42. The only claim
> based on actual facts you can try to publish on the press is this:

The 1:47 was what I calculated yesterday, but it seems to be 1:42 now.

But I'd measure popularity in current usage, not in cumulative uptime.

> "The Gentoo kernels are 18 times more popular than the SUSE kernels
> among KLive users (a service offered in his spare time by a SUSE
> contractor that tries to gather data from many users worldwide)."
>...

Slightly modified version is now in my .signature (I omitted the 
"spare time" part since the information is still true and sounds better 
without it - it would be cencorship if you'd try to force me to not 
omit it).

cu
Adrian

-- 

  The Gentoo kernels are 42 times more popular than the SUSE kernels 
  among KLive users (a service by SuSE contractor Andrea Arcangeli that 
  gathers data about kernels from many users worldwide).

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

On Thu, Jul 27, 2006 at 05:05:08PM +0200, Adrian Bunk wrote:
> Slightly modified version is now in my .signature (I omitted the 
> "spare time" part since the information is still true and sounds better 
> without it - it would be cencorship if you'd try to force me to not 
> omit it).

Well, a signature like that, perfectly match the IQ of who writes it,
please add a link to the klive homepage too if you don't mind.

This IMHO would be even more hilarious:

"I'm happy with ext2" by Adrain Bunk, the guy who pretends to provide
useful feedback in the reiser4 discussion.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Adrian and Andrea, you both write a lot of useful patches, so let's
remember that, and try to all get along.....

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[andrea]

> remember that, and try to all get along.....

FWIW the last thing I can be interested about is to get along with
somebody that gratuitously attacks me every time he can.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

andrea@cpushare.com wrote:

>
>
>As far as I'm concerned the thing I like less of reiser4 is the plugin
>thing, I'd be less concerned if that was a microkernel (fuse-like)
>userland plugin system. 
>

Performance.  If your plugin is performance valuing, it needs to be in
kernel.   Also, FUSE does not have per-file plugins (correct me if I err
here).  It would be nice to see it pay attention to reiser4 pluginids,
and thus become per-file.

Thanks for your other words, which I will not comment on because I agree
with them.;-)

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

Adrian Bunk wrote:

>[1] the conclusion itself might or might not be true
>    e.g. there _could_ be an 1:5 ratio between reiser4 and ext3 users
>    but your data is not in any way able to support or reject this 
>    statement
>  
>
It does however suggest that my surprise at how people at the last 
Linux Conference I went to all seemed to know that there exists a
Reiser4 may be due to it being more widely used than I would have
guessed.  Maybe there is some coolness factor to having the faster FS
that you can't get from any Distro that is enough to overcome the hassle
of compiling reiser4progs and a kernel before inserting the DVD.  I
would not have guessed we had 1/5th of ext3's usage even among lkml
readers.....  I guess the market contains more people who like
technology than I was guessing.  Maybe there is a positive word of mouth
effect going on too.

It would be nice if SuSE and others at least made it an unsupported
option at install time.  I shall have to find the time to go asking them
all.....

Hans

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Horst H. von Brand]

Hans Reiser <reiser@namesys.com> wrote:
> Adrian Bunk wrote:
> >[1] the conclusion itself might or might not be true
> >    e.g. there _could_ be an 1:5 ratio between reiser4 and ext3 users
> >    but your data is not in any way able to support or reject this 
> >    statement

> It does however suggest that my surprise at how people at the last 
> Linux Conference I went to all seemed to know that there exists a
> Reiser4

I'd be rather surprised to find someone at a Linux conference who hasn't at
least heard of the recurrent flamewars on the topic here...

>         may be due to it being more widely used than I would have
> guessed.  Maybe there is some coolness factor to having the faster FS
> that you can't get from any Distro that is enough to overcome the hassle
> of compiling reiser4progs and a kernel before inserting the DVD.

Not seen any data backing up the "faster", let alone so much faster that
the hassle (and the risk) would make it worth trying... No, Gentoo folks
don't count, around here they are fond of claiming that their self-compiled
systems are at least twice as fast as a binary distribution with the exact
same software.

>                                                                   I
> would not have guessed we had 1/5th of ext3's usage even among lkml
> readers.....

I'd guess something of the order of 1/100, for testing purposes and idle
curiosity.

>              I guess the market contains more people who like
> technology than I was guessing.  Maybe there is a positive word of mouth
> effect going on too.

LKML (and Linux conferences) are exactly the places where you /only/ find
this kind of people...

> It would be nice if SuSE and others at least made it an unsupported
> option at install time.  I shall have to find the time to go asking them
> all.....

At least Fedora is trying hard to just follow upstream packages (in this
case, Linus' kernels) with the absolute minimum of local patches. It makes
good sense, as it reduces the up-front work, and (more important) minimizes
the pain when the later official version is somehow incompatible with the
previews.

If you want ReiserFS 4 to get more exposure, do the legwork to get it into
the official kernel. Distributions should be reluctant to pick it up as
long as its status as an official Linux filesystem is in question.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Steve Lord]

Theodore Tso wrote:
> On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote:

>> Consider what happened with XFS as the article writer mentions.  I met
>> the original XFS team, led by two very senior developers (Jim Grey, and
>> another fellow whose name I am blanking on, forgive me, I learned much
>> from him in just a few conversations).  

Not sure who Jim Grey was, he never worked on XFS, ah well.

> 
> I believe you are referring to Jim Mostek and Steve Lord, and yes,
> they were very talented developers and engineers.  I very much enjoyed
> talking to them at various filesystem and Linux conferences and
> workshops.
> 
>> supervision.  What happened?  They got hassled.  Instead of learning
>> from them, welcoming into our community two very senior developers who
>> knew a lot more than any of us about the topics they chose to speak
>> about, they got hassled, they get ignored, they felt rejected, and left
>> the Linux community forever, never to return.  
> 
> That's hardly what happened.  SGI went through layoffs, and they were
> hit.  See:  http://slashdot.org/articles/01/05/26/0743254.shtml
> 

Ted, you of all people should know not believe all you read on
slashdot ;-) 'Linuxcare helping out with the funding' ha!

Both Jim Mostek and I left under our own steam at different times, Jim
in 2000 and myself in 2003. SGI still has great technology to work on
and, but the you can only take so many years of bad financial results
and watching people get layed off.

I still work on Linux, and follow development as much as I can. I keep
trying  to get back to OLS, but circumstances keep conspiring against
me, maybe next year.

Steve Lord

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Nikita Danilov]

Steve Lord writes:
 > Theodore Tso wrote:
 > > On Fri, Jul 21, 2006 at 01:46:18PM -0600, Hans Reiser wrote:
 > 
 > >> Consider what happened with XFS as the article writer mentions.  I met
 > >> the original XFS team, led by two very senior developers (Jim Grey, and
 > >> another fellow whose name I am blanking on, forgive me, I learned much
 > >> from him in just a few conversations).  
 > 
 > Not sure who Jim Grey was, he never worked on XFS, ah well.

I believe the (mis-)reference is to a famous data-base person, co-author
of "Transaction Processing". He is with Microsoft now
(http://research.microsoft.com/~Gray/JimGrayHomePageSummary.htm).

 > 
 > Steve Lord

Nikita.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Theodore Tso]

On Mon, Jul 24, 2006 at 11:53:08AM +0400, Nikita Danilov wrote:
> 
> I believe the (mis-)reference is to a famous data-base person, co-author
> of "Transaction Processing". He is with Microsoft now
> (http://research.microsoft.com/~Gray/JimGrayHomePageSummary.htm).
> 

That's what I thought when I saw the name Jim Gray, but as far as I
knew he never worked on XFS and never left the Linux community
dejected because Linux kernel coding standards requirements before
changes were allowed to be merged, when Hans did his name dropping
thing.

(I mean geez, if you want really high standards before new code is
accepted, take a look at Open Solaris; they have *such* a heavyweight
process, with two mandatory signoffs by core Solaris engineers who
both have to do a line-by-line review, and with a promise of on-disk
and ABI compatibility *forever* ---- that we do more commits in a week
than they do in a year....)

						- Ted

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Olivier Galibert]

On Mon, Jul 24, 2006 at 06:30:23AM -0400, Theodore Tso wrote:
> (I mean geez, if you want really high standards before new code is
> accepted, take a look at Open Solaris; they have *such* a heavyweight
> process, with two mandatory signoffs by core Solaris engineers who
> both have to do a line-by-line review, and with a promise of on-disk
> and ABI compatibility *forever* ---- that we do more commits in a week
> than they do in a year....)

That sounds almost like gcc, only worse.

I think there is something of a problem currently, tough.  It is
getting too hard to get code in if you're not a maintainer for an
existing subsystem (reiser4, suspend2...), and too easy when you're a
maintainer (ext4, uswsusp...).  Ext patches don't get reviewed much
outside of the developpers, and they go in pretty much without
discussion in any case, except when Linus blows a fuse.  Reiser4 would
have be in without discussion if it had been a set of patches through
time to reiser3, and would have been called 4 only when Linus yelled.
I suspect some balancing would be useful.

  OG.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Theodore Tso]

On Mon, Jul 24, 2006 at 01:35:34PM +0200, Olivier Galibert wrote:
>Ext patches don't get reviewed much
> outside of the developpers, and they go in pretty much without
> discussion in any case, except when Linus blows a fuse.

Um, you're kidding, right?  We certainly don't make the assumption
that we can violate CodingStyle willy nilly and stuff in yacc grammers
into ext3 and assume that no one will push back.

In fact we did a lot of work to make sure the patches were clean and
mostly ready to be accepted to mainline even before we made the first
proposal to push extents to LKML.

> I think there is something of a problem currently, tough.  It is
> getting too hard to get code in if you're not a maintainer for an
> existing subsystem (reiser4, suspend2...), and too easy when you're a
> maintainer (ext4, uswsusp...).  

It's not fair to assume that the only reason why non-maintainers have
a harder time getting changes is because their changes are getting
more intensive review.  (Although it is the case that we probably do
need to get better at reviewing changes that go in via git trees.)  A
much more important effect is that non-maintainers aren't familiar
with coding and patch submission guidelines.  For example, in
suspend2, Nigel first tried with patches that were too monolithic, and
then his next series was too broken down such that it was too hard to
review (and "git bisect" wouldn't work).  And of course, there are
people who assume that the rules shouldn't apply to their filesystem...

						- Ted

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Olivier Galibert]

On Mon, Jul 24, 2006 at 09:39:39AM -0400, Theodore Tso wrote:
> On Mon, Jul 24, 2006 at 01:35:34PM +0200, Olivier Galibert wrote:
> >Ext patches don't get reviewed much
> > outside of the developpers, and they go in pretty much without
> > discussion in any case, except when Linus blows a fuse.
> 
> Um, you're kidding, right?  We certainly don't make the assumption
> that we can violate CodingStyle willy nilly and stuff in yacc grammers
> into ext3 and assume that no one will push back.

I'm not kidding.  I do recognise that the ext* maintainers do a very
good and clean job.


> In fact we did a lot of work to make sure the patches were clean and
> mostly ready to be accepted to mainline even before we made the first
> proposal to push extents to LKML.

I'm no talking about extends only.  Ext3 now is very, very different
than the ext2+journal it was at the start, with backwards-incompatible
format changes added all the time[1].  These changes went in
no-questions-asked.


> > I think there is something of a problem currently, tough.  It is
> > getting too hard to get code in if you're not a maintainer for an
> > existing subsystem (reiser4, suspend2...), and too easy when you're a
> > maintainer (ext4, uswsusp...).  
> 
> It's not fair to assume that the only reason why non-maintainers have
> a harder time getting changes is because their changes are getting
> more intensive review.

"only", no, definitively not.  The impact is non-negligible though.


> (Although it is the case that we probably do
> need to get better at reviewing changes that go in via git trees.)

Ohh yes, a lot better.  Just look at ALSA, most of SNDRV_HWDEP* should
never have gotten in in the first place, especially some recent ones
like SNDRV_HWDEP_IFACE_SB_RC, and even some (SNDRV_HWDEP_IFACE_USX2Y*)
are security holes the size of Cleveland.  I need to continue
documenting the Alsa kernel interface, but I need a new bucket, the
first one is overflowing.


> A much more important effect is that non-maintainers aren't familiar
> with coding and patch submission guidelines.  For example, in
> suspend2, Nigel first tried with patches that were too monolithic,
> and then his next series was too broken down such that it was too
> hard to review (and "git bisect" wouldn't work).

All his submissions since 2004 or so?  It's a little easy to limit
oneself to the last two ones.


> And of course, there are people who assume that the rules shouldn't
> apply to their filesystem...

It may be a little hard to remove XFS at that point though...  And,
while it's not a filesystem, I'd love to be pointed to the technical
discussion deciding whether uswsusp is a good idea.

  OG.

[1] All optional, I know.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Theodore Tso]

On Mon, Jul 24, 2006 at 05:38:53PM +0200, Olivier Galibert wrote:
> I'm no talking about extends only.  Ext3 now is very, very different
> than the ext2+journal it was at the start, with backwards-incompatible
> format changes added all the time[1].  These changes went in
> no-questions-asked.

The patches indeed were reviewed and changes made in response to the
reviews.  The philosophical/design question about whether or not
optional features which, if enabled, would prevent older kernels to
mount the filesystem was not asked, no.  But that doesn't mean that
the code was not reviewed; it was.

I would also note that we didn't intimate that we knew better than the
reviewers, or question their motives, or otherwise insult the
reviewers such that they might decide they have better things to do
than to review our patches, and that might have had something to do
with how the code got in relatively painlessly....

						- Ted

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Olivier Galibert]

On Mon, Jul 24, 2006 at 12:17:55PM -0400, Theodore Tso wrote:
> I would also note that we didn't intimate that we knew better than the
> reviewers, or question their motives, or otherwise insult the
> reviewers such that they might decide they have better things to do
> than to review our patches, and that might have had something to do
> with how the code got in relatively painlessly....

Now that has a very high impact :-) Hans is not very good at the
diplomacy game.

The fact that the ext maintainers are very, very good helps quite a
lot too.  But I think it doesn't change the fact that if r4 has been a
set of patches through time to r3, good or not, there wouldn't be a
discussion.


It's maybe the lack of an official development branch, but it looks
like the kernel development has become very risk-averse, and the bar
is set much higher to accept anything that looks relatively new.  Any
reason is good to have it dropped, cosmetic or not.

Just to give you an idea, if the criteria applied to suspend2 or
reiser4 had been applied to everything else, we wouldn't have at least
XFS[1], ALSA[2], sysfs[3] and DRM[4].  Whether it is good or bad is an
interesting question itself.  But before, code just had to be
reasonably sane, and it was expected to be fixed through time.  Some
even has been (sysfs got better).  Now it has to attain an ever moving
level of perfection before it has a chanc to be accepted.

Unless you're a maintainer, that is, for which you can get pretty much
anything in that doesn't immediatly break the compile through git
trees.  Thankfully, most of the maintainers are sane.


Don't you think this is a problem?

  OG.


[1] CodingStyle is for the weak

[2] You call that a kernel interface?  HWDEP?  ioctl to read/write
    samples instead of read/write?

[3] Lifetime rules and locking are for other people to care about

[4] Oh my, people are still trying to fix that one

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Jeff Garzik]

Olivier Galibert wrote:
> The fact that the ext maintainers are very, very good helps quite a
> lot too.  But I think it doesn't change the fact that if r4 has been a
> set of patches through time to r3, good or not, there wouldn't be a
> discussion.

That's a huuuuuge leap of logic.

Metadata plugins found in reiser4 are far better done at the VFS level, 
than burying "reiser4 can look like ext2, if it wishes" functionality 
inside the filesystem.

I guarantee such a patch to reiser3 would get rejected.


> It's maybe the lack of an official development branch, but it looks
> like the kernel development has become very risk-averse, and the bar
> is set much higher to accept anything that looks relatively new.  Any
> reason is good to have it dropped, cosmetic or not.

New stuff goes in all the time.

The bar is set too high in some cases (read: SCSI subsystem 
submissions), but reiser4 submission cannot be generalized as you have 
done here.  There are very real issues present, that need to be dealt with.


> Just to give you an idea, if the criteria applied to suspend2 or
> reiser4 had been applied to everything else, we wouldn't have at least
> XFS[1], ALSA[2], sysfs[3] and DRM[4].  Whether it is good or bad is an
> interesting question itself.  But before, code just had to be
> reasonably sane, and it was expected to be fixed through time.  Some
> even has been (sysfs got better).  Now it has to attain an ever moving
> level of perfection before it has a chanc to be accepted.

reiser4 tries to be another VFS.  That's a bit more than needing 
additional minor fixes over time.

	Jeff

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Pavel Machek]

Hi!

> > A much more important effect is that non-maintainers aren't familiar
> > with coding and patch submission guidelines.  For example, in
> > suspend2, Nigel first tried with patches that were too monolithic,
> > and then his next series was too broken down such that it was too
> > hard to review (and "git bisect" wouldn't work).
> 
> All his submissions since 2004 or so?  It's a little easy to limit
> oneself to the last two ones.

Nigel did not do any submissions in 2004 or so. Check your fact, that
stuff was marked 'RFC' and yes I did comment on it.

He did 1 (one) submission that looked like SubmittingPatches at the
first sight, and that was very recent.

Stop spreading lies.

							Pavel
-- 
Thanks for all the (sleeping) penguins.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Olivier Galibert]

On Wed, Jul 26, 2006 at 01:08:06PM +0000, Pavel Machek wrote:
> Hi!
> 
> > > A much more important effect is that non-maintainers aren't familiar
> > > with coding and patch submission guidelines.  For example, in
> > > suspend2, Nigel first tried with patches that were too monolithic,
> > > and then his next series was too broken down such that it was too
> > > hard to review (and "git bisect" wouldn't work).
> > 
> > All his submissions since 2004 or so?  It's a little easy to limit
> > oneself to the last two ones.
> 
> Nigel did not do any submissions in 2004 or so. Check your fact, that
> stuff was marked 'RFC' and yes I did comment on it.

2004-09-16 submission:

http://lkml.org/lkml/2004/9/16/76
http://lkml.org/lkml/2004/9/16/77
http://lkml.org/lkml/2004/9/16/78
http://lkml.org/lkml/2004/9/16/81
http://lkml.org/lkml/2004/9/16/82
http://lkml.org/lkml/2004/9/16/83
http://lkml.org/lkml/2004/9/16/86
http://lkml.org/lkml/2004/9/16/87
http://lkml.org/lkml/2004/9/16/89
http://lkml.org/lkml/2004/9/16/90


2004-11-24 submission:

http://lkml.org/lkml/2004/11/24/93
http://lkml.org/lkml/2004/11/24/94
http://lkml.org/lkml/2004/11/24/95
http://lkml.org/lkml/2004/11/24/96
http://lkml.org/lkml/2004/11/24/97
http://lkml.org/lkml/2004/11/24/98
http://lkml.org/lkml/2004/11/24/100
http://lkml.org/lkml/2004/11/24/101
[58 mails or so total]


> He did 1 (one) submission that looked like SubmittingPatches at the
> first sight, and that was very recent.
> 
> Stop spreading lies.

I am awaiting your apologies.

  OG.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Alan Cox]

Ar Mer, 2006-07-26 am 13:08 +0000, ysgrifennodd Pavel Machek:
> He did 1 (one) submission that looked like SubmittingPatches at the
> first sight, and that was very recent.
> 
> Stop spreading lies.

Pavel, I think you might want to check your facts and then apologize.

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

On Mon, 24 Jul 2006 06:30:23 EDT, Theodore Tso said:
> both have to do a line-by-line review, and with a promise of on-disk
> and ABI compatibility *forever* ---- that we do more commits in a week

"forever"? Man, that's hard-core. Is that *really* the guideline, or is
it some "might as well be forever in this industry" rule like "5 years"?




[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Paul Jackson]

Hans wrote:
> Jim Grey

Sounds like the name of Jim Gray, the famous database and transaction
processing expert of IBM, Tandem, DEC and Microsoft.

Neither SGI nor XFS has had the honor of working with Jim Gray.

The original XFS developers and designers include Geoff Peck, Jeff
Glover, Adam Sweeney and Doug Doucette, from what I can see from my
notes dating back to 1993/1994 on this.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion

[Hans Reiser]

No, it was not him, sigh, I'll have to ask Jim Mostek who it was if I
can find a valid email for Jim.  There was another guy who was Jim
Mostek's peer.  I might have the name wrong.