Re: I request inclusion of reiser4 in the mainline kernel

Re: I request inclusion of reiser4 in the mainline kernel

[James Lamanna]

On 9/20/05, Stephen Pollei <stephen.pollei@gmai.com> wrote:
>On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
> > Horst von Brand wrote:
> > >Nikita Danilov <nikita@clusterfs.com> wrote:
> > >It is supposed to go into the kernel, which is not exactly warning-free.

> > Is that what this thread boils down to, that you guys think the compile
> > should fail not warn?

> > I don't care if it fails or warns at compile time, but you shouldn't
> > misuse/abuse a warning by potentialily introducing an unrelated bug.
>
> So if you had
>#if defined(DEBUG_THIS) || defined(DEBUG_THAT)
>int znode_is_loaded(const struct znode *z);
> #else
> int znode_is_loaded(const struct znode *z)
>  __attribute__((__warn_broken__("unavailible when not debuging")));
> #endif
> That would be great with me.. except __warn_broken__ or the like
> doesn't exist AFAIK :-<
> Closest thing is __attribute((__deprecated__)) but thats not quite right.
> > > As was said before: It it is /really/ wrong, arrange for it not to compile
> > > or not to link. If it isn't, well... then it wasn't that wrong anyway.

What about #warning / #error in this case?

#if defined(DEBUG_THIS) || defined(DEBUG_THAT)
    int znode_is_loaded(const struct znode *z);
#else
    #error znode_is_loaded is unavailable when not debugging
#endif

That would certainly break the compile.

-- James Lamanna

Re: I request inclusion of reiser4 in the mainline kernel

[Vadim Lobanov]

On Tue, 20 Sep 2005, James Lamanna wrote:

> On 9/20/05, Stephen Pollei <stephen.pollei@gmai.com> wrote:
> >On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
> > > Horst von Brand wrote:
> > > >Nikita Danilov <nikita@clusterfs.com> wrote:
> > > >It is supposed to go into the kernel, which is not exactly warning-free.
>
> > > Is that what this thread boils down to, that you guys think the compile
> > > should fail not warn?
>
> > > I don't care if it fails or warns at compile time, but you shouldn't
> > > misuse/abuse a warning by potentialily introducing an unrelated bug.
> >
> > So if you had
> >#if defined(DEBUG_THIS) || defined(DEBUG_THAT)
> >int znode_is_loaded(const struct znode *z);
> > #else
> > int znode_is_loaded(const struct znode *z)
> >  __attribute__((__warn_broken__("unavailible when not debuging")));
> > #endif
> > That would be great with me.. except __warn_broken__ or the like
> > doesn't exist AFAIK :-<
> > Closest thing is __attribute((__deprecated__)) but thats not quite right.
> > > > As was said before: It it is /really/ wrong, arrange for it not to compile
> > > > or not to link. If it isn't, well... then it wasn't that wrong anyway.
>
> What about #warning / #error in this case?
>
> #if defined(DEBUG_THIS) || defined(DEBUG_THAT)
>     int znode_is_loaded(const struct znode *z);
> #else
>     #error znode_is_loaded is unavailable when not debugging
> #endif
>
> That would certainly break the compile.

Except that breaks the compile unconditionally, not just when someone
tries to use the function when they shouldn't. I don't think a flat
error will work here, but instead something along the lines of a
__attribute__((error)) on the function is needed.

> -- James Lamanna
> -

-VadimL

Re: I request inclusion of reiser4 in the mainline kernel

[James Lamanna]

On 9/20/05, Vadim Lobanov <vlobanov@speakeasy.net> wrote:
> On Tue, 20 Sep 2005, James Lamanna wrote:
>
> > On 9/20/05, Stephen Pollei <stephen.pollei@gmai.com> wrote:
> > >On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
> > > > Horst von Brand wrote:
> > > > >Nikita Danilov <nikita@clusterfs.com> wrote:

> > What about #warning / #error in this case?
> >
> > #if defined(DEBUG_THIS) || defined(DEBUG_THAT)
> >     int znode_is_loaded(const struct znode *z);
> > #else
> >     #error znode_is_loaded is unavailable when not debugging
> > #endif
> >
> > That would certainly break the compile.
>
> Except that breaks the compile unconditionally, not just when someone
> tries to use the function when they shouldn't. I don't think a flat
> error will work here, but instead something along the lines of a
> __attribute__((error)) on the function is needed.

Oh duh. It would wouldn't it :)
Too much on my mind today.

>
> > -- James Lamanna
> > -
>
> -VadimL
>

-- James Lamanna

I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

All objections have now been addressed so far as I can discern.

    The VFS layering issue was addressed after 2 months of recoding.

    The undesired type safe lists were removed after ~ a man week of coding.

    Cosmetic issues regarding line length, etc., were addressed.

    Numerous ~ one line changes were made that I will not address here.

    The assertions were left in, with akpm's ok.

    Pseudo files were removed.

    dependency on !4k stacks was removed and stack usage was fixed.

    reiser4_drop_inode was removed.

    our div64_32 was replaced with the linux one

I request that reiser4 be included.  Technically, we submitted 9 months
before the deadline for 2.6.14, though I am sure the point will be
argued.  We would have submitted our feedback fixes on monday but we
lost the type safe lists argument over the weekend before monday, so it
delayed us.

There have been no bug reports concerning the new code. 

If we get lucky, we might also have a compression plugin working by the
time 2.6.14 ships, it just needs some mmap fixes to work.  Then the
benchmarks will be truly excessive.....  even after we rewrite them
because they currently generate files that compress too well to be
realistic.....

:)

Thanks to all for your kind suggestions of improvements to our work, and
the time you invested in providing us with feedback.  It will be easier
to use meta-. to browse our code now that the type safe lists are gone,
etc., etc.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Fri, Sep 16, 2005 at 10:05:08AM -0700, Hans Reiser wrote:
> All objections have now been addressed so far as I can discern.
> 
>     The VFS layering issue was addressed after 2 months of recoding.
> 
>     The undesired type safe lists were removed after ~ a man week of coding.
> 
>     Cosmetic issues regarding line length, etc., were addressed.
> 
>     Numerous ~ one line changes were made that I will not address here.
> 
>     The assertions were left in, with akpm's ok.
> 
>     Pseudo files were removed.
> 
>     dependency on !4k stacks was removed and stack usage was fixed.
> 
>     reiser4_drop_inode was removed.
> 
>     our div64_32 was replaced with the linux one

You completely ignore my last review comments.    And that was just the
errors sticking out from an half an error look.  I'll do a deeper review,
but ocfs has a higher priority right now.

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

more trivial review comments ontop of the previous one, after looking
at things:

 - please never use list_for_each in new code but list_for_each_entry
 - never use kernel_thread in new code but kthread_*
 - do_sendfile duplicates the common sendfile code.  why aren't you
   using the generic code?
 - there's tons of really useless assertation of the category
   discussed in the last thread
 - there's tons of deep pagecache messing in there.  normally this
   shouldn't be a filesystem, and if this breaks because of VM changes you'll
   have to fix it, don't complain..
 - you still do your plugin mess in ->readpage.  honsetly could you
   please explain why mpage_readpage{,s} don't work for you?
 - (issues with the read/write path already addresses in the previous thread)
 - looking at ->d_count in ->release is wrong
 - still has security plugin stuff that duplicates LSM
 - why do underlying attributes change when VFS inode doesn't change?
   if not please rip out most of getattr_common
 - link_common S_ISDIR doesn't make sense, VFS takes care of it
 - please use the generic_readlink infrastructure

additinoal comment is that the code is very messy, very different
from normal kernel style, full of indirections and thus hard to read.
real review will take some time.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Christoph Hellwig wrote:

>additinoal comment is that the code is very messy, very different
>from normal kernel style, full of indirections and thus hard to read.
>

Most of my customers remark that Namesys code is head and shoulders
above the rest of the kernel code.  So yes, it is different.  In
particular, they cite the XFS code as being so incredibly hard to read
that its unreadability is worth hundreds of thousands of dollars in
license fees for me.  That's cash received, from persons who read it
all, not commentary made idly.

May I suggest that you work on the XFS code instead?  Surely with all of
this energy you have, you could improve XFS a lot before it gets
accepted into the kernel. 

As for the indirections, if you figure out how to make VFS indirections
easy to follow, the same technique should be applicable to Reiser4, and
I will be happy to fix it. 

Hans

(Note for the record: I actually think XFS acceptance was delayed too
long, and I think that XFS is a great filesystem, but a rhetorical point
needed to be made......)

Re: I request inclusion of reiser4 in the mainline kernel

[Kyle Moffett]

[CC list trimmed to relevant people, no need to spam Linus' and  
Andrew's mailboxes, they have enough to do as it is]

On Sep 16, 2005, at 15:39:48, Hans Reiser wrote:
> Christoph Hellwig wrote:
>> additinoal comment is that the code is very messy, very different
>> from normal kernel style, full of indirections and thus hard to read.
>
> Most of my customers remark that Namesys code is head and shoulders
> above the rest of the kernel code.  So yes, it is different.

And yet thousands and thousands of people, businesses, etc, say that  
the Linux kernel code is miles above all the commercial software out  
there. Please leave the worthless rhetoric out of a technical  
discussion.  The issue stands that in many ways the Reiser4 code does  
not exactly follow Documentation/CodingStyle and does not match most  
of the rest of the kernel, making it hard to read for other kernel  
developers.  If you were just doing this forever as an external  
kernel patch, nobody would give a damn.  On the other hand, you're  
trying to get it included in the upstream kernel, which means that  
those same "other kernel developers" for whom it is hard to read may  
be expected to maintain it until the end of time.  Given this, it  
seems perfectly reasonable to ask that it be cleaned up.

> In particular, they cite the XFS code as being so incredibly hard  
> to read that its unreadability is worth hundreds of thousands of  
> dollars in license fees for me.

How does XFS have _anything_ to do with Reiser4?  A technical  
discussion is no place for political pissing contest.

> [more useless posturing snipped]

> As for the indirections, if you figure out how to make VFS  
> indirections easy to follow, the same technique should be  
> applicable to Reiser4, and I will be happy to fix it.

That's not his responsibility, it's _yours_.  If you want your stuff  
included in the the kernel, you need to make sure it is sufficiently  
acceptable.  Besides, this is just one complaint of the many he  
made.  This may not be particularly solvable, but there are a number  
of other points he made that you guys need to try to resolve.

> (Note for the record: I actually think XFS acceptance was delayed  
> too long, and I think that XFS is a great filesystem, but a  
> rhetorical point needed to be made......)

See above.  Rhetoric has little or no place here.  Such comments are  
why Reiser4 typically triggers massive flamewars when it is mentioned  
on the LKML.

Cheers,
Kyle Moffett

--
Somone asked me why I work on this free (http://www.fsf.org/ 
philosophy/) software stuff and not get a real job. Charles Shultz  
had the best answer:

"Why do musicians compose symphonies and poets write poems? They do  
it because life wouldn't have any meaning for them if they didn't.  
That's why I draw cartoons. It's my life."
   -- Charles Shultz

Re: I request inclusion of reiser4 in the mainline kernel

[Denis Vlasenko]

On Friday 16 September 2005 22:52, Kyle Moffett wrote:
> [CC list trimmed to relevant people, no need to spam Linus' and  
> Andrew's mailboxes, they have enough to do as it is]
> 
> On Sep 16, 2005, at 15:39:48, Hans Reiser wrote:
> > Christoph Hellwig wrote:
> >> additinoal comment is that the code is very messy, very different
> >> from normal kernel style, full of indirections and thus hard to read.
> >
> > Most of my customers remark that Namesys code is head and shoulders
> > above the rest of the kernel code.  So yes, it is different.
> 
> And yet thousands and thousands of people, businesses, etc, say that  
> the Linux kernel code is miles above all the commercial software out  
> there. Please leave the worthless rhetoric out of a technical  
> discussion.  The issue stands that in many ways the Reiser4 code does  
> not exactly follow Documentation/CodingStyle and does not match most  
> of the rest of the kernel, making it hard to read for other kernel  
> developers.  If you were just doing this forever as an external  
> kernel patch, nobody would give a damn.  On the other hand, you're  
> trying to get it included in the upstream kernel, which means that  
> those same "other kernel developers" for whom it is hard to read may  
> be expected to maintain it until the end of time.  Given this, it  
> seems perfectly reasonable to ask that it be cleaned up.

I think it makes sense to supply examples from reiser4 code
which you want to be changed. "It's ugly" is not specific enough.
--
vda

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Denis Vlasenko wrote:

>
>>And yet thousands and thousands of people, businesses, etc, say that  
>>the Linux kernel code is miles above all the commercial software out  
>>there. 
>>
Not the commercial software I have worked with.  IBM code, government
procured code, both are much more readable code than Linux Kernel
standards.  I am sure there is no shortage of bad IBM code and bad
government code, but my personal experiences were that it was much
better commented.   Your statement sounds like something you want to
believe.

That all said, the kernel code is getting better.....  if the rest of
the kernel was as well commented as akpm's code I would not be
complaining at all.

Re: I request inclusion of reiser4 in the mainline kernel

[Lennart Sorensen]

On Fri, Sep 16, 2005 at 12:39:48PM -0700, Hans Reiser wrote:
> Most of my customers remark that Namesys code is head and shoulders
> above the rest of the kernel code.  So yes, it is different.  In
> particular, they cite the XFS code as being so incredibly hard to read
> that its unreadability is worth hundreds of thousands of dollars in
> license fees for me.  That's cash received, from persons who read it
> all, not commentary made idly.
> 
> May I suggest that you work on the XFS code instead?  Surely with all of
> this energy you have, you could improve XFS a lot before it gets
> accepted into the kernel. 
> 
> As for the indirections, if you figure out how to make VFS indirections
> easy to follow, the same technique should be applicable to Reiser4, and
> I will be happy to fix it. 
> 
> (Note for the record: I actually think XFS acceptance was delayed too
> long, and I think that XFS is a great filesystem, but a rhetorical point
> needed to be made......)

Well my experience with XFS for about 6 months using 2.6 kernels has
been about as good as my experience with reiserfs 3.6 back when 2.4
was fairly new.

That's why I run ext3.

I don't need my filesystem locking up, leaking memory all over the
place, causing processes to be killed byt the out of memory handler,
etc.  I will stick with what works all the time.

Performance and nifty features are fun, but only when they don't break
your system.

Len Sorensen

Re: I request inclusion of reiser4 in the mainline kernel

[Lennart Sorensen]

On Fri, Sep 16, 2005 at 04:50:45PM -0400, Lennart Sorensen wrote:
> Well my experience with XFS for about 6 months using 2.6 kernels has
> been about as good as my experience with reiserfs 3.6 back when 2.4
> was fairly new.
> 
> That's why I run ext3.
> 
> I don't need my filesystem locking up, leaking memory all over the
> place, causing processes to be killed byt the out of memory handler,
> etc.  I will stick with what works all the time.
> 
> Performance and nifty features are fun, but only when they don't break
> your system.

In other words that means: Neither was ready for use when they were
included in the kernel and should probably have had big warning signs in
the kernel config for them.  They also seem sufficiently complex that
fixing them is very hard work.

Len Sorensen

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Lennart Sorensen wrote:

>
> Neither was ready for use when they were
>included in the kernel and should probably have had big warning signs in
>the kernel config for them. 
>  
>
They did have warning signs: they were labeled experimental....  as is
reiser4. 

At some point developers and their limited size mailing list simply
cannot make things crash, and then it needs to go in to the main kernel
labeled experimental.

Of course, the reiser4 code is not as stable as it was before the
changes Christoph asked for.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Fri, Sep 16, 2005 at 12:39:48PM -0700, Hans Reiser wrote:
> Christoph Hellwig wrote:
> 
> >additinoal comment is that the code is very messy, very different
> >from normal kernel style, full of indirections and thus hard to read.
> >
> 
> Most of my customers remark that Namesys code is head and shoulders
> above the rest of the kernel code.  So yes, it is different.  In
> particular, they cite the XFS code as being so incredibly hard to read
> that its unreadability is worth hundreds of thousands of dollars in
> license fees for me.  That's cash received, from persons who read it
> all, not commentary made idly.

It's very different from kernel style, and it's hard to read for us kernel
developers.  And yes, I don't think XFS is the most easy to read code either,
quite contrary.  But it's at least half a magnitude less bad than reiser4
code..

Re: I request inclusion of reiser4 in the mainline kernel

[Denis Vlasenko]

On Saturday 17 September 2005 12:22, Christoph Hellwig wrote:
> On Fri, Sep 16, 2005 at 12:39:48PM -0700, Hans Reiser wrote:
> > Christoph Hellwig wrote:
> > 
> > >additinoal comment is that the code is very messy, very different
> > >from normal kernel style, full of indirections and thus hard to read.
> > >
> > 
> > Most of my customers remark that Namesys code is head and shoulders
> > above the rest of the kernel code.  So yes, it is different.  In
> > particular, they cite the XFS code as being so incredibly hard to read
> > that its unreadability is worth hundreds of thousands of dollars in
> > license fees for me.  That's cash received, from persons who read it
> > all, not commentary made idly.
> 
> It's very different from kernel style, and it's hard to read for us kernel
> developers.  And yes, I don't think XFS is the most easy to read code either,
> quite contrary.  But it's at least half a magnitude less bad than reiser4
> code..

At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more time
to optimize code size, but:

reiser4        2557872 bytes
xfs            3306782 bytes
--
vda

Re: I request inclusion of reiser4 in the mainline kernel

[Denis Vlasenko]

> At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more time
> to optimize code size, but:
> 
> reiser4        2557872 bytes
> xfs            3306782 bytes

And modules sizes:

reiser4.ko        442012 bytes
xfs.ko            494337 bytes
--
vda

Re: I request inclusion of reiser4 in the mainline kernel

[Chris White]

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

CC-List trimmed

On Saturday 17 September 2005 20:15, Denis Vlasenko wrote:
> > At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more
> > time to optimize code size, but:
> >
> > reiser4        2557872 bytes
> > xfs            3306782 bytes
>
> And modules sizes:
>
> reiser4.ko        442012 bytes
> xfs.ko            494337 bytes

All this is fine and dandy, but saying "My code is better than yours!!" still 
doesn't solve the issue this thread hopes to achieve, that being "I'd like to 
get reiser4 into the kernel".  There seems to be a lot of (historical?) 
tension present here, but all that seems to be doing is making things worse.  
PLEASE keep this thing a tad on par.  Keeping this up is hurting everyone 
more than helping.  I wish I could say something as simple as "let's just be 
friends", but that's saying a lot.  I can say this though: this is open 
source, and that means that our source is open, and we should be too.

Chris White

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: I request inclusion of reiser4 in the mainline kernel

[Denis Vlasenko]

On Sunday 18 September 2005 03:34, Chris White wrote:
> CC-List trimmed
> 
> On Saturday 17 September 2005 20:15, Denis Vlasenko wrote:
> > > At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more
> > > time to optimize code size, but:
> > >
> > > reiser4        2557872 bytes
> > > xfs            3306782 bytes
> >
> > And modules sizes:
> >
> > reiser4.ko        442012 bytes
> > xfs.ko            494337 bytes
> 
> All this is fine and dandy, but saying "My code is better than yours!!" still 
> doesn't solve the issue this thread hopes to achieve, that being "I'd like to 
> get reiser4 into the kernel".  There seems to be a lot of (historical?) 
> tension present here, but all that seems to be doing is making things worse.  
> PLEASE keep this thing a tad on par.  Keeping this up is hurting everyone 
> more than helping.  I wish I could say something as simple as "let's just be 
> friends", but that's saying a lot.  I can say this though: this is open 
> source, and that means that our source is open, and we should be too.

I am trying to say that I think that Hans is being treated a bit unfairly.
His fs is new and has fairly complex on-disk structure and complex journalling
machinery, yet his source and object code is smaller than xfs which already
is accepted. This is no easy feat I guess.

Maybe xfs shouldn't be accepted too, this may be an answer.

Let's look at the code. Hans' code is not _that_ awful. Yet people
(not all of them, but some) do not point to specific things which they
want to be fixed/improved. I see blanket arguments like "your code is hard
to read". Well. Maybe spend a minute on what exactly is hard to read,
or do we require Hans to be able to read minds from the distance?

This is it. I do not say "accept reiser4 NOW", I am saying "give Hans
good code review".
--
vda

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
> This is it. I do not say "accept reiser4 NOW", I am saying "give Hans
> good code review".

After he did his basic homework.  Note that reviewing hans code is probably
at the very end of everyones todo list because every critizm of his code
starts a huge flamewar where hans tries to attack everyone not on his
party line personally.

I've said I'm gonna do a proper review after he has done the basic homework,
which he seems to have half-done now at least.  Right now he hasn't finished
that and there's much more exciting filesystems like ocfs2 around that
are much easier to read and actually have developers that you can have
a reasonable conversation with.  (and that unlike hans actually try to
improve core code where it makes sense for them)

Re: I request inclusion of reiser4 in the mainline kernel

[Denis Vlasenko]

On Sunday 18 September 2005 15:06, Christian Iversen wrote:
> On Sunday 18 September 2005 12:26, Christoph Hellwig wrote:
> > On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
> > > This is it. I do not say "accept reiser4 NOW", I am saying "give Hans
> > > good code review".
> >
> > After he did his basic homework.  Note that reviewing hans code is probably
> > at the very end of everyones todo list because every critizm of his code
> > starts a huge flamewar where hans tries to attack everyone not on his
> > party line personally.
> >
> > I've said I'm gonna do a proper review after he has done the basic
> > homework, which he seems to have half-done now at least.  Right now he
> > hasn't finished that and there's much more exciting filesystems like ocfs2
> > around [...]
> 
> Now _what_ good does that sentence do us? I've been following this this since 
> the primary reiser filesystem was number 3, and the kernel everybody was 
> using was 2.4.10. You've probably been following this list for far longer, 
> but is that really an excuse for rudeness?
> 
> reiser4 has many, many extremely interesting features. I'm sure anybody is 
> more than willing to go into detail with them, but saying that "ocfs2 is much 
> more exiting" is just plain bashing, and it's not fair to Hans, to Namesys, 
> or to every one of us who can't wait for reiser4 in mainline. 

"every one of us who can't wait" do not count, because they do nothing.

If you want reiser4 included into mainline, do something. Like download
a patch and try to use it.

Last time I tried, it didn't work. Kernel locked up. Namesys was quick
with fix for the lockup, but then "ls ." failed to work. I sent all
the data (kernel version, fs image, etc) to Namesys but after several
email iterations it died out with no resolution.

I will try again sometime. Maybe it got better.

> Could you please keep your personal idea of which filesystem is more 
> interesting to yourself? It doesn't help anybody accomplish anything. 

Your reply wasn't polite/useful either.
--
vda

Re: I request inclusion of reiser4 in the mainline kernel

[David Masover]

Denis Vlasenko wrote:

> If you want reiser4 included into mainline, do something. Like download
> a patch and try to use it.

Alright...

> Last time I tried, it didn't work. Kernel locked up. Namesys was quick
> with fix for the lockup, but then "ls ." failed to work. I sent all
> the data (kernel version, fs image, etc) to Namesys but after several
> email iterations it died out with no resolution.

When was "last time"?

> I will try again sometime. Maybe it got better.

I have three boxes running Reiser4 for everything except /boot, and no 
problems yet, except an occasional missing feature, like a 
repacker/resizer.  One's a Pentium 3, the other two are amd64s.

I've had a total of one crash each on the amd64s, and one of those was 
while playing a game, and could easily have been the nvidia drivers.  I 
can't reproduce the other one, and the box has been fine since -- and 
both amd64s are overclocked by 600 mhz, so I have a sneaking suspicion 
that it might have been hardware.

No crashes yet on the Pentium 3, which isn't overclocked at all.

No lost data yet either, in fact, I recovered from an essential 'rm -rf' 
of a Reiser4 partition, so I could even say Reiser4 (or rather, 
fsck.reiser4) has *found* data for me.



For a long time, it's been painfully obvious that the reasons Reiser4 
isn't in the kernel all have to do with things like coding style and 
politics.  At this point, if I tried to do anything more than be an 
active user, I'd be so far out of my depth I'd need a life jacket.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Christian Iversen wrote:

>On Sunday 18 September 2005 12:26, Christoph Hellwig wrote:
>  
>
>>On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
>>    
>>
>>>This is it. I do not say "accept reiser4 NOW", I am saying "give Hans
>>>good code review".
>>>      
>>>
>>After he did his basic homework.  Note that reviewing hans code is probably
>>at the very end of everyones todo list because every critizm of his code
>>starts a huge flamewar where hans tries to attack everyone not on his
>>party line personally.
>>
>>I've said I'm gonna do a proper review after he has done the basic
>>homework, which he seems to have half-done now at least.  Right now he
>>hasn't finished that and there's much more exciting filesystems like ocfs2
>>around [...]
>>    
>>
>
>Now _what_ good does that sentence do us? I've been following this this since 
>the primary reiser filesystem was number 3, and the kernel everybody was 
>using was 2.4.10. You've probably been following this list for far longer, 
>but is that really an excuse for rudeness?
>
>reiser4 has many, many extremely interesting features. I'm sure anybody is 
>more than willing to go into detail with them, but saying that "ocfs2 is much 
>more exiting" is just plain bashing, and it's not fair to Hans, to Namesys, 
>or to every one of us who can't wait for reiser4 in mainline. 
>
>Could you please keep your personal idea of which filesystem is more 
>interesting to yourself? It doesn't help anybody accomplish anything. 
>
>  
>
Hellwig, people who write slow file systems should not lecture their
measurably superiors on how to code.  Oh, and I should mention that
other people besides me have measured reiser4, and concluded it is twice
the speed of the other Linux filesystems, so don't go claiming it is
just my benchmarks.   What you are doing is keeping me from doing a real
code review myself by keeping my guys so busy that they don't have time
to review the fixmes I inserted and would insert more of if I thought
they had time for them.   If you were as well suited to doing code
reviews as I am, you would have written a faster file system yourself.  
Anybody can find things to fix in someone else's code, and it can go on
for years if they want it to.  I could get what you do from hiring a
college junior, and if it was a good university I'd probably learn more
from that junior in college than from you.  We are doing work, and you
are getting in the way.  Nobody who wants reiser4 views your
contributions as the least bit positive.  I fear you will delay us until
ext3 can catch up.

What you are is someone who substitutes social connections for technical
ability.  You measurably can't code as well as we can, so once it
conforms to VFS interface requirements, please go away.

Re: I request inclusion of reiser4 in the mainline kernel

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1210 bytes --]

On Sun, 18 Sep 2005 22:16:11 PDT, Hans Reiser said:

> Hellwig, people who write slow file systems should not lecture their
> measurably superiors on how to code.  Oh, and I should mention that
> other people besides me have measured reiser4, and concluded it is twice
> the speed of the other Linux filesystems, so don't go claiming it is
> just my benchmarks.   What you are doing is keeping me from doing a real
> code review myself by keeping my guys so busy that they don't have time
> to review the fixmes I inserted and would insert more of if I thought
> they had time for them.

Hans, unfortunately the most obvious reading of the above is "Reiser4 is so
damned fast because it doesn't bother doing sanity-checking".  If there's still
more "fixmes" to be inserted that *you* know of, and there are so many that
there's no time to fix them, why is this being submitted for inclusion?

On Sun, 18 Sep 2005 22:09:08 PDT, Hans Reiser said:
> Of course, the reiser4 code is not as stable as it was before the
> changes Christoph asked for.

This sort of claim requires proof - can you point at *specific* things that
were less stable after you fixed the code, including explaining why they're
less stable?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Valdis.Kletnieks@vt.edu wrote:

>
>
> Hans, unfortunately the most obvious reading of the above is
> "Reiser4 is so damned fast because it doesn't bother doing
> sanity-checking".

Hmm, you seem to have forgotten the Hellwig complaints about too many
assertions..... ;-)

Algorithms make a difference Valdis, they make a big difference.
> If there's still more "fixmes" to be inserted that *you* know of,
> and there are so many that there's no time to fix them, why is this
> being submitted for inclusion?

The fixmes are not bugs, those got fixed, they are "this code would be
cleaner if written another way", or, most commonly, "where is the
comment that ought to be here?"

> On Sun, 18 Sep 2005 22:09:08 PDT, Hans Reiser said:
>
>> Of course, the reiser4 code is not as stable as it was before the
>> changes Christoph asked for.
>
>
> This sort of claim requires proof - can you point at *specific*
> things that were less stable after you fixed the code, including
> explaining why they're less stable?

The 4k stacks got a bug report or two.   Generally speaking, I don't
trust any large number of lines of code of changes to be bug free, and
the VFS stuff was a large number of lines.

Re: I request inclusion of reiser4 in the mainline kernel

[michael chang]

On 9/18/05, Christoph Hellwig <hch@infradead.org> wrote:
> On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
> > This is it. I do not say "accept reiser4 NOW", I am saying "give Hans
> > good code review".
> 
> After he did his basic homework.  Note that reviewing hans code is probably
> at the very end of everyones todo list because every critizm of his code
> starts a huge flamewar where hans tries to attack everyone not on his
> party line personally.
> 
> I've said I'm gonna do a proper review after he has done the basic homework,
> which he seems to have half-done now at least.  Right now he hasn't finished

Explain to us all what is this "basic homework" of which you speak.

> that and there's much more exciting filesystems like ocfs2 around that

This is exciting to... whom?  The only thing that appears remotely
interesting about it is that it's made by Oracle and apparently is
supposed to be geared toward parallel server whatsits.  This might be
helpful to corporations, but seems senseless toward many consumers. 
(I'm assuming there's still at least one consumer left who still uses
Linux.)

> are much easier to read and actually have developers that you can have
> a reasonable conversation with.  (and that unlike hans actually try to

Is that Hans' fault, or the fault of your lot?  Why can't we all just get along?

Give Hans a chance; and please try to understand, even if he's hard to
work with.  Discriminate him because he's not a developer you can talk
with, and I believe that's like discriminating a guy in a wheelchair
because he can't run with you when you jog in the morning.

> improve core code where it makes sense for them)

Not everyone has the same "common sense" that you do.  Explain, fully,
with reasoning, and reproducable back-up statistics on common
hardware, what code is wrong, and what must be written instead.  We'd
like to be efficient, and it's not being efficient to play a guessing
game with us.  If you don't have the time to review, then please hold
off on replying until you have a through review of at least part of
the code.

Unless you object fully to one particular, fixable thing that isn't
the core of Reiser4, it'd be nice for you to wait on replying --
vagueness is not helpful to development in any way.  Are we supposed
to be the million monkeys randomly typing on a million typewriters
waiting for someone to give you code that you like, one in a million
years?

Also, let's say that Reiser4 doesn't get into the kernel, as maybe XFS
or ext2 or ext3 had never gotten into the kernel.  How would their
development be now as opposed to how we see it, when they have gotten
into the kernel?  I don't see anything wrong with the idea of letting
what seems a mostly mature FS into the kernel; that is how most bugs
are found in the first place.  Of course, there is nothing wrong with
putting huge warnings on the FS; I'd recommend them, considering that
some people are having funky problems with the patch.

I'm willing to go compare Reiser4 to ext2/3 as like H.264 to Mpeg-2. 
Indeed, H.264 crashes some computers, similar to Reiser4 might crash
some machines, but this is merely because Reiser4 explores new
concepts, meaning it may require hardyier hardware than ext2/3, as
H.264 requries hardier hardware than Mpeg-2.  I believe that the
concept is the same. (And all the same, media companies are embracing
H.264 - why can't you guys embrace this new idea also?) Change is
hard.  Besides, if Reiser4 is truely that flawed, we will see in a few
releases, and then afterwards if it's proven to everyone that Reiser4
is completely unrepairable and hopeless, it can then be ditched and
thrown into the trash.

My apologies for my rudeness, but I am rather fond of the developments
in Reiser4, and would love to see it in the kernel.  I am fond of
Linux too, and the work that you guys do, and it would dissapoint me
sorrily if Reiser4 never makes it into the Linux kernel.  Feel free to
say I'm a tad biased; I will say now that I probably have much less
merit in the field than you guys do.

-- 
~Mike
 - Just my two cents
 - No man is an island, and no man is unable.

Re: I request inclusion of reiser4 in the mainline kernel

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 896 bytes --]

On Sun, 18 Sep 2005 13:22:27 EDT, michael chang said:

> Give Hans a chance; and please try to understand, even if he's hard to
> work with.  Discriminate him because he's not a developer you can talk
> with, and I believe that's like discriminating a guy in a wheelchair
> because he can't run with you when you jog in the morning.

There's nothing wrong with discriminating against the guy in the wheelchair
under some circumstances - for instance, when your track team needs a new
high jumper.

Similarly, when the goal is to build a set of developers that can actually
get work accomplished, poor interpersonal communication skills can be a
major problem.

If the problem is that Hans and the rest of the kernel developers don't get
along, perhaps the most expedient thing would be for Hans to step out of the
way and have somebody else from Namesys (or elsewhere even) act as the interface.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

michael chang <thenewme91@gmail.com> wrote:
> On 9/18/05, Christoph Hellwig <hch@infradead.org> wrote:
> > On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
> > > This is it. I do not say "accept reiser4 NOW", I am saying "give Hans
> > > good code review".

> > After he did his basic homework.  Note that reviewing hans code is probably
> > at the very end of everyones todo list because every critizm of his code
> > starts a huge flamewar where hans tries to attack everyone not on his
> > party line personally.

> > I've said I'm gonna do a proper review after he has done the basic
> > homework, which he seems to have half-done now at least.  Right now he
> > hasn't finished

> Explain to us all what is this "basic homework" of which you speak.

The required cleanups have been posted (in outline at least), several
times, by unrelated people.

> > that and there's much more exciting filesystems like ocfs2 around that

> This is exciting to... whom?

To Cristoph, obviously. You should thank him for doing the (hard, boring,
thankless) work of reviewing code for free. Even if it isn't yours. As he
is doing it as community service, I wouldn't dare blame him for picking
whatever he likes most, for whatever reasons.

>                               The only thing that appears remotely
> interesting about it is that it's made by Oracle and apparently is
> supposed to be geared toward parallel server whatsits.  This might be
> helpful to corporations, but seems senseless toward many consumers. 

Cristoph finds it interesting, that should be enough for everybody not
paying him for doing the work.

> (I'm assuming there's still at least one consumer left who still uses
> Linux.)

Count me in. That doesn't mean I agree with ReiserFS' goals...

> > are much easier to read and actually have developers that you can have
> > a reasonable conversation with.  (and that unlike hans actually try to

> Is that Hans' fault, or the fault of your lot?  Why can't we all just get
> along?

Hans is one person, and he has managed to alienate a most of the LKML
bunch. Sure, there are very abrasive people here, but there are plenty that
are extremely helpful to newbies that /really/ want to learn how to do
things right.

> Give Hans a chance; and please try to understand, even if he's hard to
> work with.  Discriminate him because he's not a developer you can talk
> with, and I believe that's like discriminating a guy in a wheelchair
> because he can't run with you when you jog in the morning.

Please consider that most people here are volunteers, they owe nobody
nothing. Quite the contrary: if somebody wants to unload their code (and
its future maintenance) on the kernel crew, they are in /great/ debt if it
gets accepted.

> > improve core code where it makes sense for them)

> Not everyone has the same "common sense" that you do.  Explain, fully,
> with reasoning, and reproducable back-up statistics on common hardware,
> what code is wrong, and what must be written instead.  We'd like to be
> efficient, and it's not being efficient to play a guessing game with us.
> If you don't have the time to review, then please hold off on replying
> until you have a through review of at least part of the code.

Can't do. It is mostly an artistic sense of taste.

> Unless you object fully to one particular, fixable thing that isn't the
> core of Reiser4, it'd be nice for you to wait on replying -- vagueness is
> not helpful to development in any way.  Are we supposed to be the million
> monkeys randomly typing on a million typewriters waiting for someone to
> give you code that you like, one in a million years?

You are the ones that want to benefit from having your code in the kernel.
You evaluate if the (standard!) cycle of "Code, propose, get rejected, fix,
repropose, ... until finally accepted" works for you or just isn't worth
the bother.

> Also, let's say that Reiser4 doesn't get into the kernel, as maybe XFS
> or ext2 or ext3 had never gotten into the kernel.  How would their
> development be now as opposed to how we see it, when they have gotten
> into the kernel?  I don't see anything wrong with the idea of letting
> what seems a mostly mature FS into the kernel; that is how most bugs
> are found in the first place.  Of course, there is nothing wrong with
> putting huge warnings on the FS; I'd recommend them, considering that
> some people are having funky problems with the patch.

Just unloading some untested code on unsuspecting, innocent users is not
very nice, is it?

> I'm willing to go compare Reiser4 to ext2/3 as like H.264 to Mpeg-2. 
> Indeed, H.264 crashes some computers, similar to Reiser4 might crash
> some machines, but this is merely because Reiser4 explores new
> concepts, meaning it may require hardyier hardware than ext2/3, as
> H.264 requries hardier hardware than Mpeg-2.

Either one crashing the machine is unacceptable (in principle at least). A
filesystem is so central to "everything is a file" that filesystem problems
are doubly unacceptable. There are lots of reports of ReiserFS 3
filesystems completely destroyed by minor hardware flakiness. And that has
/never/ been fixed, as the developers just went off to do the "next cool
thing". That history weighs against ReiserFS, heavily.

>                                               I believe that the
> concept is the same. (And all the same, media companies are embracing
> H.264 - why can't you guys embrace this new idea also?) Change is
> hard.  Besides, if Reiser4 is truely that flawed, we will see in a few
> releases, and then afterwards if it's proven to everyone that Reiser4
> is completely unrepairable and hopeless, it can then be ditched and
> thrown into the trash.

It is cheaper for everybody involved to throw it in the thrash /before/
lots of people are dependent on it, and throw it out then. Just consider
the pain caused by including (and now ditching) devfs.

> My apologies for my rudeness, but I am rather fond of the developments
> in Reiser4, and would love to see it in the kernel.  I am fond of
> Linux too, and the work that you guys do, and it would dissapoint me
> sorrily if Reiser4 never makes it into the Linux kernel.  Feel free to
> say I'm a tad biased; I will say now that I probably have much less
> merit in the field than you guys do.

All this has been discussed in BIGNUM flamewars already, neither your nor
my post helps a bit in either way (except in fanning the flames), so please
let's drop this.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[David Masover]

Horst von Brand wrote:
> There are lots of reports of ReiserFS 3
> filesystems completely destroyed by minor hardware flakiness.

Honestly, this is one of the things I like about Linux.  If I have 
memory errors, Windows will just keep running, occasionally something 
will crash, you restart it, never suspecting just how corrupt things are 
getting under the hood.  On Linux, I generally get kernel panics pretty 
quickly, so I run memtest86 and replace the RAM.

If my hardware is flaky, I consider it my job to replace it, not the job 
of all my software to magically compensate for it.  If I lose data, oh 
well, I have backups.  If I didn't, I was asking for trouble anyway.

Re: I request inclusion of reiser4 in the mainline kernel

[Dan Oglesby]

David Masover wrote:

> Horst von Brand wrote:
>
>> There are lots of reports of ReiserFS 3
>> filesystems completely destroyed by minor hardware flakiness.
>
>
> Honestly, this is one of the things I like about Linux.  If I have 
> memory errors, Windows will just keep running, occasionally something 
> will crash, you restart it, never suspecting just how corrupt things 
> are getting under the hood.  On Linux, I generally get kernel panics 
> pretty quickly, so I run memtest86 and replace the RAM.
>
> If my hardware is flaky, I consider it my job to replace it, not the 
> job of all my software to magically compensate for it.  If I lose 
> data, oh well, I have backups.  If I didn't, I was asking for trouble 
> anyway.

I'm of the same opinion.  If I have hardware that has a problem, and 
causes downtime, it gets replaced or repaired.  I don't switch to a 
different piece of software to compensate for broken hardware.

With that said, I have seen ReiserFS expose hardware that had problems.  
Hardware was repaired, and ReiserFS rides again.

--Dan

Re: I request inclusion of reiser4 in the mainline kernel

[PFC]

> I'm of the same opinion.  If I have hardware that has a problem, and  
> causes downtime, it gets replaced or repaired.  I don't switch to a  
> different piece of software to compensate for broken hardware.
>
> With that said, I have seen ReiserFS expose hardware that had problems.   
> Hardware was repaired, and ReiserFS rides again.

	This summer :

	Coming back from vacation, looking at the logs, I saw that the cupboard  
router-server had kernel-panicked almost daily and rebooted itself  
automatically. I also had a lot of corrupted BitTorrent downloads. I could  
have blamed reiserfs, or bittorrent. But instead, I opened the case and  
found the CPU was overheating due to the fan being clogged by an  
unbelievable amount of accumulated dust and crap.

	reiserfs was still happy, I ran a fsck just to be sure, no errors. fhew.  
I wonder how it's possible. Given the state of the CPU fan, everything  
should have been wiped out.

	I have an all-reiser4 laptop (except /boot) and it's great. No problems  
whatsoever, it flies. Pentium-M kicks ass.
	My jukebox PC is half reiser3 and (since a few months) half reiser4,  
running fine, on the cheapest possible motherboard, and the no-name RAM,  
with an underclocked Duron. The hardware is so bad I had to underclock the  
PC133 to PC100. It has never crashed in 4 years, or got any data  
corruption. Crap hardware is actually sometimes pretty good if you  
underclock it (just have to get lucky). With windows, it used to  
bluescreen just by plugging a cable in the ethernet port.
	My server is all reiser3 too.

	I could have used other filesystems but reiserfs Just Works. No horror  
stories to tell, sorry. I  like reiserfs.
	I don't care it there were very old versions that crashed. I don't care  
about Linux 2.0 or 1 either. Or Netscape 2. That's the past now.

Re: I request inclusion of reiser4 in the mainline kernel

[Kyle Moffett]

Enough with the "It works for me" arguments!!!  Seriously!!!  Yes, we  
know, it works for some people, but not for others.  There are a  
billion factors.  If you have a technical argument, technical point,  
useful detailed bug report, patch, etc to make, do so, but otherwise  
SHUT UP!!!

I'm interested in doing productive code-review so we can get the  
sucker into the kernel at some point (yes, it will almost certainly  
happen eventually), but with all the bitching and non-technical  
rhetoric going on, some of you people are really wasting a lot of our  
time when we're trying to converse usefully with Hans about code- 
related issues.  If you don't have something _productive_ to say on  
the thread, don't post!

Can we keep the Reiser4 S/N ratio at a manageable level, please?

Cheers,
Kyle Moffett

--
I lost interest in "blade servers" when I found they didn't throw  
knives at people who weren't supposed to be in your machine room.
   -- Anthony de Boer

Re: I request inclusion of reiser4 in the mainline kernel

[Dr.Dre]

I have a bug report for the first time about reiser4 in 2.6.14-rc1-mm1
with 4k stacks,
preempt and smp. It is the first time I face a bug after using reiser4
for about a year. Well I had to with 4k stacks right ?
firefox has triggerred the bug twice and I had to fsck the filesystem
with --fix --build-fs which fixed the error. After fixing it any
attempt to access the drive resulted in a 'Bus error' message. A sync
and reboot using the sysreq mechanism returned me to a working system.

Sorry if this is not the place to report the bug, or if doesn't get
attatched to the reiser4 thread, I am new to LKML. Thanks in advance.

------------[ cut here ]------------
kernel BUG at <bad filename>:59883!
invalid operand: 0000 [#1]
PREEMPT SMP
last sysfs file: /class/sound/seq/dev
Modules linked in: snd_seq_instr snd_seq_midi_emul snd_seq_midi
snd_seq_midi_event snd_seq firmware_class nls_utf8 nls_cp864 vfat fat
nls_base af_packet joydev tsdev ohci_hcd ehci_hcd yealink usbhid
mousedev nvidia snd_pcm_oss snd_mixer_oss video via_rhine uhci_hcd
usbcore tpm_nsc tpm_infineon tpm_atmel tpm thermal speedstep_lib
snd_cmipci gameport snd_pcm snd_page_alloc snd_opl3_lib snd_timer
snd_hwdep snd_mpu401_uart snd_rawmidi snd_seq_device snd soundcore
shpchp pci_hotplug rtc processor loop intel_agp agpgart i2c_i801
i2c_core fan cpufreq_userspace cpufreq_stats freq_table
cpufreq_powersave cpufreq_ondemand cpufreq_conservative container
button battery
CPU:    0
EIP:    0060:[<c01d56fb>]    Tainted: P      VLI
EFLAGS: 00010297   (2.6.14-rc1-mm1)
EIP is at sub_from_ctx_grabbed+0x2b/0x30
eax: 00000000   ebx: 00000000   ecx: 00000001   edx: 00000000
esi: d24deec0   edi: df69e800   ebp: d50fc9e0   esp: cea25d8c
ds: 007b   es: 007b   ss: 0068
Process firefox-bin (pid: 10393, threadinfo=cea25000 task=de659050)
Stack: 00000001 00000000 c01d63c0 d24deec0 00000001 00000000 de202680 d50fc9e0
      d50fc9e0 ce6cb8d4 c01d8594 d50fc9e0 00000001 00000000 de202680 d50fc9e0
      de2026b4 c01d85e7 de202680 d50fc9e0 00000000 de202680 c01d861e de202680
Call Trace:
 [<c01d63c0>] grabbed2flush_reserved_nolock+0x30/0x70
 [<c01d8594>] do_jnode_make_dirty+0xf4/0x120
 [<c01d85e7>] jnode_make_dirty_locked+0x27/0x40
 [<c01d861e>] znode_make_dirty+0x1e/0x90
 [<c01ef1b5>] update_sd_at+0xc5/0x1f0
 [<c01ef32d>] update_sd+0x4d/0x70
 [<c01ee5fb>] write_sd_by_inode_common+0x8b/0x90
 [<c01e37c8>] reiser4_dirty_inode+0x18/0x70
 [<c0180883>] __mark_inode_dirty+0xb3/0x190
 [<c01784c4>] update_atime+0x54/0x80
 [<c01f1aee>] read_unix_file+0x35e/0x3c0
 [<c015d316>] vfs_read+0xa6/0x140
 [<c015d64d>] sys_read+0x3d/0x70
 [<c0102d7b>] sysenter_past_esp+0x54/0x79
Code: 56 53 8b 74 24 0c 8b 5c 24 14 8b 4c 24 10 8b 56 78 8b 46 74 39
da 76 0d 29 c8 19 da 89 46 74 89 56 78 5b 5e c3 72 04 39 c8 73 ed <0f>
0b eb e9 90 8b 4c 24 04 8b 41 74 8b 51 78 03 44 24 08 13 54
 <6>note: firefox-bin[10393] exited with preempt_count 3


Please request any extra info you need.

Thanks and keep up the good work.

Re: I request inclusion of reiser4 in the mainline kernel

[Marc Perkel]

PFC wrote:

>
>
>> I'm of the same opinion.  If I have hardware that has a problem, and  
>> causes downtime, it gets replaced or repaired.  I don't switch to a  
>> different piece of software to compensate for broken hardware.
>>
>> With that said, I have seen ReiserFS expose hardware that had 
>> problems.   Hardware was repaired, and ReiserFS rides again.
>
>
>

Agreed - if the hardware has problem and anything is readable I'm happy. 
When I was sysadmin at EFF we got a bunch of IBM Deathstar drives - and 
for those who experiences this - every one of them fails. But they 
usually fail slowly. What amazed me was I would stat to see seek errors 
- sector not found and I would copy off everything I could onto a new 
drive before I lost anything. And - I thought it was amazing that I 
usually managed to get all the important stuff. So - I give reiser 
credit for being somewhat resiliant.

here's the way I see it. This isn't like Hans Reiser is some unknown guy 
who has some wild idea that we all don't know. ReiserFS is a majoy 
player in the Linux world and many people like it the best. Several 
distros use Reiser as their default install. So to me this gives him 
more than average standing and the way I see it - there has to be a good 
reason to NOT merge it rather than a reason TO merge it.

So - is Reiser4 going to break anything? If not - what is the reason to 
not do it?

-- 
Marc Perkel - marc@perkel.com

Spam Filter: http://www.junkemailfilter.com
    My Blog: http://marc.perkel.com

Re: I request inclusion of reiser4 in the mainline kernel

[Marc Perkel]

For what it's worth sometimes people get emotional and frustrated and 
sometimes people can be difficult at thimes to work with. But - for what 
it's worth - I think people should ignore some of that as human nature 
and look at the big picture.

And the big picture is ....

Hans has make a huge contribution with Reiser 3 and eventually Reiser 4 
is going to be something that will greatly enhance the kernel and 
advance Linux the way Reiser 3 has done. Wasn't Resiser 3 the first 
journalling file system for Linux that actually worked?

So - I say - if it doesn't break anything else - why not throw it into 
the mail kernel? It will get there eventually and if it's there sooner 
them more people will be out there trying to break it and it will 
develop faster. I don't know personally how stable it is - but from what 
I understand it is winning the speed tests and that will shave some time 
off of everything the rest of us do. And even if it is somewhat broken - 
in a few months it will all be fixed.

And Hans is apparently ready to take abuse if it's broken.

So I say - lets do it already.

My 2 centz ....

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Horst von Brand wrote:

>
>>>that and there's much more exciting filesystems like ocfs2 around that
>>>      
>>>
>
>  
>
>>This is exciting to... whom?
>>    
>>
>
>To Cristoph, obviously. You should thank him for doing the (hard, boring,
>thankless) work of reviewing code for free. Even if it isn't yours. As he
>is doing it as community service, I wouldn't dare blame him for picking
>whatever he likes most, for whatever reasons.
>  
>
Well maybe he should just go away then and save his and our time. 
Reiser4 works just fine without Christoph.  Users are happy with it. 
None of them have asked for his help.  I don't consider Christoph to be
qualified to work on our filesystem.  I would not hire him if he applied
--- he is not capable of innovative work.

Reiser4 is far from perfect, but it is ready for more users.

>
>>Is that Hans' fault, or the fault of your lot?  Why can't we all just get
>>along?
>>    
>>
>
>Hans is one person, and he has managed to alienate a most of the LKML
>bunch. Sure, there are very abrasive people here, but there are plenty that
>are extremely helpful to newbies that /really/ want to learn how to do
>things right.
>  
>
Yes, but the helpful ones have nothing to do with VFS.  Linux has lost
filesystem developers because of the VFS team, developers who I can tell
you were very very  gifted DARPA researchers who decided to work on BSD
because they had too much dignity to develop a filesystem for  Linux.  I
assure you that no one on the VFS team is as bright or capable as one of
the fellows I know of that they abused away.

>>If you don't have the time to review, then please hold off on replying
>>until you have a through review of at least part of the code.
>>    
>>
>
>Can't do. It is mostly an artistic sense of taste.
>  
>
Yes, which is why people who have not written a serious filesystem
should not instruct those who have written the measurably fastest one.

> Also, let's say that Reiser4 doesn't get into the kernel, as maybe XFS
>
>>or ext2 or ext3 had never gotten into the kernel.  How would their
>>development be now as opposed to how we see it, when they have gotten
>>into the kernel?  I don't see anything wrong with the idea of letting
>>what seems a mostly mature FS into the kernel; that is how most bugs
>>are found in the first place.  Of course, there is nothing wrong with
>>putting huge warnings on the FS; I'd recommend them, considering that
>>some people are having funky problems with the patch.
>>    
>>
>
>Just unloading some untested code on unsuspecting, innocent users is not
>very nice, is it?
>  
>
Christoph is not testing.  We have tested, our mailing list has tested.....

> There are lots of reports of ReiserFS 3
>
>filesystems completely destroyed by minor hardware flakiness. And that has
>/never/ been fixed, as the developers just went off to do the "next cool
>thing". That history weighs against ReiserFS, heavily.
>  
>
We are supposed to write a filesystem so that overheating CPUs do not
make it crash?

Prejudice is a very simple phenomenom.  When either ext3 or ReiserFS V3
crash it is almost always due to bad hardware.  Prejudice is the process
of remembering that one filesystem crashed due to bad hardware and not
remembering that the other one crashed.

It is remarkably simple how it works: people who use Reiser4 want it in,
people who use ext3 and don't want to have a choice of something else
don't want it in.  That was true of V3, and it is true of V4.  My point
of course is that those who have used V4 know more about it than those
who haven't......

I think Alan Cox is the only poster who has no intention of using
Reiser4 but said at one point that he thinks it should go in.

V3 is obsoleted by V4 in every way.  V3 is old code that should be
marked as deprecated as soon as V4 has passed mass testing.   V4 is far
superior in its coding style also.  Having V3 in and V4 out is at this
point just stupid. 

This whole thing reminds me of an IBMer who told me that he thought that
IBM lost to MS because they called OS/2 by a name other than DOS.  The
sad thing is he was probably right. 

V4, as it is today, is as much superior to V3 as OS/2 was to DOS.  Any
distro or user who would stay with V3 for new installs once we have
passed mass testing is nuts.  We need the mass testing.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Hans Reiser writes:

[...]

 > Well maybe he should just go away then and save his and our time. 
 > Reiser4 works just fine without Christoph.  Users are happy with it. 
 > None of them have asked for his help.  I don't consider Christoph to be
 > qualified to work on our filesystem.  I would not hire him if he applied
 > --- he is not capable of innovative work.

Hans, it seems you misunderstand what is going on.

Necessary prerequisite for reiser4 inclusion is a review by file system
people. Christoph is doing this _work_. You know what it is to review
somebody else code? It's a hard work. He is doing it for free.

If he goes away---reiser4 wouldn't somehow magically be immediately
included into kernel. It'll wait for review---for the next person to
come and to do this work. And that person will most likely point to the
very same things that hch does now only after a delay. Maybe with
slightly more polished style (hair style included) though. :-)

So, please be a little more patient.

 > 
 > Reiser4 is far from perfect, but it is ready for more users.

This is not what is being discussed, stop red-herring, please.

[...]

 > >
 > Yes, which is why people who have not written a serious filesystem
 > should not instruct those who have written the measurably fastest one.

Somebody has to review reiser4 code. If you can persuade somebody else
to do the review, do this.

[...]

 > 
 > V4, as it is today, is as much superior to V3 as OS/2 was to DOS.  Any

One way in which half-OS was superior to DOS was how spectacularly the
former project failed after a lot of hype. :-)

 > distro or user who would stay with V3 for new installs once we have
 > passed mass testing is nuts.  We need the mass testing.
 > 
 > Hans
 > 

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

>One way in which half-OS was superior to DOS was how spectacularly the
>former project failed after a lot of hype. :-)
>  
>
OS/2 was technically superior to windows 95.  The only failure was the
marketing/herd movement issues.  I used OS/2, it was better.

Re: I request inclusion of reiser4 in the mainline kernel

[Alan Cox]

On Sul, 2005-09-18 at 22:44 -0700, Hans Reiser wrote:
> We are supposed to write a filesystem so that overheating CPUs do not
> make it crash?

The reverse - and before you lose data.

> I think Alan Cox is the only poster who has no intention of using
> Reiser4 but said at one point that he thinks it should go in.

If its clean enough then yes, like any other fs. Until then no.

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

Hans Reiser <reiser@namesys.com> wrote:
> Horst von Brand wrote:
> >>>that and there's much more exciting filesystems like ocfs2 around that

> >>This is exciting to... whom?

> >To Cristoph, obviously. You should thank him for doing the (hard, boring,
> >thankless) work of reviewing code for free. Even if it isn't yours. As he
> >is doing it as community service, I wouldn't dare blame him for picking
> >whatever he likes most, for whatever reasons.

> Well maybe he should just go away then and save his and our time. 

Your choice.

> Reiser4 works just fine without Christoph.  Users are happy with it. 
> None of them have asked for his help.  I don't consider Christoph to be
> qualified to work on our filesystem.  I would not hire him if he applied
> --- he is not capable of innovative work.

How do you know?

> Reiser4 is far from perfect, but it is ready for more users.

OK.

> >>Is that Hans' fault, or the fault of your lot?  Why can't we all just get
> >>along?

> >Hans is one person, and he has managed to alienate a most of the LKML
> >bunch. Sure, there are very abrasive people here, but there are plenty that
> >are extremely helpful to newbies that /really/ want to learn how to do
> >things right.

> Yes, but the helpful ones have nothing to do with VFS.

It just so happens that VFS is a central piece of Linux, and it is has been
carefully engineered over the years, for /lots/ of different filesystems. I
can understand that they don't want to change it willy-nilly.

>                                                        Linux has lost
> filesystem developers because of the VFS team, developers who I can tell
> you were very very  gifted DARPA researchers

Care to give names?

>                                              who decided to work on BSD
> because they had too much dignity to develop a filesystem for  Linux.

Their loss, I presume... and you clearly think the same way, because if it
weren't you wouldn't be around here trying to push ReiserFS into Linux, and
would be bothering one of the BSDs instead.

>                                                                        I
> assure you that no one on the VFS team is as bright or capable as one of
> the fellows I know of that they abused away.

Yo don't know the whole "VFS team", so...

> >>If you don't have the time to review, then please hold off on replying
> >>until you have a through review of at least part of the code.

> >Can't do. It is mostly an artistic sense of taste.

> Yes, which is why people who have not written a serious filesystem
> should not instruct those who have written the measurably fastest one.

It might be fast, but if the code is horrible, I don't want anything to do
with it. I prefer the code on which I depend reviewable, thank you very
much.

> >> Also, let's say that Reiser4 doesn't get into the kernel, as maybe XFS
> >>or ext2 or ext3 had never gotten into the kernel.  How would their
> >>development be now as opposed to how we see it, when they have gotten
> >>into the kernel?  I don't see anything wrong with the idea of letting
> >>what seems a mostly mature FS into the kernel; that is how most bugs
> >>are found in the first place.  Of course, there is nothing wrong with
> >>putting huge warnings on the FS; I'd recommend them, considering that
> >>some people are having funky problems with the patch.

> >Just unloading some untested code on unsuspecting, innocent users is not
> >very nice, is it?

> Christoph is not testing.  We have tested, our mailing list has
> tested.....

I'm not talking about Cristoph, I'm talking about unsuspecting users of
Linus' kernel (and users of distributions that will end up enabling
everything under Kconfig "just in case").

> >                                 There are lots of reports of ReiserFS 3
> >filesystems completely destroyed by minor hardware flakiness. And that has
> >/never/ been fixed, as the developers just went off to do the "next cool
> >thing". That history weighs against ReiserFS, heavily.

> We are supposed to write a filesystem so that overheating CPUs do not
> make it crash?

No... but at least try not to get completely hosed in that case.

> Prejudice is a very simple phenomenom.  When either ext3 or ReiserFS V3
> crash it is almost always due to bad hardware.  Prejudice is the process
> of remembering that one filesystem crashed due to bad hardware and not
> remembering that the other one crashed.

No. Here it is starkly remembering the cases where the crash got people to
loose /all/ their data, versus at best dimly remembering cases where a file
got into lost+found and had to be restored.

> It is remarkably simple how it works: people who use Reiser4 want it in,

Fair enough.

> people who use ext3 and don't want to have a choice of something else
> don't want it in.

As a ext3 user, I could not care less for XFS, or JFS, or... What I /do/
care about is the code quality of the kernel. And people who don't want to
(or can't) work nicely with the kernel crew just will screw it up, so I'd
veto their code.

>                    That was true of V3, and it is true of V4.  My point
> of course is that those who have used V4 know more about it than those
> who haven't......

People who work all day on the kernel and have looked at the code, and get
misdirected reports and outcries, know more about it than occassional
users...

> I think Alan Cox is the only poster who has no intention of using
> Reiser4 but said at one point that he thinks it should go in.

I too think that Linux can only win with more filesystems, as long as they
are maintainable and don't mean a burden on the core kernel crew. Note that
this means code in the style of the core kernel, and a developer community
that works well with the core crew.

> V3 is obsoleted by V4 in every way.  V3 is old code that should be
> marked as deprecated as soon as V4 has passed mass testing.   V4 is far
> superior in its coding style also.  Having V3 in and V4 out is at this
> point just stupid. 

V3 /is/ being used by many people. Just decreeing it has to be pulled out
"because V4 is better" is just irresponsible. And that is exactly the
attitude I don't like.

> This whole thing reminds me of an IBMer who told me that he thought that
> IBM lost to MS because they called OS/2 by a name other than DOS.  The
> sad thing is he was probably right. 

The IBMer was dead wrong, OS/2 was strangled by lack of applications.

> V4, as it is today, is as much superior to V3 as OS/2 was to DOS.  Any
> distro or user who would stay with V3 for new installs once we have
> passed mass testing is nuts.  We need the mass testing.

And people who have existing setups don't count? Cautious people who won't
touch a new filesystem with a ten foot pole until it has got mass testing
in a mainstream distribution for a couple of years are irrelevant?

BTW, your post (and my answer, for that matter) don't add any value to
LKML, so please stop here.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Horst von Brand wrote:

> Care to give names?

Not publicly, no.  If akpm or Linus asks, I will happily encourage
either of them to try to win him back.

>  
>
>>                                             who decided to work on BSD
>>because they had too much dignity to develop a filesystem for  Linux.
>>    
>>
>
>Their loss, I presume...
>
everyone's loss, it is a negative sum game.

> and you clearly think the same way, because if it
>weren't you wouldn't be around here trying to push ReiserFS into Linux, and
>would be bothering one of the BSDs instead.
>  
>
I committed to Linux way back when it was obscure compared to BSD..... 
and I like the way the GPL allows me to make some small amount of money
selling licenses in addition to the GPL, and the way MS can't reuse my
code for free without attribution.

>
>  
>
>>V3 is obsoleted by V4 in every way.  V3 is old code that should be
>>marked as deprecated as soon as V4 has passed mass testing.   V4 is far
>>superior in its coding style also.  Having V3 in and V4 out is at this
>>point just stupid. 
>>    
>>
>
>V3 /is/ being used by many people. Just decreeing it has to be pulled out
>"because V4 is better" is just irresponsible. And that is exactly the
>attitude I don't like.
>  
>
I didn't say V3 should be pulled out, I said marked as deprecated.   And
left alone so that it gets only bug fixes and not new features.

>  
>
>>This whole thing reminds me of an IBMer who told me that he thought that
>>IBM lost to MS because they called OS/2 by a name other than DOS.  The
>>sad thing is he was probably right. 
>>    
>>
>
>The IBMer was dead wrong, OS/2 was strangled by lack of applications.
>  
>
Uh, no, it ran all the DOS applications better than DOS did.  Seriously,
it did.  I remember DOS applications that were easier to make work under
OS/2 due to 640k something or another.  I pray the memory of exactly
what does not return to me.;-)

>  
>
>>V4, as it is today, is as much superior to V3 as OS/2 was to DOS.  Any
>>distro or user who would stay with V3 for new installs once we have
>>passed mass testing is nuts.  We need the mass testing.
>>    
>>
>
>And people who have existing setups don't count?
>
Of course they count.  I don't expect any users to convert from V3 to V4
on an existing OS installation until we have a robustly working
convertfs, and even then most of them won't.  Please note the phrase
"new installs", it was very intentional.

> Cautious people who won't
>touch a new filesystem with a ten foot pole until it has got mass testing
>in a mainstream distribution for a couple of years are irrelevant?
>  
>
I fully expect that there will be people using V3 long past the time
when better filesystems are available and working and stable, the world
has a lot of lag time to it.;-)

Re: I request inclusion of reiser4 in the mainline kernel

[Pavel Machek]

Hi!

> V3 is obsoleted by V4 in every way.  V3 is old code that should be
> marked as deprecated as soon as V4 has passed mass testing.   V4 is far
> superior in its coding style also.  Having V3 in and V4 out is at this
> point just stupid. 

Really? Last time I checked, even V3's fsck was not too great. [In
fact I never could make it run stable enough to even _test_ it
properly].

Do you have working fsck for V4? Until then, you should not claim that
users should switch. Journalling does not help you, if you have
unexpected kernel problem or hardware trouble, fsck _is_ mandatory.

Can V4 survive few hours of test below?
								Pavel

#!/bin/bash
#
# fscktest
#
# Usage: 
#	 Make sure output is logged somewhere
#        First, run fscktest -p as root
#	 Then you can run fscktest as normal user...
#

prepare() {
	SIZE=100000
	echo "Creating file..."
	cat /dev/zero | head -c $[$SIZE*1024] > test
	echo "Making filesystem..."
	mkfs.$FS test
	echo "Mounting..."
	mount test -o loop /mnt || exit "Cant mount"
	echo "Copying files..."
	cp -a /bin /mnt
	cp -a /usr/bin /mnt
	cp -a /usr/src/linux /mnt
	echo "Syncing..."
	sync
	echo "Unmounting..."
	umount /mnt
	echo "Moving..."
	mv test fsck.okay
	echo "All done."
}

FS=ext2
if [ .$1 == .-p ]; then
	prepare
	exit
	fi
RUN=0
while true; do
	RUN=$[$RUN+1]
	echo "Run #$RUN"
	echo Preparing...
	cat fsck.okay > fsck.damaged
	echo Damaging...
	dd if=/dev/urandom of=fsck.damaged count=10240 seek=7 conv=notrunc
	cp fsck.damaged fsck.test
	echo First check...
	fsck.$FS -fy fsck.damaged
	RESULT=$?
	if [ $RESULT != 1 -a $RESULT != 2 -a $RESULT != 0 ]; then
		echo "Fsck failed in bad way (result = $RESULT)"
		exit
		fi
	echo Second check...
	fsck.$FS -fy fsck.damaged
	RESULT=$?
	if [ $RESULT != 0 ]; then
		echo "Fsck lied about its success (result = $RESULT)"
		exit
		fi
	done

-- 
if you have sharp zaurus hardware you don't need... you know my address

Re: I request inclusion of reiser4 in the mainline kernel

[David Masover]

Pavel Machek wrote:
> Hi!
> 
> 
>>V3 is obsoleted by V4 in every way.  V3 is old code that should be
>>marked as deprecated as soon as V4 has passed mass testing.   V4 is far
>>superior in its coding style also.  Having V3 in and V4 out is at this
>>point just stupid. 
> 
> 
> Really? Last time I checked, even V3's fsck was not too great. [In
> fact I never could make it run stable enough to even _test_ it
> properly].
> 
> Do you have working fsck for V4?

Already saved me once.  But then, I should have had backups.

And personally, if it was my FS, I'd stop working on fsck after it was 
able to "check".  That's what it's for.  To fix an FS, you wipe it and 
restore from backups.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

David Masover wrote:

>
> And personally, if it was my FS, I'd stop working on fsck after it was
> able to "check".  That's what it's for.  To fix an FS, you wipe it and
> restore from backups.
>
>
Umm, this is going too far David.  Our fsck should work, and we will
give his script to Vitaly to play with and comment on.

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

David Masover <ninja@slaphack.com> wrote:
> Pavel Machek wrote:
> >>V3 is obsoleted by V4 in every way.  V3 is old code that should be
> >>marked as deprecated as soon as V4 has passed mass testing.   V4 is far
> >>superior in its coding style also.  Having V3 in and V4 out is at this
> >> point just stupid.

> > Really? Last time I checked, even V3's fsck was not too great. [In
> > fact I never could make it run stable enough to even _test_ it
> > properly].
> > Do you have working fsck for V4?

> Already saved me once.  But then, I should have had backups.

And if you haven't? Backing up 200+ GiB is no peanuts...

> And personally, if it was my FS, I'd stop working on fsck after it was
> able to "check".  That's what it's for.  To fix an FS, you wipe it and
> restore from backups.

Remind me /never/ to go near a filesystem you are in any way involved with.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[Theodore Ts'o]

On Tue, Sep 20, 2005 at 12:18:46PM -0600, Jonathan Briggs wrote:
> 
> I use Reiser3 and Reiser4 on all my systems and fsck has always worked
> even if it has been much slower than I would like.  The only problems
> I've experienced have been on the same level as when an ext2/3
> filesystem fsck dumps several directories of unlabeled files into lost
> +found.

You've obviously never kept several dozen reiserfs filesystem images
(for use with Xen or User-Mode Linux) on a reiserfs filesystem, and
then had a hardware failure bad enough that the fsck had to try to
rebuild the b-tree, I take it?

						- Ted

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Theodore Ts'o wrote:

>On Tue, Sep 20, 2005 at 12:18:46PM -0600, Jonathan Briggs wrote:
>  
>
>>I use Reiser3 and Reiser4 on all my systems and fsck has always worked
>>even if it has been much slower than I would like.  The only problems
>>I've experienced have been on the same level as when an ext2/3
>>filesystem fsck dumps several directories of unlabeled files into lost
>>+found.
>>    
>>
>
>You've obviously never kept several dozen reiserfs filesystem images
>(for use with Xen or User-Mode Linux) on a reiserfs filesystem, and
>then had a hardware failure bad enough that the fsck had to try to
>rebuild the b-tree, I take it?
>  
>
That is fixed in V4.  Until people start to use V4 they should compress
their V3 backup images that they store on V3, or store them on separate
partitions.  I regret that fixing it without a disk format change was
not possible.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Vitaly Fertman]

On Wednesday 21 September 2005 01:11, Theodore Ts'o wrote:
> On Tue, Sep 20, 2005 at 12:18:46PM -0600, Jonathan Briggs wrote:
> > 
> > I use Reiser3 and Reiser4 on all my systems and fsck has always worked
> > even if it has been much slower than I would like.  The only problems
> > I've experienced have been on the same level as when an ext2/3
> > filesystem fsck dumps several directories of unlabeled files into lost
> > +found.
> 
> You've obviously never kept several dozen reiserfs filesystem images
> (for use with Xen or User-Mode Linux) on a reiserfs filesystem, and
> then had a hardware failure bad enough that the fsck had to try to
> rebuild the b-tree, I take it?

loop device XOR encryption for your images is the simple solution 
for reiserfs V3.

-- 
Thanks,
Vitaly Fertman

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Thanks much for this test script, Vitaly, please give it a try and tell
me how it goes.

Hans

Pavel Machek wrote:

>Hi!
>
>  
>
>>V3 is obsoleted by V4 in every way.  V3 is old code that should be
>>marked as deprecated as soon as V4 has passed mass testing.   V4 is far
>>superior in its coding style also.  Having V3 in and V4 out is at this
>>point just stupid. 
>>    
>>
>
>Really? Last time I checked, even V3's fsck was not too great. [In
>fact I never could make it run stable enough to even _test_ it
>properly].
>
>Do you have working fsck for V4? Until then, you should not claim that
>users should switch. Journalling does not help you, if you have
>unexpected kernel problem or hardware trouble, fsck _is_ mandatory.
>
>Can V4 survive few hours of test below?
>								Pavel
>
>#!/bin/bash
>#
># fscktest
>#
># Usage: 
>#	 Make sure output is logged somewhere
>#        First, run fscktest -p as root
>#	 Then you can run fscktest as normal user...
>#
>
>prepare() {
>	SIZE=100000
>	echo "Creating file..."
>	cat /dev/zero | head -c $[$SIZE*1024] > test
>	echo "Making filesystem..."
>	mkfs.$FS test
>	echo "Mounting..."
>	mount test -o loop /mnt || exit "Cant mount"
>	echo "Copying files..."
>	cp -a /bin /mnt
>	cp -a /usr/bin /mnt
>	cp -a /usr/src/linux /mnt
>	echo "Syncing..."
>	sync
>	echo "Unmounting..."
>	umount /mnt
>	echo "Moving..."
>	mv test fsck.okay
>	echo "All done."
>}
>
>FS=ext2
>if [ .$1 == .-p ]; then
>	prepare
>	exit
>	fi
>RUN=0
>while true; do
>	RUN=$[$RUN+1]
>	echo "Run #$RUN"
>	echo Preparing...
>	cat fsck.okay > fsck.damaged
>	echo Damaging...
>	dd if=/dev/urandom of=fsck.damaged count=10240 seek=7 conv=notrunc
>	cp fsck.damaged fsck.test
>	echo First check...
>	fsck.$FS -fy fsck.damaged
>	RESULT=$?
>	if [ $RESULT != 1 -a $RESULT != 2 -a $RESULT != 0 ]; then
>		echo "Fsck failed in bad way (result = $RESULT)"
>		exit
>		fi
>	echo Second check...
>	fsck.$FS -fy fsck.damaged
>	RESULT=$?
>	if [ $RESULT != 0 ]; then
>		echo "Fsck lied about its success (result = $RESULT)"
>		exit
>		fi
>	done
>
>  
>

Re: I request inclusion of reiser4 in the mainline kernel

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 555 bytes --]

On Tue, 20 Sep 2005 23:28:12 +0400, Roman I Khimov said:
> --nextPart1692600.LIfSYN1P7A

> Maybe I'm doing something wrong here, but ext2 have failed on second check
> of first pass with
> 
> Second check...
> e2fsck 1.34 (25-Jul-2003)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure

> fsck.damaged: ***** FILE SYSTEM WAS MODIFIED *****
> fsck.damaged: 1345/25064 files (1.7% non-contiguous), 94063/100000 blocks
> fsck lied about its success (result = 1)

What was the return value and output from the *first* fsck? 

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: I request inclusion of reiser4 in the mainline kernel

[Theodore Ts'o]

On Tue, Sep 20, 2005 at 04:15:41PM -0400, Valdis.Kletnieks@vt.edu wrote:
> On Tue, 20 Sep 2005 23:28:12 +0400, Roman I Khimov said:
> > --nextPart1692600.LIfSYN1P7A
> 
> > Maybe I'm doing something wrong here, but ext2 have failed on second check
> > of first pass with
> > 
> > Second check...
> > e2fsck 1.34 (25-Jul-2003)
> > Pass 1: Checking inodes, blocks, and sizes
> > Pass 2: Checking directory structure
> 
> > fsck.damaged: ***** FILE SYSTEM WAS MODIFIED *****
> > fsck.damaged: 1345/25064 files (1.7% non-contiguous), 94063/100000 blocks
> > fsck lied about its success (result = 1)

>From the fsck man page:

       The exit code returned by fsck is the sum of the following conditions:
            0    - No errors
            1    - File system errors corrected
            2    - System should be rebooted
            4    - File system errors left uncorrected
            8    - Operational error
            16   - Usage or syntax error
            32   - Fsck canceled by user request
            128  - Shared library error

An exit code of 1 means that filesystem errors were corrected
(successfully).  

This is a convention that has been around for a long time (since at
least BSD 4.x).

						- Ted

Re: I request inclusion of reiser4 in the mainline kernel

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]

On Tue, 20 Sep 2005 17:17:13 EDT, "Theodore Ts'o" said:
>
> An exit code of 1 means that filesystem errors were corrected
> (successfully).  

Right.  The problem is that this was a *second* check, after the first one
terminated with exit code 0, 1, or 2.  Thus, it *should* have exited with 0.

The *first* check lied - if there were unfixed errors, it should have exited
with exit 4.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: I request inclusion of reiser4 in the mainline kernel

[Pavel Machek]

Hi!

> > Can V4 survive few hours of test below?
> 
> Maybe I'm doing something wrong here, but ext2 have failed on second check 
> of first pass with

No, you have probably just found bug in e2fsck...

> Second check...
> e2fsck 1.34 (25-Jul-2003)

I have 1.38 here, so yours is too old.

OTOH if reiser4 survives that for 80 cycles... that's pretty good.

									Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

Re: I request inclusion of reiser4 in the mainline kernel

[Pavel Machek]

Hi!

> > > Second check...
> > > e2fsck 1.34 (25-Jul-2003)
> > 
> > I have 1.38 here, so yours is too old.
> 
> I'll compile something new tomorrow and try to retest it.
>  
> > OTOH if reiser4 survives that for 80 cycles... that's pretty good.
> 
> Actually 125 before I've got completely tired of HDD noise. :)

At tytso's request... could you put some reiser3 and reiser4
filesystem images inside test data?
								Pavel
-- 
if you have sharp zaurus hardware you don't need... you know my address

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Pavel Machek wrote:

>Hi!
>
>  
>
>>>>Second check...
>>>>e2fsck 1.34 (25-Jul-2003)
>>>>        
>>>>
>>>I have 1.38 here, so yours is too old.
>>>      
>>>
>>I'll compile something new tomorrow and try to retest it.
>> 
>>    
>>
>>>OTOH if reiser4 survives that for 80 cycles... that's pretty good.
>>>      
>>>
>>Actually 125 before I've got completely tired of HDD noise. :)
>>    
>>
>
>At tytso's request... could you put some reiser3 and reiser4
>filesystem images inside test data?
>								Pavel
>  
>
For V3 they will break things.  For V4, sure, let's test it, I'd like to
confirm that we fixed things.

Re: I request inclusion of reiser4 in the mainline kernel

[Theodore Ts'o]

On Tue, Sep 20, 2005 at 09:51:33AM +0200, Pavel Machek wrote:
> Do you have working fsck for V4? Until then, you should not claim that
> users should switch. Journalling does not help you, if you have
> unexpected kernel problem or hardware trouble, fsck _is_ mandatory.
> 
> Can V4 survive few hours of test below?

The script could be improved by select random locations to damage the
filesystem, instead of hard-coding the seek=7 value.  Seek=7 is good
for testing ext2/ext3 filesystems, but it may not be ideal for other
filesystems.

Another interesting refinement would be to analyze the resulting
filesystem after it has been repaired to determine how much data could
be salvaged by the fsck program.   

There is a very interesting paper that I coincidentally just came
across today that talks about making filesystems robust against
various different forms of failures of modern disk systems.  It is
going to be presented at the upcoming 2005 SOSP conference.

	http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf

It's definitely worth a read.  A few comments about it; first of all,
I know nothing about this modified "iron ext3" (ixt3) discussed in the
paper aside from what's the paper itself.  It would be interesting to
see what they have done with it.  Secondly, I _think_ sct has already
fixed the problems discussed in the paper with respect to inadequate
write squelching after an I/O failure writing to the ext3 journal, but
we need to chat with the paper's authors to confirm that, and if there
still is a problem, obviously we need to fix it.  Third of all, I'll
note that the paper does takes an approving note of the fact that
Reiserfs (v3) always panic's when it detects a write fault, so for
those folks in the Reiser team who might have a persecution complex,
relax, the whole world isn't out to get you.  :-)

						- Ted

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Thanks Ted, I'll ask Vitaly to read the paper, and tell us what he
thinks should be learned from it for V4.

Hans

Theodore Ts'o wrote:

>On Tue, Sep 20, 2005 at 09:51:33AM +0200, Pavel Machek wrote:
>  
>
>>Do you have working fsck for V4? Until then, you should not claim that
>>users should switch. Journalling does not help you, if you have
>>unexpected kernel problem or hardware trouble, fsck _is_ mandatory.
>>
>>Can V4 survive few hours of test below?
>>    
>>
>
>The script could be improved by select random locations to damage the
>filesystem, instead of hard-coding the seek=7 value.  Seek=7 is good
>for testing ext2/ext3 filesystems, but it may not be ideal for other
>filesystems.
>
>Another interesting refinement would be to analyze the resulting
>filesystem after it has been repaired to determine how much data could
>be salvaged by the fsck program.   
>
>There is a very interesting paper that I coincidentally just came
>across today that talks about making filesystems robust against
>various different forms of failures of modern disk systems.  It is
>going to be presented at the upcoming 2005 SOSP conference.
>
>	http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
>
>It's definitely worth a read.  A few comments about it; first of all,
>I know nothing about this modified "iron ext3" (ixt3) discussed in the
>paper aside from what's the paper itself.  It would be interesting to
>see what they have done with it.  Secondly, I _think_ sct has already
>fixed the problems discussed in the paper with respect to inadequate
>write squelching after an I/O failure writing to the ext3 journal, but
>we need to chat with the paper's authors to confirm that, and if there
>still is a problem, obviously we need to fix it.  Third of all, I'll
>note that the paper does takes an approving note of the fact that
>Reiserfs (v3) always panic's when it detects a write fault, so for
>those folks in the Reiser team who might have a persecution complex,
>relax, the whole world isn't out to get you.  :-)
>
>						- Ted
>
>
>  
>

Re: I request inclusion of reiser4 in the mainline kernel

[Ric Wheeler]

Theodore Ts'o wrote:

> 
> Another interesting refinement would be to analyze the resulting
> filesystem after it has been repaired to determine how much data could
> be salvaged by the fsck program.   

We use reiserfs3 to store data and have had very good luck in getting 
data off of real world, hard disk failures.  In our case, we have a 
digital signature which is used to compare validate the content of the 
files after recovery so we have an extra comfort level with the results 
of a repaired file system...

Working on getting fsck to run reliably and quickly on large file 
systems (any type of file system) is certainly a big item on our wish 
list.  With relatively small file system (320GB) we can spend hours 
trying to recover.

> 
> There is a very interesting paper that I coincidentally just came
> across today that talks about making filesystems robust against
> various different forms of failures of modern disk systems.  It is
> going to be presented at the upcoming 2005 SOSP conference.
> 
> 	http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
> 
> It's definitely worth a read.  A few comments about it; first of all,
> I know nothing about this modified "iron ext3" (ixt3) discussed in the
> paper aside from what's the paper itself.  It would be interesting to
> see what they have done with it.  Secondly, I _think_ sct has already
> fixed the problems discussed in the paper with respect to inadequate
> write squelching after an I/O failure writing to the ext3 journal, but
> we need to chat with the paper's authors to confirm that, and if there
> still is a problem, obviously we need to fix it.  Third of all, I'll
> note that the paper does takes an approving note of the fact that
> Reiserfs (v3) always panic's when it detects a write fault, so for
> those folks in the Reiser team who might have a persecution complex,
> relax, the whole world isn't out to get you.  :-)
> 
> 						- Ted

We have been sponsors of this work at Wisconsin - a good group with 
interesting results.

As an earlier thread on lkml showed this summer, we still have a long 
way to go to getting consistent error semantics in face of media 
failures between the various file systems.  I am not sure that we even 
have consensus on what that default behavior should be between 
developers, so image how difficult life is for application writers who 
want to try to ride through or write automated "HA" recovery scripts for 
systems with large numbers of occasionally flaky IO devices ;-)

Ric

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Ric Wheeler wrote:

> As an earlier thread on lkml showed this summer, we still have a long
> way to go to getting consistent error semantics in face of media
> failures between the various file systems.  I am not sure that we even
> have consensus on what that default behavior should be between
> developers, so image how difficult life is for application writers who
> want to try to ride through or write automated "HA" recovery scripts
> for systems with large numbers of occasionally flaky IO devices ;-)

If you'd like to form a committee to standardize these things, I will
ask Vitaly to work with you on that committee, and to have ReiserFS3+4
conform to the standards that result.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Ric Wheeler]

Hans Reiser wrote:
> Ric Wheeler wrote:
> 
> 
>>As an earlier thread on lkml showed this summer, we still have a long
>>way to go to getting consistent error semantics in face of media
>>failures between the various file systems.  I am not sure that we even
>>have consensus on what that default behavior should be between
>>developers, so image how difficult life is for application writers who
>>want to try to ride through or write automated "HA" recovery scripts
>>for systems with large numbers of occasionally flaky IO devices ;-)
> 
> 
> If you'd like to form a committee to standardize these things, I will
> ask Vitaly to work with you on that committee, and to have ReiserFS3+4
> conform to the standards that result.
> 
> Hans

I am not a big fan of formal committees, but would be happy to take part 
in any effort to standardize, code and test the result...

ric

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Ric Wheeler wrote:

> Hans Reiser wrote:
>
>> Ric Wheeler wrote:
>>
>>
>>> As an earlier thread on lkml showed this summer, we still have a long
>>> way to go to getting consistent error semantics in face of media
>>> failures between the various file systems.  I am not sure that we even
>>> have consensus on what that default behavior should be between
>>> developers, so image how difficult life is for application writers who
>>> want to try to ride through or write automated "HA" recovery scripts
>>> for systems with large numbers of occasionally flaky IO devices ;-)
>>
>>
>>
>> If you'd like to form a committee to standardize these things, I will
>> ask Vitaly to work with you on that committee, and to have ReiserFS3+4
>> conform to the standards that result.
>>
>> Hans
>
>
> I am not a big fan of formal committees, but would be happy to take
> part in any effort to standardize, code and test the result...
>
> ric
>
>
>
The committee could simply exchange a set of emails, and agree on
things.  I doubt it needs to get all complicated.  I suggest you contact
all the folks you want to be consistent with each other, send us an
email asking us to all try to work together, and then ask for proposals
on what we should all conform to.  Distill the proposals, and then
suggest a common solution.  With luck, we will all just say yes.:)

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Gregory Maxwell]

On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
> > I am not a big fan of formal committees, but would be happy to take
> > part in any effort to standardize, code and test the result...
> The committee could simply exchange a set of emails, and agree on
> things.  I doubt it needs to get all complicated.  I suggest you contact
> all the folks you want to be consistent with each other, send us an
> email asking us to all try to work together, and then ask for proposals
> on what we should all conform to.  Distill the proposals, and then
> suggest a common solution.  With luck, we will all just say yes.:)

Another goal of the group should be to formulate a requested set of
changes or extensions to the makers of drives and other storage
systems.  For example, it might be advantageous to be able to disable
bad block relocation and allow the filesystem to perform the action. 
The reason for this is because relocates slaughter streaming read
performance, but the filesystem could still contiguously allocate
around them...

Perhaps a more implementable alternative is just a method to find out
which sectors have been relocated so the data can be moved off of them
and they be avoided. (and potentially they be 'derelocated' to
preserve the relocation space)

Ditto for other layers.. if a filesystem has some internal integrity
function and a raid sweep has found that the parity doesn't agree, it
would be nice if the FS could check all possible decodings and see if
there is one that is more sane than all the others... This is even
more useful when you have raid-6 and there is a lot more potential
decoding.

Also things like bubbling up to userspace.. If there is an
unrecoverable read error in a file found during operation or an
automated scan, it should show up in syslog with some working complete
path to the file (as canonical as the fs can provide), and hopefully
an offset to the error. Then my package manager could see if this is a
file replaceable out of a package or if it's user data... If it's user
data, my backup scripts can check the access time on the file and
silently restore it from backup if the file hasn't changed. ... only
leaving me an email "Dear operator, I saved your butt yet again
--love, mr computer"

And finally operator policy.. I'd like corrupted user files to become
permission denied until I run some command to make the accessible,
don't let me hang my apps trying to access them..

Re: I request inclusion of reiser4 in the mainline kernel

[Ric Wheeler]

Gregory Maxwell wrote:

>On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
>  
>
>
>Another goal of the group should be to formulate a requested set of
>changes or extensions to the makers of drives and other storage
>systems.  For example, it might be advantageous to be able to disable
>bad block relocation and allow the filesystem to perform the action. 
>The reason for this is because relocates slaughter streaming read
>performance, but the filesystem could still contiguously allocate
>around them...
>
>  
>
I think that would be a bad idea - that is how drives used to work and 
it made the higher level file system code handle odd stuff.

In the field, what we tend to see is that drives that accumulate more 
than a very small percentage of bad blocks (say 100 out of many 
millions) tend to be on their way out and need replacing.

>Perhaps a more implementable alternative is just a method to find out
>which sectors have been relocated so the data can be moved off of them
>and they be avoided. (and potentially they be 'derelocated' to
>preserve the relocation space)
>  
>
I think that this kind of information is already at hand via smart, 
etc.  You could write an application to query this data base, but to do 
the reverse mapping from block number to file is not easy (i.e., you 
need to fibmap each file in the file system in order to construct the 
mapping).

>Ditto for other layers.. if a filesystem has some internal integrity
>function and a raid sweep has found that the parity doesn't agree, it
>would be nice if the FS could check all possible decodings and see if
>there is one that is more sane than all the others... This is even
>more useful when you have raid-6 and there is a lot more potential
>decoding.
>  
>
One thing we do on our boxes is to run a sweep program that does a 
"read-verify" command that allows us to flag bad sectors on the platter 
without transferring data, polluting caches, etc.  A second, repair 
phase goes in and pokes at the suspect sectors trying to force a remap.  
If you have the original data (as in the raid case), you can rewrite the 
sector and all is well.  If not, you need to unmount, re-fsck and try to 
revalidate the contents of individual files (this is where the digital 
signatures comes in handy).

>Also things like bubbling up to userspace.. If there is an
>unrecoverable read error in a file found during operation or an
>automated scan, it should show up in syslog with some working complete
>path to the file (as canonical as the fs can provide), and hopefully
>an offset to the error. Then my package manager could see if this is a
>file replaceable out of a package or if it's user data... If it's user
>data, my backup scripts can check the access time on the file and
>silently restore it from backup if the file hasn't changed. ... only
>leaving me an email "Dear operator, I saved your butt yet again
>--love, mr computer"
>  
>
Good idea, but we don't have that reverse mapping at hand for most file 
systems.

>And finally operator policy.. I'd like corrupted user files to become
>permission denied until I run some command to make the accessible,
>don't let me hang my apps trying to access them..
>  
>

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Ric Wheeler wrote:

> Gregory Maxwell wrote:
>
>> On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
>>  
>>
>>
>> Another goal of the group should be to formulate a requested set of
>> changes or extensions to the makers of drives and other storage
>> systems.  For example, it might be advantageous to be able to disable
>> bad block relocation and allow the filesystem to perform the action.
>> The reason for this is because relocates slaughter streaming read
>> performance, but the filesystem could still contiguously allocate
>> around them...
>>
>>  
>>
The words were attributed to me, but were not mine.

Sometimes mua's do that to one.

Re: I request inclusion of reiser4 in the mainline kernel

[Ric Wheeler]

Hans Reiser wrote:
> Ric Wheeler wrote:
> 
> 
>>Gregory Maxwell wrote:
>>
>>
>>>On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
>>> 
>>>
>>>
>>>Another goal of the group should be to formulate a requested set of
>>>changes or extensions to the makers of drives and other storage
>>>systems.  For example, it might be advantageous to be able to disable
>>>bad block relocation and allow the filesystem to perform the action.
>>>The reason for this is because relocates slaughter streaming read
>>>performance, but the filesystem could still contiguously allocate
>>>around them...
>>>
>>> 
>>>
> 
> The words were attributed to me, but were not mine.
> 
> Sometimes mua's do that to one.
> 

Sorry - my error in trying to trim my response ended up trimming the 
proper attribution to Gregory,

ric

Re: I request inclusion of reiser4 in the mainline kernel

[Vitaly Fertman]

On Wednesday 21 September 2005 07:05, Hans Reiser wrote:
> Ric Wheeler wrote:
> 
> > Hans Reiser wrote:
> >
> >> Ric Wheeler wrote:
> >>
> >>
> >>> As an earlier thread on lkml showed this summer, we still have a long
> >>> way to go to getting consistent error semantics in face of media
> >>> failures between the various file systems.  I am not sure that we even
> >>> have consensus on what that default behavior should be between
> >>> developers, so image how difficult life is for application writers who
> >>> want to try to ride through or write automated "HA" recovery scripts
> >>> for systems with large numbers of occasionally flaky IO devices ;-)
> >>
> >>
> >>
> >> If you'd like to form a committee to standardize these things, I will
> >> ask Vitaly to work with you on that committee, and to have ReiserFS3+4
> >> conform to the standards that result.
> >>
> >> Hans
> >
> >
> > I am not a big fan of formal committees, but would be happy to take
> > part in any effort to standardize, code and test the result...

I would be happy to take part in it too.

> > ric
> >
> >
> >
> The committee could simply exchange a set of emails, and agree on
> things.  I doubt it needs to get all complicated.  I suggest you contact
> all the folks you want to be consistent with each other, send us an
> email asking us to all try to work together, and then ask for proposals
> on what we should all conform to.  Distill the proposals, and then
> suggest a common solution.  With luck, we will all just say yes.:)
 
-- 
Thanks,
Vitaly Fertman

Re: I request inclusion of reiser4 in the mainline kernel

[Gregory Maxwell]

On 9/20/05, Theodore Ts'o <tytso@mit.edu> wrote:
> The script could be improved by select random locations to damage the
> filesystem, instead of hard-coding the seek=7 value.  Seek=7 is good
> for testing ext2/ext3 filesystems, but it may not be ideal for other
> filesystems.

What would be interesting would be to overwrite random blocks in an
sub-exponentially increasing fashion, fsck and measure file loss at
every step. You fail the test if the system panics reading a FS that
passed a fsck. It would be interesting to chart files lost and files
silently corrupted over time...

Another interesting thought would be to snapshot a file system over
and over again while it's got a disk workout suite running on it..
Then fsck the snapshots, and check for the amount of data loss and
corruption.

> There is a very interesting paper that I coincidentally just came
> across today that talks about making filesystems robust against
> various different forms of failures of modern disk systems.  It is
> going to be presented at the upcoming 2005 SOSP conference.
> 
>         http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf

Very interesting indeed, although it almost seems silly to tackle the
difficult problem of making filesystems highly robust against oddball
failure modes while our RAID subsystem falls horribly on it's face in
the fairly common (and conceptually easy to handle) failure mode of a
raid-5 where two disks have single unreadable blocks on differing
parts of the disk. (the current raid system hits one bad block, fails
the whole disk, then you attempt a rebuild and while reading hits the
other bad block and downs the array).

Re: I request inclusion of reiser4 in the mainline kernel

[Ric Wheeler]

Gregory Maxwell wrote:
> On 9/20/05, Theodore Ts'o <tytso@mit.edu> wrote:
> 
>>There is a very interesting paper that I coincidentally just came
>>across today that talks about making filesystems robust against
>>various different forms of failures of modern disk systems.  It is
>>going to be presented at the upcoming 2005 SOSP conference.
>>
>>        http://www.cs.wisc.edu/adsl/Publications/iron-sosp05.pdf
> 
> 
> Very interesting indeed, although it almost seems silly to tackle the
> difficult problem of making filesystems highly robust against oddball
> failure modes while our RAID subsystem falls horribly on it's face in
> the fairly common (and conceptually easy to handle) failure mode of a
> raid-5 where two disks have single unreadable blocks on differing
> parts of the disk. (the current raid system hits one bad block, fails
> the whole disk, then you attempt a rebuild and while reading hits the
> other bad block and downs the array).

I don't have any problem with fixing RAID code, but I would not describe 
the errors in this paper as oddball.  Believe me, we see a lot of disk 
issues and these are real examples of failures that happen to real file 
systems ;-)

Ric

Re: I request inclusion of reiser4 in the mainline kernel

[David Greaves]

Gregory Maxwell wrote:

>Very interesting indeed, although it almost seems silly to tackle the
>difficult problem of making filesystems highly robust against oddball
>failure modes while our RAID subsystem falls horribly on it's face in
>the fairly common (and conceptually easy to handle) failure mode of a
>raid-5 where two disks have single unreadable blocks on differing
>parts of the disk. (the current raid system hits one bad block, fails
>the whole disk, then you attempt a rebuild and while reading hits the
>other bad block and downs the array).
>  
>
who's not keeping up with the linux-raid list then ;)

David
PS I'm sure assistance would be appreciated in testing and reviewing
this few day old feature - or indeed the newer 'add a new disk to the
array' feature.

Re: I request inclusion of reiser4 in the mainline kernel

[Gregory Maxwell]

On 9/23/05, David Greaves <david@dgreaves.com> wrote:
> who's not keeping up with the linux-raid list then ;)
>
> David
> PS I'm sure assistance would be appreciated in testing and reviewing
> this few day old feature - or indeed the newer 'add a new disk to the
> array' feature.

After posting that I checked linux-raid.... Thanked the author,
patched a box, but got called out of town before I could test
anything. :)

This is an important development. ... and it's about darn time!

Re: I request inclusion of reiser4 in the mainline kernel

[Kyle Moffett]

On Sep 18, 2005, at 13:22:27, michael chang wrote:
> On 9/18/05, Christoph Hellwig <hch@infradead.org> wrote:
>
>> On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
>>
>>> This is it. I do not say "accept reiser4 NOW", I am saying "give  
>>> Hans good code review".
>>
>> that and there's much more exciting filesystems like ocfs2 around  
>> that
>
> This is exciting to... whom?

To the people that review the code.  We're all volunteers here; if  
your filesystem is so ugly and hard to read that the code reviewers  
don't feel like finding time to slog through the mess, then it  
probably means that you need to clean the code, document it better,  
make it simpler to understand, fix the coding-style, etc.

> The only thing that appears remotely interesting about it is that  
> it's made by Oracle and apparently is supposed to be geared toward  
> parallel server whatsits.  This might be helpful to corporations,  
> but seems senseless toward many consumers.  (I'm assuming there's  
> still at least one consumer left who still uses Linux.)

Like I said above, we're all volunteers.  Personally, I find OCFS2  
_much_ more interesting than reiser4, because it has a lot of cool  
networked lock-managing algorithms that (given my current limited  
understanding), work black magic.  Given this, I'm a lot more likely  
to spend time reading the OCFS2 code because its interesting than I  
am reading reiser4 code, even though somebody eventually probably  
needs to do said review.  Hans' personal attacks on the people who  
have criticized his code make such tasks even less personally  
gratifying (IE: less interesting).  I think some people are likely  
hoping right now that if they put off the reiser4 code review long  
enough, maybe the authors will take a hint and have it a bit cleaner  
by the time they finally do get around to the review.

> Give Hans a chance; and please try to understand, even if he's hard  
> to work with.  Discriminate him because he's not a developer you  
> can talk with, and I believe that's like discriminating a guy in a  
> wheelchair because he can't run with you when you jog in the morning.

When you start getting into obscure discrimination analogies, the  
discussion has become _way_ nontechnical.  If this goes this goes any  
further, somebody's probably going to compare a kernel developer to a  
Nazi or Hitler, invoking Godwin's law and effectively killing the  
thread.  Please get this back onto a technical bent or drop it.

> Not everyone has the same "common sense" that you do.  Explain,  
> fully, with reasoning, and reproducable back-up statistics on  
> common hardware, what code is wrong, and what must be written  
> instead.  We'd like to be efficient, and it's not being efficient  
> to play a guessing
> game with us.  If you don't have the time to review, then please  
> hold  off on replying until you have a through review of at least  
> part of the code.

Christoph has noted a number of things in previous emails.  I just  
looked through the latest released code and several of them are still  
valid.  I would look through the latest code to see what is still  
missing, but I can't get it on account of it being in bitkeeper,  
which I don't have installed and don't intend to install.

> I'm willing to go compare...  [massive nontechnical rhetoric snipped]

Unless you have technical arguments to contribute (and you indicate  
that you do not), please to not spam the LKML with useless rhetoric- 
filled emails that most of us will delete because we have too many  
other things to do to bother reading or responding to.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L++++(+ 
++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+++ t+(+ 
++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  !y?(-)
------END GEEK CODE BLOCK------

Re: I request inclusion of reiser4 in the mainline kernel

[michael chang]

On 9/18/05, Kyle Moffett <mrmacman_g4@mac.com> wrote:
> On Sep 18, 2005, at 13:22:27, michael chang wrote:
> > On 9/18/05, Christoph Hellwig <hch@infradead.org> wrote:
> >
> >> On Sun, Sep 18, 2005 at 01:21:23PM +0300, Denis Vlasenko wrote:
> >>
> >>> This is it. I do not say "accept reiser4 NOW", I am saying "give
> >>> Hans good code review".
> >>
> >> that and there's much more exciting filesystems like ocfs2 around
> >> that
> >
> > This is exciting to... whom?
> 
> To the people that review the code.  We're all volunteers here; if
> your filesystem is so ugly and hard to read that the code reviewers
> don't feel like finding time to slog through the mess, then it
> probably means that you need to clean the code, document it better,
> make it simpler to understand, fix the coding-style, etc.
> 
> > The only thing that appears remotely interesting about it is that
> > it's made by Oracle and apparently is supposed to be geared toward
> > parallel server whatsits.  This might be helpful to corporations,
> > but seems senseless toward many consumers.  (I'm assuming there's
> > still at least one consumer left who still uses Linux.)
> 
> Like I said above, we're all volunteers.  Personally, I find OCFS2
> _much_ more interesting than reiser4, because it has a lot of cool
> networked lock-managing algorithms that (given my current limited
> understanding), work black magic.  Given this, I'm a lot more likely
> to spend time reading the OCFS2 code because its interesting than I
> am reading reiser4 code, even though somebody eventually probably
> needs to do said review.  Hans' personal attacks on the people who
> have criticized his code make such tasks even less personally
> gratifying (IE: less interesting).  I think some people are likely
> hoping right now that if they put off the reiser4 code review long
> enough, maybe the authors will take a hint and have it a bit cleaner
> by the time they finally do get around to the review.
> 
> > Give Hans a chance; and please try to understand, even if he's hard
> > to work with.  Discriminate him because he's not a developer you
> > can talk with, and I believe that's like discriminating a guy in a
> > wheelchair because he can't run with you when you jog in the morning.
> 
> When you start getting into obscure discrimination analogies, the
> discussion has become _way_ nontechnical.  If this goes this goes any
> further, somebody's probably going to compare a kernel developer to a
> Nazi or Hitler, invoking Godwin's law and effectively killing the
> thread.  Please get this back onto a technical bent or drop it.
> 
> > Not everyone has the same "common sense" that you do.  Explain,
> > fully, with reasoning, and reproducable back-up statistics on
> > common hardware, what code is wrong, and what must be written
> > instead.  We'd like to be efficient, and it's not being efficient
> > to play a guessing
> > game with us.  If you don't have the time to review, then please
> > hold  off on replying until you have a through review of at least
> > part of the code.
> 
> Christoph has noted a number of things in previous emails.  I just
> looked through the latest released code and several of them are still
> valid.  I would look through the latest code to see what is still
> missing, but I can't get it on account of it being in bitkeeper,
> which I don't have installed and don't intend to install.
> 
> > I'm willing to go compare...  [massive nontechnical rhetoric snipped]
> 
> Unless you have technical arguments to contribute (and you indicate
> that you do not), please to not spam the LKML with useless rhetoric-
> filled emails that most of us will delete because we have too many
> other things to do to bother reading or responding to.
> 

Alright, I concede.

Personally, I'm not much of a techie compared to you guys; I'm only in
High School, and I have a mental disorder
(http://en.wikipedia.org/wiki/Asperger's_Syndrome), so I'll stop here,
and hope that you guys can resolve this yourselves.  Good luck to all,
and hopefully there will be a peaceful resolution here.

-- 
~Mike
 - Just my two cents
 - No man is an island, and no man is unable.

Re: I request inclusion of reiser4 in the mainline kernel

[Alan Cox]

On Sul, 2005-09-18 at 13:22 -0400, michael chang wrote:
> This is exciting to... whom?  The only thing that appears remotely
> interesting about it is that it's made by Oracle and apparently is
> supposed to be geared toward parallel server whatsits.

Which no current included fs supports. And parallel file systems btw get
exciting for everyone once you have virtualisation.

> Is that Hans' fault, or the fault of your lot?  Why can't we all just get along?

Insufficient drugs ;) ?

> work with.  Discriminate him because he's not a developer you can talk
> with, and I believe that's like discriminating a guy in a wheelchair
> because he can't run with you when you jog in the morning.

Hans can learn to work with people, most folks in wheelchairs cannot
take lessons and walk. Many of them have tried months of physiotherapy.
to learn to walk again. I think your comparison is insulting to a lot of
the disabled.

> Also, let's say that Reiser4 doesn't get into the kernel, as maybe XFS
> or ext2 or ext3 had never gotten into the kernel.  How would their

Linus refused ext3 initially. It went in because it had a userbase,
vendors shipping it and reliable clean code. Saying "no" a lot is really
rather important to keeping the kernel maintainable. I regularly meet
cases we should have said "no" a lot louder 8)

> I'm willing to go compare Reiser4 to ext2/3 as like H.264 to Mpeg-2. 
> Indeed, H.264 crashes some computers, similar to Reiser4 might crash
> some machines, but this is merely because Reiser4 explores new

It doesn't matter if reiser4 causes crashes. It matters that people can
fix them, that they are actively fixed and the code is maintainable. It
will have bugs, all complex code has bugs. Hans team have demonstrated
the ability to fix some of those bugs fast, but we also all remember
what happened with reiser3 later on despite early fast fixing.

One big reason we jump up and down so much about the coding style is
that its the one thing that ensures someone else can maintain and fix
code that the author has abandoned, doesn't have time to fix or that
needs access to specific hardware the authors may not have.

Alan

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Alan Cox wrote:

>
>It doesn't matter if reiser4 causes crashes. It matters that people can
>fix them, that they are actively fixed and the code is maintainable. It
>will have bugs, all complex code has bugs. Hans team have demonstrated
>the ability to fix some of those bugs fast, but we also all remember
>what happened with reiser3 later on despite early fast fixing.
>  
>
What was that?

>One big reason we jump up and down so much about the coding style is
>that its the one thing that ensures someone else can maintain and fix
>code that the author has abandoned, doesn't have time to fix or that
>needs access to specific hardware the authors may not have.
>
So why is the code in the kernel so hard to read then?

Linux kernel code is getting better, and Andrew Morton's code is
especially good, but for the most part it's unnecessarily hard to read. 
Look at the elevator code for instance.  Ugh.

I differ in one major aspect from some.  That is, the only coding style
requirement I have of those who work for me is that it must be easy to
read.  That is because at every company I can remember where someone was
gungho about advocating that code be written in a specific defined
style, that someone was always the one with the least readable code.

I have a simple punishment for those who violate my requirement: I go
through the code line by line with them, which they always hate for some
reason, and help them comment and clarify it.  1-2 sessions of this, and
they usually change how they code so that they don't have to go through
it again with me.

Asking for readable code is not that different from asking for readable
novels: if you try to define what is required rather than teaching
instance by instance, you can only get in the way of the artist rather
than instructing.

That is why I just say "make it easy to read and I don't care how you do
that so long as it works."

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

Thanks a lot for the roundless attacks, but I'm done.

I'll stop helping you to review stuff because it's just totally pointless,
you ignore most of the review anyway and start personal attacks.

Please find someone else to review your code for inclusion, that can stand
beeing attacked everytime they write an email.  Before you should probably
fix up various bits I wrote up and especialy make sure to get rid of
all duplication of generic I/O code or explain in detail why you need it
and fix your own implementations.

And next time you request review request and inclusion please use nicer
words than 'I request inclusion'.  There's no right to get code included
in the kernel tree, it's a honor.

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Mon, Sep 19, 2005 at 10:01:45AM +0100, Christoph Hellwig wrote:
> Thanks a lot for the roundless attacks, but I'm done.

groundless..

Re: I request inclusion of reiser4 in the mainline kernel

[Andrew Morton]

Christoph Hellwig <hch@infradead.org> wrote:
>
> Before you should probably
>  fix up various bits I wrote up and especialy make sure to get rid of
>  all duplication of generic I/O code or explain in detail why you need it
>  and fix your own implementations.

Yup, thanks.  I've made a record of your comments in the changelog for
-mm's reiser4-only.patch.

Re: I request inclusion of reiser4 in the mainline kernel

[Alan Cox]

On Sul, 2005-09-18 at 22:07 -0700, Hans Reiser wrote:
> >the ability to fix some of those bugs fast, but we also all remember
> >what happened with reiser3 later on despite early fast fixing.
> >  
> >
> What was that?

Jeff Mahoney added file attributes to reiserfs3, you whined and pointed
people at the yet to be released reiserfs4. Someone fixed the 4K stack
on reiserfs3, you whined. Chris Mason added other fixes like
data=journal support to get some kind of journal feature parity with
ext3, you complained and ask it not to be added.

> That is why I just say "make it easy to read and I don't care how you do
> that so long as it works."

Perhaps you do. The kernel follows a coding style. It isn't my coding
style but like everyone else except you I try and follow it.

Alan

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Alan Cox wrote:

>On Sul, 2005-09-18 at 22:07 -0700, Hans Reiser wrote:
>  
>
>>>the ability to fix some of those bugs fast, but we also all remember
>>>what happened with reiser3 later on despite early fast fixing.
>>> 
>>>
>>>      
>>>
>>What was that?
>>    
>>
>
>Jeff Mahoney added file attributes to reiserfs3, you whined and pointed
>people at the yet to be released reiserfs4.
>
If you benchmarked that code, you might understand why I "whined."  You
can't just create a file per directory and stuff the attributes in it
and expect good performance.  Let's not forget that there was no
documentation, no design document, no design review, no QA process.

It is always a judgment call to decide what should be deferred to the
next major release and what should go into a stable branch.  File
attributes are a significant portion of the bugs that V3 has had.  File
attributes got added so that a marketer would have a bullet point added,
which can be very important and I am genuinely eager to work hard to
make marketers happy, but to the extent I get to decide, it will never
happen at the cost of coding it the wrong way.

Jeff is a great guy, and his bitmap related code is great stuff with
good design and solid empirical work behind it.  You have to really
understand the difference between V3 and V4 to appreciate that it was
not feasible for him to code xattrs for V3 the right way, because it
would be a disk format change and a nightmare to do it.  The code was
doomed by V3's lack of plugins before it was even written.  There is a
reason why V4 came into being....

If added to V4, xattrs would be higher performance and cleaner to
implement.  It would be far better to have spent the programming effort
on adding them to V4 and getting V4 out a little sooner.

I won't convince you of this one but it is also my reason: They are
inelegant semantics.

I don't remember the details of the 4k stack and journaling issues you
describe, so I will say nothing.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Alan Cox wrote:

>
>Perhaps you do. The kernel follows a coding style. It isn't my coding
>style but like everyone else except you I try and follow it.
>  
>
I also don't care enough about coding style issues to resist them.;-) 
We have conformed to the coding style issues that were pointed out, and
as more are pointed out we will conform to them.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Jens Axboe]

On Sun, Sep 18 2005, Hans Reiser wrote:
> Linux kernel code is getting better, and Andrew Morton's code is
> especially good, but for the most part it's unnecessarily hard to read. 
> Look at the elevator code for instance.  Ugh.

That's pretty bold, coming from someone who cannot even figure out how
to follow the style guidelines of the kernel.

The elevator core api is so trivial, I don't believe you find this hard
to read. The individual io schedulers tend to have comments in functions
that aren't immediately obvious. So I'm curious, what do part makes you
go 'ugh'?

So please explain or refrain from making stupid unwarranted comments
about other peoples code.

> That is why I just say "make it easy to read and I don't care how you do
> that so long as it works."

Then why do you insist on commenting your code differently than everyone
else ('everyone' means the kernel here, since that is your target)?

-- 
Jens Axboe

Re: I request inclusion of reiser4 in the mainline kernel

[Nick Piggin]

Hans Reiser wrote:

>So why is the code in the kernel so hard to read then?
>
>Linux kernel code is getting better, and Andrew Morton's code is
>especially good, but for the most part it's unnecessarily hard to read. 
>Look at the elevator code for instance.  Ugh.
>
>

What's wrong with the elevator code?

The elevator code was one of the first things I got involved with
as a complete kernel newbie, and I was able to follow the current
code well enough to make a new IO scheduler, and extend the
elevator API sufficiently to provide the fairly unique
capabilities I needed.

If it is the elevator *API* you are worried about, that is fairly
trivial and well documented by Jens and myself in
Documentation/block/biodoc.txt, along with an overview of some key
ideas useful for iosched implementors.

as-iosched.c itself is IMO reasonably well commented (at least
the non-trivial, non-boilerplate functions). That is not to say it
is trivial to understand because it is a fairly complex state
machine and heuristics, but at less than 2000 lines of very well
contained code it is not an impossible task to understand it.

If that is too much for you, noop-iosched.c implements a fully
working io scheduler in exactly 94 lines, including whitespace and
comments.

What are your specific concerns? I would be interested in helping
to fix any valid ones you have.

Thanks,
Nick


Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nick Piggin wrote:

> Hans Reiser wrote:
>
>> So why is the code in the kernel so hard to read then?
>>
>> Linux kernel code is getting better, and Andrew Morton's code is
>> especially good, but for the most part it's unnecessarily hard to
>> read. Look at the elevator code for instance.  Ugh.
>>
>>
>
> What's wrong with the elevator code?
>
The name for one.  There is no elevator algorithm anywhere in it.  There
is a least block number first algorithm that was called an elevator, but
it does not have the properties described by Ousterhout and sundry CS
textbooks describing elevator algorithms.  The textbook algorithms are
better than least block number first, and it is interesting how nobody
fixed the mislabeling of the algorithm once Linux had gotten to the
point that it was striving for more than making gcc be able to run on it. 

cfq is good code though for many usage patterns. 

I would say more, but I need to talk a customer into ok'ing releasing
some code first, so I can only say what I knew before doing the work for
that customer at this time.

If you would like many more details of coding/commenting inelegance, ask
Nate Diller after the customer oks his talking about it, which will
happen more easily if we say nothing that we did not know before the
work for them until we first get their ok.....

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Nick Piggin]

On Mon, 2005-09-19 at 23:28 -0700, Hans Reiser wrote:
> Nick Piggin wrote:

> > What's wrong with the elevator code?
> >
> The name for one.  There is no elevator algorithm anywhere in it.  There
> is a least block number first algorithm that was called an elevator, but

Well the terminology changed to "io scheduler" now, however the
residual "elevator" name found in places doesn't cause anyone
any problems and there isn't much reason to change it other than
the desire to break things.

> it does not have the properties described by Ousterhout and sundry CS
> textbooks describing elevator algorithms.  The textbook algorithms are
> better than least block number first, and it is interesting how nobody
> fixed the mislabeling of the algorithm once Linux had gotten to the
> point that it was striving for more than making gcc be able to run on it. 
> 

There is no least block number first io scheduler now. And the
deadline scheduler is basically an elevator algorithm with
deadlines.

> cfq is good code though for many usage patterns. 
> 

But that is not a true elevator algorithm either... so what are
you trying to say?

> I would say more, but I need to talk a customer into ok'ing releasing
> some code first, so I can only say what I knew before doing the work for
> that customer at this time.
> 
> If you would like many more details of coding/commenting inelegance, ask
> Nate Diller after the customer oks his talking about it, which will
> happen more easily if we say nothing that we did not know before the
> work for them until we first get their ok.....
> 

I happen to think that the "elevator code" is quite nice, the
block layer side, the interface itself, and the io schedulers
too. So I wouldn't fix up anything unless someone came to me
with an issue :)

Anyway, let's kill this subthread. I just don't think you proved
much of a point by picking a random kernel subsystem to point to,
whether such criticism was justified or not.

But if you really need to , I instead suggest badmouthing devfs.
That is sure to get you on the good side of the VFS guys! :)

Nick

-- 
SUSE Labs, Novell Inc.



Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nick Piggin wrote:

>On Mon, 2005-09-19 at 23:28 -0700, Hans Reiser wrote:
>  
>
>>Nick Piggin wrote:
>>    
>>
>
>  
>
>>>What's wrong with the elevator code?
>>>
>>>      
>>>
>>The name for one.  There is no elevator algorithm anywhere in it.  There
>>is a least block number first algorithm that was called an elevator, but
>>    
>>
>
>Well the terminology changed to "io scheduler" now, however the
>residual "elevator" name found in places doesn't cause anyone
>any problems and there isn't much reason to change it other than
>the desire to break things.
>  
>
Did you really say that?    I mean, come on, can't you at least manage a
"well, it ought to get changed but I am busy with something more
exciting to me".

>  
>
>>it does not have the properties described by Ousterhout and sundry CS
>>textbooks describing elevator algorithms.  The textbook algorithms are
>>better than least block number first, and it is interesting how nobody
>>fixed the mislabeling of the algorithm once Linux had gotten to the
>>point that it was striving for more than making gcc be able to run on it. 
>>
>>    
>>
>
>There is no least block number first io scheduler now. And the
>deadline scheduler is basically an elevator algorithm with
>deadlines.
>  
>
Ask Nate about this after he gets an ok from the customer to disclose
his work.  It is not so simple as you claim.

>  
>
>>cfq is good code though for many usage patterns. 
>>
>>    
>>
>
>But that is not a true elevator algorithm either... so what are
>you trying to say?
>  
>
I am trying to be balanced.  2.6 was a dramatic improvement over 2.4,
and cfq seems to work well.

>But if you really need to , I instead suggest badmouthing devfs.
>That is sure to get you on the good side of the VFS guys! :)
>  
>
Devfs was a good idea in its essence.  http://kerneltrap.org/node/5665
suggests pretty clearly that the hostility of the VFS guys caused no one
to want to invest enough into devfs to make it viable compared to udev.

They were inappropriately nasty to Mr. Gooch, who was kind enough to
contribute an idea that Linux needed.  They could have been helpful and
assisting, and instead they were the opposite.  As they are with everyone.

Re: I request inclusion of reiser4 in the mainline kernel

[Jens Axboe]

On Tue, Sep 20 2005, Hans Reiser wrote:
> >>The name for one.  There is no elevator algorithm anywhere in it.  There
> >>is a least block number first algorithm that was called an elevator, but
> >>    
> >>
> >
> >Well the terminology changed to "io scheduler" now, however the
> >residual "elevator" name found in places doesn't cause anyone
> >any problems and there isn't much reason to change it other than
> >the desire to break things.
> >  
> >
> Did you really say that?    I mean, come on, can't you at least manage a
> "well, it ought to get changed but I am busy with something more
> exciting to me".

Seeing as you are the one that is apparently bothered by the misnomer,
it follows that you would be the one submitting a patch for this. Not
that it would be accepted though, I don't see much point in renaming
functions and breaking drivers just because of a slightly bad name. The
io schedulers are all called foo-iosched.c, it's only the simple core
api that uses the 'elevator' description.

-- 
Jens Axboe

Re: I request inclusion of reiser4 in the mainline kernel

[Lorenzo Allegrucci]

On Tuesday 20 September 2005 13:42, Jens Axboe wrote:
> On Tue, Sep 20 2005, Hans Reiser wrote:
> > >>The name for one.  There is no elevator algorithm anywhere in it.  There
> > >>is a least block number first algorithm that was called an elevator, but
> > >>    
> > >>
> > >
> > >Well the terminology changed to "io scheduler" now, however the
> > >residual "elevator" name found in places doesn't cause anyone
> > >any problems and there isn't much reason to change it other than
> > >the desire to break things.
> > >  
> > >
> > Did you really say that?    I mean, come on, can't you at least manage a
> > "well, it ought to get changed but I am busy with something more
> > exciting to me".
> 
> Seeing as you are the one that is apparently bothered by the misnomer,
> it follows that you would be the one submitting a patch for this. Not
> that it would be accepted though, I don't see much point in renaming
> functions and breaking drivers just because of a slightly bad name. The
> io schedulers are all called foo-iosched.c, it's only the simple core
> api that uses the 'elevator' description.

Why not just rename the kernel option "elevator" to "iosched" ?

--- elevator.c  2005-09-20 15:26:19.000000000 +0200
+++ elevator.c.iosched  2005-09-20 15:27:11.000000000 +0200
@@ -178,7 +178,7 @@
        return 0;
 }

-__setup("elevator=", elevator_setup);
+__setup("iosched=", elevator_setup);

 int elevator_init(request_queue_t *q, char *name)
 {

Re: I request inclusion of reiser4 in the mainline kernel

[Jens Axboe]

On Tue, Sep 20 2005, Lorenzo Allegrucci wrote:
> On Tuesday 20 September 2005 13:42, Jens Axboe wrote:
> > On Tue, Sep 20 2005, Hans Reiser wrote:
> > > >>The name for one.  There is no elevator algorithm anywhere in it.  There
> > > >>is a least block number first algorithm that was called an elevator, but
> > > >>    
> > > >>
> > > >
> > > >Well the terminology changed to "io scheduler" now, however the
> > > >residual "elevator" name found in places doesn't cause anyone
> > > >any problems and there isn't much reason to change it other than
> > > >the desire to break things.
> > > >  
> > > >
> > > Did you really say that?    I mean, come on, can't you at least manage a
> > > "well, it ought to get changed but I am busy with something more
> > > exciting to me".
> > 
> > Seeing as you are the one that is apparently bothered by the misnomer,
> > it follows that you would be the one submitting a patch for this. Not
> > that it would be accepted though, I don't see much point in renaming
> > functions and breaking drivers just because of a slightly bad name. The
> > io schedulers are all called foo-iosched.c, it's only the simple core
> > api that uses the 'elevator' description.
> 
> Why not just rename the kernel option "elevator" to "iosched" ?
> 
> --- elevator.c  2005-09-20 15:26:19.000000000 +0200
> +++ elevator.c.iosched  2005-09-20 15:27:11.000000000 +0200
> @@ -178,7 +178,7 @@
>         return 0;
>  }
> 
> -__setup("elevator=", elevator_setup);
> +__setup("iosched=", elevator_setup);
> 
>  int elevator_init(request_queue_t *q, char *name)
>  {

Because I know at least SUSE uses this name for setting a different io
scheduler on boot. And there are users out there that have added the
options to their boot loader config.

So let me repeat - we are not going to break any existing setups for no
good reason. End of discussion.

-- 
Jens Axboe

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Lorenzo Allegrucci <l.allegrucci@gmail.com> writes:

[...]

>
> Why not just rename the kernel option "elevator" to "iosched" ?

At least update Documentation/kernel-parameters.txt to be consistent,
but I think kernel boot options are considered to be a part of the "user
space API" and, as such, cannot be changed that easily.

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nikita Danilov wrote:

>Lorenzo Allegrucci <l.allegrucci@gmail.com> writes:
>
>[...]
>
>  
>
>>Why not just rename the kernel option "elevator" to "iosched" ?
>>    
>>
>
>At least update Documentation/kernel-parameters.txt to be consistent,
>but I think kernel boot options are considered to be a part of the "user
>space API" and, as such, cannot be changed that easily.
>
>Nikita.
>
>
>  
>
You could make both names work, and then deprecate "elevator".....

Re: I request inclusion of reiser4 in the mainline kernel

[Randy.Dunlap]

On Tue, 20 Sep 2005, Lorenzo Allegrucci wrote:

> On Tuesday 20 September 2005 13:42, Jens Axboe wrote:
> > On Tue, Sep 20 2005, Hans Reiser wrote:
> > > >>The name for one.  There is no elevator algorithm anywhere in it.  There
> > > >>is a least block number first algorithm that was called an elevator, but
> > > >>
> > > >>
> > > >
> > > >Well the terminology changed to "io scheduler" now, however the
> > > >residual "elevator" name found in places doesn't cause anyone
> > > >any problems and there isn't much reason to change it other than
> > > >the desire to break things.
> > > >
> > > >
> > > Did you really say that?    I mean, come on, can't you at least manage a
> > > "well, it ought to get changed but I am busy with something more
> > > exciting to me".
> >
> > Seeing as you are the one that is apparently bothered by the misnomer,
> > it follows that you would be the one submitting a patch for this. Not
> > that it would be accepted though, I don't see much point in renaming
> > functions and breaking drivers just because of a slightly bad name. The
> > io schedulers are all called foo-iosched.c, it's only the simple core
> > api that uses the 'elevator' description.
>
> Why not just rename the kernel option "elevator" to "iosched" ?
>
> --- elevator.c  2005-09-20 15:26:19.000000000 +0200
> +++ elevator.c.iosched  2005-09-20 15:27:11.000000000 +0200
> @@ -178,7 +178,7 @@
>         return 0;
>  }
>
> -__setup("elevator=", elevator_setup);
> +__setup("iosched=", elevator_setup);
>
>  int elevator_init(request_queue_t *q, char *name)
>  {
> -

if you are serious, also update
Documentation/kernel-parameters.txt (s/elevator/iosched/)
[which would make some sense]

-- 
~Randy

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Jens Axboe wrote:

>
>Seeing as you are the one that is apparently bothered by the misnomer,
>it follows that you would be the one submitting a patch for this. Not
>that it would be accepted though, I don't see much point in renaming
>functions and breaking drivers just because of a slightly bad name. The
>io schedulers are all called foo-iosched.c, it's only the simple core
>api that uses the 'elevator' description.
>
>  
>
He asked for an example of messy code, I gave one.  Nate can give
details on other messiness in that code.

Reiser4 has flaws also....

Give all the code time, and it will improve.  The "elevator" code has
gotten a LOT better.

Re: I request inclusion of reiser4 in the mainline kernel

[Jens Axboe]

On Tue, Sep 20 2005, Hans Reiser wrote:
> Jens Axboe wrote:
> 
> >
> >Seeing as you are the one that is apparently bothered by the misnomer,
> >it follows that you would be the one submitting a patch for this. Not
> >that it would be accepted though, I don't see much point in renaming
> >functions and breaking drivers just because of a slightly bad name. The
> >io schedulers are all called foo-iosched.c, it's only the simple core
> >api that uses the 'elevator' description.
> >
> >  
> >
> He asked for an example of messy code, I gave one.  Nate can give
> details on other messiness in that code.

You never gave any details on why the code is "messy" or made you go
"ugh", so no you gave no such examples. Which is a pretty odd position
to put yourself in to be honest, anyone can make silly unsubstantiated
claims.

> Reiser4 has flaws also....

All code has flaws, nothing is perfect. It's not the point. Forgive me
for being a little offended that you call my code messy for no obvious
reason.

-- 
Jens Axboe

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

Hans Reiser <reiser@namesys.com> wrote:
> Nick Piggin wrote:
> > Hans Reiser wrote:
> >> So why is the code in the kernel so hard to read then?
> >>
> >> Linux kernel code is getting better, and Andrew Morton's code is
> >> especially good, but for the most part it's unnecessarily hard to
> >> read. Look at the elevator code for instance.  Ugh.

> > What's wrong with the elevator code?

> The name for one.  There is no elevator algorithm anywhere in it.

IO schedulers are commonly called "elevators", even though most of them
aren't. Standard operating system terminology.

>                                                                    There
> is a least block number first algorithm that was called an elevator, but
> it does not have the properties described by Ousterhout and sundry CS
> textbooks describing elevator algorithms.  The textbook algorithms are
> better than least block number first,

Funny that the "texbook algorithms" aren't used in real life. Wonder why...

>                                       and it is interesting how nobody
> fixed the mislabeling of the algorithm once Linux had gotten to the
> point that it was striving for more than making gcc be able to run on it. 

Could you /please/ stop your snide remarks on the code and its authors? If
for nothing else, the very people you are insulting in public are the exact
ones that will decide if they take on the work of auditing and integrating
your code.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Horst von Brand wrote:

>
>
>Funny that the "texbook algorithms" aren't used in real life. Wonder why...
>  
>
Try BSD.  If the BSD book can be believed, they use"texbook algorithms".

;-)

Re: I request inclusion of reiser4 in the mainline kernel

[Jens Axboe]

On Tue, Sep 20 2005, Hans Reiser wrote:
> Horst von Brand wrote:
> 
> >
> >
> >Funny that the "texbook algorithms" aren't used in real life. Wonder why...
> >  
> >
> Try BSD.  If the BSD book can be believed, they use"texbook algorithms".

Even the BSD people are looking for better algorithms. To be honest, I
think any OS using the classic elevator sort is a joke. It's only really
suited for batch processing.

-- 
Jens Axboe

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Hans Reiser writes:
 > Horst von Brand wrote:
 > 
 > >
 > >
 > >Funny that the "texbook algorithms" aren't used in real life. Wonder why...
 > >  
 > >
 > Try BSD.  If the BSD book can be believed, they use"texbook algorithms".

The "textbook" one-way elevator (as indeed exemplified by FreeBSD's
src/sys/kern/subr_disk.c:bioq_disksort()) has well-known weaknesses. For
example,

    dd if=/dev/zero of=FILE

can easily monopolize device queue and starve accesses to the blocks
with low block numbers.

 > 
 > ;-)

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nikita Danilov wrote:

>Hans Reiser writes:
> > Horst von Brand wrote:
> > 
> > >
> > >
> > >Funny that the "texbook algorithms" aren't used in real life. Wonder why...
> > >  
> > >
> > Try BSD.  If the BSD book can be believed, they use"texbook algorithms".
>
>The "textbook" one-way elevator (as indeed exemplified by FreeBSD's
>src/sys/kern/subr_disk.c:bioq_disksort()) has well-known weaknesses. For
>example,
>
>    dd if=/dev/zero of=FILE
>
>can easily monopolize device queue and starve accesses to the blocks
>with low block numbers.
>
> > 
> > ;-)
>
>Nikita.
>
>
>  
>
Yes, and one can compensate for them  fairly cleanly.  I can't say more
without the customer releasing the code first.

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Hans Reiser writes:

[...]

 > >
 > Yes, and one can compensate for them  fairly cleanly.  I can't say more
 > without the customer releasing the code first.

That's the point: text-book algorithms are usually useless as is. They
need adjustments and changes to work in real life.

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nikita Danilov wrote:

>Hans Reiser writes:
>
>[...]
>
> > >
> > Yes, and one can compensate for them  fairly cleanly.  I can't say more
> > without the customer releasing the code first.
>
>That's the point: text-book algorithms are usually useless as is. They
>need adjustments and changes to work in real life.
>
>Nikita.
>
>
>  
>
Yes, but you want to understand the textbook algorithms first before you
tweak. If you don't understand why least block number first is inferior
to real elevator.....

There is value to reading the classics of literature, even though they
are always simplistic compared to real life.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Horst von Brand wrote:

> Could you /please/ stop your snide remarks on the code and its authors? If
>
>for nothing else, the very people you are insulting in public are the exact
>ones that will decide if they take on the work of auditing and integrating
>your code.
>  
>
Our code was called messy.  It is less messy than the rest of the
kernel.  I was asked for specifics. I gave one.

Sure, let this thread die.  Can we all agree to that?

My guys will send patches responding to the technical portion of what
Hellwig said, the kthread stuff, etc., were useful comments to receive,
and I thank him for them.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Bill Davidsen]

Denis Vlasenko wrote:
> On Sunday 18 September 2005 03:34, Chris White wrote:
> 
>>CC-List trimmed
>>
>>On Saturday 17 September 2005 20:15, Denis Vlasenko wrote:
>>
>>>>At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more
>>>>time to optimize code size, but:
>>>>
>>>>reiser4        2557872 bytes
>>>>xfs            3306782 bytes
>>>
>>>And modules sizes:
>>>
>>>reiser4.ko        442012 bytes
>>>xfs.ko            494337 bytes
>>
>>All this is fine and dandy, but saying "My code is better than yours!!" still 
>>doesn't solve the issue this thread hopes to achieve, that being "I'd like to 
>>get reiser4 into the kernel".  There seems to be a lot of (historical?) 
>>tension present here, but all that seems to be doing is making things worse.  
>>PLEASE keep this thing a tad on par.  Keeping this up is hurting everyone 
>>more than helping.  I wish I could say something as simple as "let's just be 
>>friends", but that's saying a lot.  I can say this though: this is open 
>>source, and that means that our source is open, and we should be too.
> 
> 
> I am trying to say that I think that Hans is being treated a bit unfairly.
> His fs is new and has fairly complex on-disk structure and complex journalling
> machinery, yet his source and object code is smaller than xfs which already
> is accepted. This is no easy feat I guess.
> 
> Maybe xfs shouldn't be accepted too, this may be an answer.

That argument is specious, and raises the chance that someone will 
suggest that we learn from our mistakes.
> 
> Let's look at the code. Hans' code is not _that_ awful. Yet people
> (not all of them, but some) do not point to specific things which they
> want to be fixed/improved. I see blanket arguments like "your code is hard
> to read". Well. Maybe spend a minute on what exactly is hard to read,
> or do we require Hans to be able to read minds from the distance?

Was this code first written in some language which compiles into C, like 
the original C++ compiler? It just "feels funny," which clearly is not 
an objective or quantitative thing, just an impression. Your comment 
"not _that_ awful" is a perfect example of "damned by faint praise." 
However, unless someone is willing to make the argument that the code is 
too weird to maintain, style isn't an issue, is it?

Christoph has made some clear technical points, those should be 
addressed, whether you like the code style or not.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Bill Davidsen wrote:

> Denis Vlasenko wrote:
>
>>
>> Maybe xfs shouldn't be accepted too, this may be an answer.
>
>
> That argument is specious, and raises the chance that someone will
> suggest that we learn from our mistakes.

It wasn't a mistake to accept xfs, xfs is a great piece of technology. 
I haven't monitored their progress, but I am sure that they are
diligently fixing bugs as they are reported.  Guys, making a filesystem
stable is hard, and once every known test passes without crashing, there
is no substitute for masses of users finding bugs not hit by the tests,
and getting them fixed.  My point is merely that Hellwig could go and
find flaws in the filesystem he worked on, XFS, if he chose to.  I think
he should.

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Sat, Sep 17, 2005 at 01:56:14PM +0300, Denis Vlasenko wrote:
> At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more time
> to optimize code size, but:
> 
> reiser4        2557872 bytes
> xfs            3306782 bytes

and romfs is smaller than ext2, damn.  Should we remove all filesystems but
romfs now?


and yeah, if you didn't get the hint compare the feature sets.

Re: I request inclusion of reiser4 in the mainline kernel

[David Masover]

Christoph Hellwig wrote:
> On Sat, Sep 17, 2005 at 01:56:14PM +0300, Denis Vlasenko wrote:
> 
>>At least reiser4 is smaller. IIRC xfs is older than reiser4 and had more time
>>to optimize code size, but:
>>
>>reiser4        2557872 bytes
>>xfs            3306782 bytes
> 
> 
> and romfs is smaller than ext2, damn.  Should we remove all filesystems but
> romfs now?
> 
> 
> and yeah, if you didn't get the hint compare the feature sets.

XFS does have a nice feature set, sure.  So does Reiser4.

XFS can "freeze" the filesystem, take a live snapshot, and do some 
other, similar tricks.  Reiser4 can show file metadata as files 
themselves, compress on-the-fly (last I checked, the compression code is 
in there, just not being used), and pack small files incredibly well.

XFS has xattrs.  Reiser has metas, and will eventually provide an xattr 
interface to them.

You may not value Reiser's feature set, but that doesn't make it less 
complex.  Romfs is actually simpler than ext2, and its whole "feature" 
seems to be having a tiny implementation.

Re: I request inclusion of reiser4 in the mainline kernel

[Alexander Zarochentsev]

Hello,

On Friday 16 September 2005 21:40, Christoph Hellwig wrote:
> more trivial review comments ontop of the previous one, after looking
> at things:
>
>  - please never use list_for_each in new code but list_for_each_entry
>  - never use kernel_thread in new code but kthread_*

done.  thanks Christoph.

>  - do_sendfile duplicates the common sendfile code.  why aren't you
>    using the generic code?

I removed reiser4 version of do_sendfile and replaced it by 
generic_file_sendfile() under reiser4 context.
loop device still works.

>  - there's tons of really useless assertation of the category
>    discussed in the last thread
>  - there's tons of deep pagecache messing in there.  normally this
>    shouldn't be a filesystem, and if this breaks because of VM changes
> you'll have to fix it, don't complain..
>  - you still do your plugin mess in ->readpage.  honsetly could you
>    please explain why mpage_readpage{,s} don't work for you?
>  - (issues with the read/write path already addresses in the previous
> thread) - looking at ->d_count in ->release is wrong
>  - still has security plugin stuff that duplicates LSM
>  - why do underlying attributes change when VFS inode doesn't change?
>    if not please rip out most of getattr_common
>  - link_common S_ISDIR doesn't make sense, VFS takes care of it
>  - please use the generic_readlink infrastructure
>
> additinoal comment is that the code is very messy, very different
> from normal kernel style, full of indirections and thus hard to read.
> real review will take some time.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Thanks,
Alex.

Re: I request inclusion of reiser4 in the mainline kernel

[Vladimir V. Saveliev]

Hello

Christoph Hellwig wrote:
> more trivial review comments ontop of the previous one, after looking
> at things:
> 
>  - please never use list_for_each in new code but list_for_each_entry

done

>  - never use kernel_thread in new code but kthread_*

done

>  - do_sendfile duplicates the common sendfile code.  why aren't you
>    using the generic code?

fixed to use generic code.

>  - there's tons of really useless assertation of the category
>    discussed in the last thread
>  - there's tons of deep pagecache messing in there.  normally this
>    shouldn't be a filesystem, and if this breaks because of VM changes you'll
>    have to fix it, don't complain..

we are also not very happy with it. We will not complain, but try to simplify it.

>  - you still do your plugin mess in ->readpage.  honsetly could you
>    please explain why mpage_readpage{,s} don't work for you?

The reason is performance. Reiser4 uses a search through the filesystem tree to
access metadata of a file.
If reiser4 implemented its read/write via generic functions it would have to
repeat the search for every page being read/written.
It is especially disappointing because reiser4 uses extents with which it is
able in most cases to find all file pointers to data blocks with only one search
through the tree.
Another reason is multithread load.
Tree search includes complex part of providing correct concurrent read/write
access to the tree including deadlock avoidance algorithm. On multithread
filesystem load minimizing of a number of tree searches should have improved
filesystem throughput.

These are on todo list yet.
>  - (issues with the read/write path already addresses in the previous thread)
>  - looking at ->d_count in ->release is wrong
>  - still has security plugin stuff that duplicates LSM
>  - why do underlying attributes change when VFS inode doesn't change?
>    if not please rip out most of getattr_common
>  - link_common S_ISDIR doesn't make sense, VFS takes care of it
>  - please use the generic_readlink infrastructure
> 
> additinoal comment is that the code is very messy, very different
> from normal kernel style, full of indirections and thus hard to read.
> real review will take some time.

We hope to improve that with your help

Thanks

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Mon, Sep 26, 2005 at 07:03:52PM +0400, Vladimir V. Saveliev wrote:
> >  - you still do your plugin mess in ->readpage.  honsetly could you
> >    please explain why mpage_readpage{,s} don't work for you?
> 
> The reason is performance. Reiser4 uses a search through the filesystem tree to
> access metadata of a file.
> If reiser4 implemented its read/write via generic functions it would have to
> repeat the search for every page being read/written.
> It is especially disappointing because reiser4 uses extents with which it is
> able in most cases to find all file pointers to data blocks with only one search
> through the tree.
> Another reason is multithread load.
> Tree search includes complex part of providing correct concurrent read/write
> access to the tree including deadlock avoidance algorithm. On multithread
> filesystem load minimizing of a number of tree searches should have improved
> filesystem throughput.
> 
> These are on todo list yet.

Let's start with the read side.  You are using the generic sendfile now
which is the same as the generic read routine as far as the filesystem
is concerned.  What more do you need for read performance than a proper
mpage_readpage(s) that can use the extent-enabled get_blocks callback?
I'll post a patch for that ASAP, it benefits the other filesystems aswell.

I know the write path is more complex, and doing your on ->writepage(s)
is fine because that may need to do a lot of FS-specific things.  I'd
really like to see you use generic_file_write or at least the major
routines it's built from.  What's missing for you to do that?  Note
that you read/write path will need a major rewrite anyway due to the
flaws I pointed out, and I think it would benefit both you (less
maintaince overhead, feature and bug parity) and the other
filesystem/vfs/vm hackers (less duplicate functionality to worry about,
improvement you make shared by every filesystem) if you tried to
converge to use common code.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Christoph Hellwig wrote:

>On Mon, Sep 26, 2005 at 07:03:52PM +0400, Vladimir V. Saveliev wrote:
>  
>
>>> - you still do your plugin mess in ->readpage.  honsetly could you
>>>   please explain why mpage_readpage{,s} don't work for you?
>>>      
>>>
>>The reason is performance. Reiser4 uses a search through the filesystem tree to
>>access metadata of a file.
>>If reiser4 implemented its read/write via generic functions it would have to
>>repeat the search for every page being read/written.
>>It is especially disappointing because reiser4 uses extents with which it is
>>able in most cases to find all file pointers to data blocks with only one search
>>through the tree.
>>Another reason is multithread load.
>>Tree search includes complex part of providing correct concurrent read/write
>>access to the tree including deadlock avoidance algorithm. On multithread
>>filesystem load minimizing of a number of tree searches should have improved
>>filesystem throughput.
>>
>>These are on todo list yet.
>>    
>>
>
>Let's start with the read side.  You are using the generic sendfile now
>which is the same as the generic read routine as far as the filesystem
>is concerned.
>
I am not sure that is the right thing for us to be doing actually.  We
need to abstract things our own way to create a proper abstraction
underneath sys_reiser4() and the general naming system that we are
headed towards.  That means we need the "flow" abstraction as the basis
of our code.  The flow abstraction is based on the idea that we should
be able to connect to and read or write any stream of bytes, in user or
kernel space.  A flow is a stream of bytes.  That is useful for
constructing sys_reiser4() and other semantic enhancements that we
plan.  If you would like to generalize the vfs interface such that it
reads and writes to and from flows, and user space buffers are just a
special case used by read() and write(), and it is not done page at a
time, that could be good for vfs.

>  What more do you need for read performance than a proper
>mpage_readpage(s) that can use the extent-enabled
>
what does extent enabled mean?  Do you mean that it passes more than one
page to the fs?  Or do you mean that readahead windows are set by the fs
because readahead is affected by file layout?  Or both?

> get_blocks callback?
>I'll post a patch for that ASAP, it benefits the other filesystems aswell.
>
>I know the write path is more complex, and doing your on ->writepage(s)
>is fine because that may need to do a lot of FS-specific things.  I'd
>really like to see you use generic_file_write or at least the major
>routines it's built from.  What's missing for you to do that?
>
Well, to start with, reiser4_write is plugin specific.  Writing to a
file might require modifying a file that it inherits its contents from,
for instance.

I think though that there is another consideration, which is that
reiser4 is not licensed just for Linux.  We need to create a coherent
body of code that can be ported to operating systems that don't have vfs
layers specially modified for us.  The easiest way to do that is to
write our own code for the key things like read and write.

VFS is supposed to allow filesystems to have their own read and write
implementations.  That is its defined interface.  You should allow us to
use it.  Other filesystems that are less ambitious might well want to
use generic code, but reiser4 is not a yellow package with black lettering.

>  Note
>that you read/write path will need a major rewrite anyway due to the
>flaws I pointed out, and I think it would benefit both you (less
>maintaince overhead, feature and bug parity) and the other
>filesystem/vfs/vm hackers (less duplicate functionality to worry about,
>improvement you make shared by every filesystem) if you tried to
>converge to use common code.
>
>
>
>
>  
>

Re: I request inclusion of reiser4 in the mainline kernel

[Vladimir V. Saveliev]

Hello, Christoph

We have fixed most of your complains.
Would you be so kind to find some time and take a quick look at reiser4
(2.6.14-rc5-mm1 +
ftp://ftp.namesys.com/pub/reiser4-for-2.6/2.6.14-rc5-mm1/broken-out-3)


However, there are some problems:

Christoph Hellwig wrote:
> more trivial review comments ontop of the previous one, after looking
> at things:
> 
>  - please never use list_for_each in new code but list_for_each_entry
>  - never use kernel_thread in new code but kthread_*
>  - do_sendfile duplicates the common sendfile code.  why aren't you
>    using the generic code?
>  - there's tons of really useless assertation of the category
>    discussed in the last thread
>  - there's tons of deep pagecache messing in there.  normally this
>    shouldn't be a filesystem, and if this breaks because of VM changes you'll
>    have to fix it, don't complain..

I hope it is in better state now, still not perfect, though.

>  - you still do your plugin mess in ->readpage.  honsetly could you
>    please explain why mpage_readpage{,s} don't work for you?
>  - (issues with the read/write path already addresses in the previous thread)
>  - looking at ->d_count in ->release is wrong

this is not fixed yet, but it is on top of todo list. We need to find correct
way to understand when file can be packed.

>  - still has security plugin stuff that duplicates LSM
>  - why do underlying attributes change when VFS inode doesn't change?
>    if not please rip out most of getattr_common

Sorry, I am not sure what do you mean here. reiser4' getattr looks similar to
generic_fillattr but does few things differently. Please explain what is wrong
by your opinion.

>  - link_common S_ISDIR doesn't make sense, VFS takes care of it
>  - please use the generic_readlink infrastructure
> 
> additinoal comment is that the code is very messy, very different
> from normal kernel style, full of indirections and thus hard to read.
> real review will take some time.
> 

We are willing to improve reiser4 code in incremental way.
Please do not consider this additional comment as a major reason to not let
reiser4 to get included.

Thanks

Re: I request inclusion of reiser4 in the mainline kernel

[Denis Vlasenko]

On Friday 16 September 2005 20:05, Hans Reiser wrote:
> All objections have now been addressed so far as I can discern.

Random observation:

You can declare functions even if you never use them.
Thus here you can avoid using #if/#endif:

#if defined(REISER4_DEBUG) || defined(REISER4_DEBUG_MODIFY) || defined(REISER4_DEBUG_OUTPUT)
int znode_is_loaded(const znode * node /* znode to query */ );
#endif

--
vda

Re: I request inclusion of reiser4 in the mainline kernel

[George Garvey]

On Sat, Sep 17, 2005 at 02:16:20PM +0300, Denis Vlasenko wrote:
> Random observation:
> 
> You can declare functions even if you never use them.
> Thus here you can avoid using #if/#endif:
> 
> #if defined(REISER4_DEBUG) || defined(REISER4_DEBUG_MODIFY) || defined(REISER4_DEBUG_OUTPUT)
> int znode_is_loaded(const znode * node /* znode to query */ );
> #endif

   What is wrong with documentation, in your opinion?

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Denis Vlasenko writes:
 > On Friday 16 September 2005 20:05, Hans Reiser wrote:
 > > All objections have now been addressed so far as I can discern.
 > 
 > Random observation:
 > 
 > You can declare functions even if you never use them.
 > Thus here you can avoid using #if/#endif:
 > 
 > #if defined(REISER4_DEBUG) || defined(REISER4_DEBUG_MODIFY) || defined(REISER4_DEBUG_OUTPUT)
 > int znode_is_loaded(const znode * node /* znode to query */ );
 > #endif

It's other way around: declaration is guarded by the preprocessor
conditional so that nobody accidentally use znode_is_loaded() outside of
the debugging mode.

 > 
 > --
 > vda
 > 

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Stephen Pollei]

On 9/18/05, Nikita Danilov <nikita@clusterfs.com> wrote:
> Denis Vlasenko writes:
>  > On Friday 16 September 2005 20:05, Hans Reiser wrote:
>  > You can declare functions even if you never use them.
>  > Thus here you can avoid using #if/#endif:
> It's other way around: declaration is guarded by the preprocessor
> conditional so that nobody accidentally use znode_is_loaded() outside of
> the debugging mode.
Except it doesn't disallow someone from using znode_is_loaded, if you
wanted to do that you would have done this....
#if defined(REISER4_DEBUG) || defined(WHATEVER_ELSE)
int znode_is_loaded(const znode * node /* znode to query */ );
#else
#define znode_is_loaded(I_dont_care_you_are_going_to_) \
   } )die(]0now[>anyway<}}}}}}*bye*}
#endif
That way instead of silently(or -Wmissing-prototypes gving a warning)
quessing at a prototype and *maybe* geting a link time error, you get
a nice compile-time bomb-out.

So unless you have -Wmissing-prototypes and -Werror set then your
#if/#endif does very little indeed, especially with the size of kernel
it's easy to ignore yet another warning even if the missing-prototype
warning was set.
And if you would have gotten a link error then that is what you were
really depending on to save your bacon.

P.S. I'd make the define one line if gmail didn't word wrap too much.

-- 
http://dmoz.org/profiles/pollei.html
http://sourceforge.net/users/stephen_pollei/
http://www.orkut.com/Profile.aspx?uid=2455954990164098214
http://stephen_pollei.home.comcast.net/

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Stephen Pollei writes:
 > On 9/18/05, Nikita Danilov <nikita@clusterfs.com> wrote:
 > > Denis Vlasenko writes:
 > >  > On Friday 16 September 2005 20:05, Hans Reiser wrote:
 > >  > You can declare functions even if you never use them.
 > >  > Thus here you can avoid using #if/#endif:
 > > It's other way around: declaration is guarded by the preprocessor
 > > conditional so that nobody accidentally use znode_is_loaded() outside of
 > > the debugging mode.
 > Except it doesn't disallow someone from using znode_is_loaded, if you
 > wanted to do that you would have done

I just described why conditionals are used here. You can trust me: I
wrote that code. :-)

 > #if defined(REISER4_DEBUG) || defined(WHATEVER_ELSE)
 > int znode_is_loaded(const znode * node /* znode to query */ );
 > #else
 > #define znode_is_loaded(I_dont_care_you_are_going_to_) \
 >    } )die(]0now[>anyway<}}}}}}*bye*}
 > #endif
 > That way instead of silently(or -Wmissing-prototypes gving a warning)
 > quessing at a prototype and *maybe* geting a link time error, you get
 > a nice compile-time bomb-out.

Maybe, but kernel developers are supposed to watch for compiler
messages. People who use that technique definitely do.

 > 
 > So unless you have -Wmissing-prototypes and -Werror set then your
 > #if/#endif does very little indeed, especially with the size of kernel
 > it's easy to ignore yet another warning even if the missing-prototype
 > warning was set.

It's enough to monitor your own code, rather than the whole kernel.

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Benjamin Herrenschmidt]

> Maybe, but kernel developers are supposed to watch for compiler
> messages. People who use that technique definitely do.

You are making no sense. If the function isn't defined, it will break at
link time or depmod time instead of compile time, big deal... get rid of
those useless #ifdef's please.

Ben.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nikita Danilov wrote:

>
>It's enough to monitor your own code, rather than the whole kernel.
>  
>
To this I would add, the kernel should not have warnings when it
compiles.  Now THAT is a style requirement I would rigorously enforce if
I could.  I deeply regret that long ago there was a Linux scsi driver
author who left a warning in, and then after that they started
multiplying throughout the kernel.  It makes users much more confident
when they patch the kernel if there are no warnings that leave them
wondering if the new patch created the warning or if it was there before.

Hans

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

Nikita Danilov <nikita@clusterfs.com> wrote:
> Denis Vlasenko writes:
>  > On Friday 16 September 2005 20:05, Hans Reiser wrote:
>  > > All objections have now been addressed so far as I can discern.
>  > 
>  > Random observation:
>  > 
>  > You can declare functions even if you never use them.
>  > Thus here you can avoid using #if/#endif:
>  > 
>  > #if defined(REISER4_DEBUG) || defined(REISER4_DEBUG_MODIFY) || defined(REISER4_DEBUG_OUTPUT)
>  > int znode_is_loaded(const znode * node /* znode to query */ );
>  > #endif

> It's other way around: declaration is guarded by the preprocessor
> conditional so that nobody accidentally use znode_is_loaded() outside of
> the debugging mode.

Since when has a missing declaration prevented anyone calling a function in
C?!
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[Stephen Pollei]

On 9/19/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
> Nikita Danilov <nikita@clusterfs.com> wrote:
> > It's other way around: declaration is guarded by the preprocessor
> > conditional so that nobody accidentally use znode_is_loaded() outside of
> > the debugging mode.
> 
> Since when has a missing declaration prevented anyone calling a function in
> C?!
Never AFAIK... K&R, ANSI,ISO C89,  c99, whatever version that I know of...
You'd need -Werror and -Wmissing-prototype to make that worth it.
Otherwise the standard says they namesys requested "please make a good
quess as to what the args are...".
K&R didn't even have the kind of prototypes we know and love today...
So they shouldn't do this half-ass #if/#endif stuff.. either rip it
out, or be a man and make the compile fail when someone calls
znode_is_loaded , if thats what you really want. It's really over
silly anyway, as it will fail at link time if they had matching
preprocessor stuff around the function definition.
#if defined(DEBUG_THIS) || defined(DEBUG_THAT)
int znode_is_loaded(const znode * node) {return !!node->had_too_much_to_drink;}
#endif

-- 
http://dmoz.org/profiles/pollei.html
http://sourceforge.net/users/stephen_pollei/
http://www.orkut.com/Profile.aspx?uid=2455954990164098214
http://stephen_pollei.home.comcast.net/

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Stephen Pollei writes:
 > On 9/19/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
 > > Nikita Danilov <nikita@clusterfs.com> wrote:
 > > > It's other way around: declaration is guarded by the preprocessor
 > > > conditional so that nobody accidentally use znode_is_loaded() outside of
 > > > the debugging mode.
 > > 
 > > Since when has a missing declaration prevented anyone calling a function in
 > > C?!

It issues a warning, which is enough, given that reiser4 code was
warning-free most of the time.

 > Never AFAIK... K&R, ANSI,ISO C89,  c99, whatever version that I know of...
 > You'd need -Werror and -Wmissing-prototype to make that worth it.
 > Otherwise the standard says they namesys requested "please make a good

No, evil namesys is not responsible for this, Sir. :-)

 > quess as to what the args are...".
 > K&R didn't even have the kind of prototypes we know and love today...
 > So they shouldn't do this half-ass #if/#endif stuff.. either rip it
 > out, or be a man and make the compile fail when someone calls
 > znode_is_loaded , if thats what you really want. It's really over
 > silly anyway, as it will fail at link time if they had matching
 > preprocessor stuff around the function definition.

Failing (with a warning) at compilation is faster. And people are known
to went as far as to select major compiler version based on the
compilation speed.

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

Nikita Danilov <nikita@clusterfs.com> wrote:
> Stephen Pollei writes:
>  > On 9/19/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
>  > > Nikita Danilov <nikita@clusterfs.com> wrote:
>  > > > It's other way around: declaration is guarded by the preprocessor
>  > > > conditional so that nobody accidentally use znode_is_loaded()
>  > > > outside of the debugging mode.

>  > > Since when has a missing declaration prevented anyone calling a
>  > > function in C?!

> It issues a warning, which is enough, given that reiser4 code was
> warning-free most of the time.

It is supposed to go into the kernel, which is not exactly warning-free.
Besides, you don't know what idiotic new warnings the gcc people might
dream up the next round, so just relying on no warnings is extremely
unwise.

As was said before: It it is /really/ wrong, arrange for it not to compile
or not to link. If it isn't, well... then it wasn't that wrong anyway.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Horst von Brand wrote:

>Nikita Danilov <nikita@clusterfs.com> wrote:
>  
>
>
>It is supposed to go into the kernel, which is not exactly warning-free.
>  
>
While I have no passionate feelings about Nikita's ifdef, I must note
that Reiser4 will always be warning free within 3 days of my finding out
that somebody left a warning in.;-)

I hate messy code.;-)

The rest of the kernel should be fixed to be warning free.

>Besides, you don't know what idiotic new warnings the gcc people might
>dream up the next round, so just relying on no warnings is extremely
>unwise.
>  
>
I find the above unconvincing.

Is that what this thread boils down to, that you guys think the compile
should fail not warn? 

>As was said before: It it is /really/ wrong, arrange for it not to compile
>or not to link. If it isn't, well... then it wasn't that wrong anyway.
>  
>

Re: I request inclusion of reiser4 in the mainline kernel

[Horst von Brand]

Hans Reiser <reiser@namesys.com> wrote:
> Horst von Brand wrote:

[...]

> >It is supposed to go into the kernel, which is not exactly warning-free.

> While I have no passionate feelings about Nikita's ifdef, I must note
> that Reiser4 will always be warning free within 3 days of my finding out
> that somebody left a warning in.;-)

> I hate messy code.;-)

Me too. And I hate warnings. But what I hate most is code that has been
messed up to get an idiotic compiler to shut up. And it has been several
times that I've seen modifications to shut up the compiler, after which
modifications introduced bugs. The compiler then kept quiet due to the
"warning fix", when it would have screamed otherwise. Or where the compiler
was right in complaining, and the fix just did shut it up and did not fix
the real problem. Examples include gratuitous casts, "just initialize to
anything" so it doesn't warn about possible use without initialization.

> The rest of the kernel should be fixed to be warning free.

Unrealistic. Would be nice, but there are more pressing needs. And,as I
said above, just brute-forcing it warning-free without really understanding
what the warning is all about is /much/ worse than keeping an useless
warning.

> >Besides, you don't know what idiotic new warnings the gcc people might
> >dream up the next round, so just relying on no warnings is extremely
> >unwise.

> I find the above unconvincing.

That means you haven't used very many gcc versions. I've been around since
1.72 (or even earlier), and each single new gcc version (even minor
revisions) would complain about things the earlier ones thought were
A-OK. Even worse, to get correct code out of some versions you have to
write stuff that later versions find objectionable.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: I request inclusion of reiser4 in the mainline kernel

[Stephen Pollei]

On 9/20/05, Hans Reiser <reiser@namesys.com> wrote:
> Horst von Brand wrote:
> >Nikita Danilov <nikita@clusterfs.com> wrote:
> >It is supposed to go into the kernel, which is not exactly warning-free.

> Is that what this thread boils down to, that you guys think the compile
> should fail not warn?

I don't care if it fails or warns at compile time, but you shouldn't
misuse/abuse a warning by potentialily introducing an unrelated bug.

So if you had
#if defined(DEBUG_THIS) || defined(DEBUG_THAT) 
int znode_is_loaded(const struct znode *z);
#else
int znode_is_loaded(const struct znode *z)
  __attribute__((__warn_broken__("unavailible when not debuging")));
#endif
That would be great with me.. except __warn_broken__ or the like
doesn't exist AFAIK :-<
Closest thing is __attribute((__deprecated__)) but thats not quite right.

> >As was said before: It it is /really/ wrong, arrange for it not to compile
> >or not to link. If it isn't, well... then it wasn't that wrong anyway.

That is really true, if it is really wrong then make it so that trying
to do it simply breaks.
And if you are impatient then use a define to substitute crap into the
compile that will give you something you can't ignore as well. Make it
throw off sparks, get bonus points if you can make gcc segfault;->

#define znode_is_loaded(I_dont_care_you_are_going_to_) \
  } )die(]0now[>anyway<}}}}}}*bye*}

should stop a compile, but I don't think it's evil enough to cause gcc
to segfault.
If you didn't like my humor I'm sure you could code something more
concise that is as sick and twisted to crash a compile...
Hmmm. 
#if yadda yadda
int znode_is_loaded_yet_again(....)
#else
#define znode_is_loaded(z) ><<<>
/* break the compile if someone tries using it while not debuging */
#endif

That should break the parser as well. too bad there isn't a
_Pragma("error") or something... oh well.

Also note my opinion, doesn't really count if you grep the kernel
sources for pollei, you won't find anything.

-- 
http://dmoz.org/profiles/pollei.html
http://sourceforge.net/users/stephen_pollei/
http://www.orkut.com/Profile.aspx?uid=2455954990164098214
http://stephen_pollei.home.comcast.net/

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Stephen Pollei wrote:

>
>
>Also note my opinion, doesn't really count if you grep the kernel
>sources for pollei, you won't find anything.
>
>  
>
Your opinion counts, but lets see what Nikita says before I say
anything.  Nikita is more expert than I in regards to compiler tricks.

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Hans Reiser writes:
 > Stephen Pollei wrote:
 > 
 > >
 > >
 > >Also note my opinion, doesn't really count if you grep the kernel
 > >sources for pollei, you won't find anything.
 > >
 > >  
 > >
 > Your opinion counts, but lets see what Nikita says before I say
 > anything.  Nikita is more expert than I in regards to compiler tricks.

That sort of guards was useful early in reiser4 debugging, when a lot of
checks and assertions was  added (specifically, because reiser4 has many
levels of debugging), but I am not sure source code clutter is justified
now.

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nikita Danilov wrote:

>Hans Reiser writes:
> > Stephen Pollei wrote:
> > 
> > >
> > >
> > >Also note my opinion, doesn't really count if you grep the kernel
> > >sources for pollei, you won't find anything.
> > >
> > >  
> > >
> > Your opinion counts, but lets see what Nikita says before I say
> > anything.  Nikita is more expert than I in regards to compiler tricks.
>
>That sort of guards was useful early in reiser4 debugging, when a lot of
>checks and assertions was  added (specifically, because reiser4 has many
>levels of debugging), but I am not sure source code clutter is justified
>now.
>
>Nikita.
>
>
>  
>
So what do you suggest we change it to, Nikita?

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Hans Reiser writes:

[...]

 > >
 > So what do you suggest we change it to, Nikita?

Just remove #ifdef/#endif as was suggested.

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Nikita Danilov wrote:

>Hans Reiser writes:
>
>[...]
>
> > >
> > So what do you suggest we change it to, Nikita?
>
>Just remove #ifdef/#endif as was suggested.
>
>Nikita.
>
>
>  
>
Ok.  vs, please do so.

Re: I request inclusion of reiser4 in the mainline kernel

[Alexandre Oliva]

On Sep 20, 2005, Stephen Pollei <stephen.pollei@gmail.com> wrote:

> On 9/19/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
>> Nikita Danilov <nikita@clusterfs.com> wrote:
>> > It's other way around: declaration is guarded by the preprocessor
>> > conditional so that nobody accidentally use znode_is_loaded() outside of
>> > the debugging mode.
>> 
>> Since when has a missing declaration prevented anyone calling a function in
>> C?!
> Never AFAIK... K&R, ANSI,ISO C89,  c99, whatever version that I know of...

Actually...  C99 requires a declaration (not necessarily with a
prototype) before a function can be called.  A prior declaration is
required for all identifiers.  I'm not sure whether this is new in C99
or carried over from ISO C90 (AKA ANSI C89).  The fact that so many
compilers accept calls without prior declarations is a common
extension to the language, mainly for backward compatibility.

> It's really over silly anyway, as it will fail at link time if they
> had matching preprocessor stuff around the function definition.

Not really.  A compiler might optimize away the reference to the
symbol if it's say guarded by a condition whose value can be
determined to be false at compile time.  If you rely on that, moving
to a different compiler that is unable to compute the condition value,
or simply is pickier as to standard compliance, will get you errors.

-- 
Alexandre Oliva         http://www.lsd.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: I request inclusion of reiser4 in the mainline kernel

[Stephen Pollei]

On 9/20/05, Alexandre Oliva <aoliva@redhat.com> wrote:
> On Sep 20, 2005, Stephen Pollei <stephen.pollei@gmail.com> wrote:
> > On 9/19/05, Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
> >> Since when has a missing declaration prevented anyone calling a function in
> >> C?!
> > Never AFAIK... K&R, ANSI,ISO C89,  c99, whatever version that I know of...
> 
> Actually...  C99 requires a declaration (not necessarily with a
> prototype) before a function can be called.  A prior declaration is
> required for all identifiers.

OK thank you for your correction.

> I'm not sure whether this is new in C99
> or carried over from ISO C90 (AKA ANSI C89).  The fact that so many
> compilers accept calls without prior declarations is a common
> extension to the language, mainly for backward compatibility.

yep I just tested a small program with different flags...
int main(void) {
  int ret;
  ret=my_func(3);
  return 0; }

float my_func(double x) {
  return x+2.0;}

gcc -Wall test_proto.c --std=c99
and even gcc -Wall test_proto.c --std=c99 -pedantic
give me this:
test_proto.c: In function `main':
test_proto.c:6: warning: implicit declaration of function `my_func'
test_proto.c: At top level:
test_proto.c:9: warning: type mismatch with previous implicit declaration
test_proto.c:6: warning: previous implicit declaration of `my_func'
test_proto.c:9: warning: `my_func' was previously implicitly declared
to return `int'

it takes gcc -Wall test_proto.c --std=c99 -pedantic-errors to cause it
not to create the a.out .
So gcc should have caused an error as I didn't set --std=gnu99 .. bad compiler.

So I don't know howto get gcc to follow the standards in this area,
that sounds like a good thing to require.

> > It's really over silly anyway, as it will fail at link time if they
> > had matching preprocessor stuff around the function definition.

> Not really.  A compiler might optimize away the reference to the
> symbol if it's say guarded by a condition whose value can be
> determined to be false at compile time.  If you rely on that, moving
> to a different compiler that is unable to compute the condition value,
> or simply is pickier as to standard compliance, will get you errors.

True again, especially since the kernel code itself relies on that
kind of behavior in some of it's inline functions or macros for
example to cause link errors only when specific conditions arise.
so the construct:

#if conditions
/* function prototype(s) */
#else
#define whatever(a,b,c) /* something to crash the compile */
#endif
is truely the best solution, if the namesys people want to be
absolutely sure to catch all
calls to znode_is_loaded when debuging is not set....

#define znode_is_loaded(I_dont_care_you_are_going_to_) \
 } )die(]0now[>anyway<}}}}}}*bye*}
#define znode_is_loaded(z) ><<<>

Either one of the above defines should be sufficiently crappy to cause
gcc to bomb out.
I'm so sure(some say full) of it that I'm too lazy to test it out for myself.
Of course a _Pragma("error") would`a been nice.

Simply not providing a prototype to generate a warning was just
BAD(B0rken As Designed).

-- 
http://dmoz.org/profiles/pollei.html
http://sourceforge.net/users/stephen_pollei/
http://www.orkut.com/Profile.aspx?uid=2455954990164098214
http://stephen_pollei.home.comcast.net/

Re: I request inclusion of reiser4 in the mainline kernel

[Alexandre Oliva]

On Sep 20, 2005, Stephen Pollei <stephen.pollei@gmail.com> wrote:

> it takes gcc -Wall test_proto.c --std=c99 -pedantic-errors to cause it
> not to create the a.out .
> So gcc should have caused an error as I didn't set --std=gnu99 .. bad compiler.
> So I don't know howto get gcc to follow the standards in this area,
> that sounds like a good thing to require.

gnu99 is the default.  Also, the standard doesn't talk about errors or
warnings, it only requires diagnostics for ill-formed code.  Deciding
what kind of diagnostic to issue is a compiler implementation
decision.

-- 
Alexandre Oliva         http://www.lsd.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}

Re: I request inclusion of reiser4 in the mainline kernel

[Hans Reiser]

Denis Vlasenko wrote:

>On Friday 16 September 2005 20:05, Hans Reiser wrote:
>  
>
>>All objections have now been addressed so far as I can discern.
>>    
>>
>
>Random observation:
>
>You can declare functions even if you never use them.
>Thus here you can avoid using #if/#endif:
>
>#if defined(REISER4_DEBUG) || defined(REISER4_DEBUG_MODIFY) || defined(REISER4_DEBUG_OUTPUT)
>int znode_is_loaded(const znode * node /* znode to query */ );
>#endif
>
>--
>vda
>
>
>  
>
thanks.

zam, please address this.

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

I threw in your new codedrop into a compilation and the byte-order
mess is _still_ now sorted out.  Please kill the d* as struct type
crap and just use __le types directly.

Also lots of "memset with byte count of 0" warnings from sparse.

Re: I request inclusion of reiser4 in the mainline kernel

[Vladimir V. Saveliev]

Hello

Christoph Hellwig wrote:
> I threw in your new codedrop into a compilation and the byte-order
> mess is _still_ now sorted out.  

Did compiling notice this mess?
We used to compile with C=1. Should we compile somehow else before sending code out?

> Please kill the d* as struct type
> crap and just use __le types directly.
> 
ok

> Also lots of "memset with byte count of 0" warnings from sparse.
> 

Running make C=1 fs/reiser4/ with sparse from http://www.codemonkey.org.uk/projects/git-snapshots/sparse/sparse-2005-09-19.tar.gz
does not produce any warnings.
Would you please prompt how to make sparse to complain?

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Mon, Sep 19, 2005 at 01:18:58PM +0400, Vladimir V. Saveliev wrote:
> Hello
> 
> Christoph Hellwig wrote:
> > I threw in your new codedrop into a compilation and the byte-order
> > mess is _still_ now sorted out.  
> 
> Did compiling notice this mess?
> We used to compile with C=1. Should we compile somehow else before sending code out?

Log is at:

	http://verein.lst.de/~hch/reiser4.log

C=1 with whatever sparse version I had, nevermind what version it is code
is not good, just use __le types instead of structs and casting exercises,
makes the code simpler and easier to verify.

Re: I request inclusion of reiser4 in the mainline kernel

[Vladimir V. Saveliev]

Hello

Christoph Hellwig wrote:
> ...
> Looking at the actual code all these point to the spin lock obsufcation
> SPIN_LOCK_FUNCTIONS/RW_LOCK_FUNCTIONS from spin_macros.h which I told
> to get rid of in the first round of reviews. 
> ...

reiser4 spinlock macros provide following functionality:

    (1) encapsulation of locks: instead of writing spin_lock(&obj->lock),
    where obj is object of type foo, one writes spin_lock_foo(obj).

    (2) keeping information about number of locks of particular type currently
held by thread

    (3) checking that locks are acquired in the proper order.

    (4) collection of spin lock contention statistics


I agree that (1) is not very necessary. (2) and (4) helped a lot in early
debugging. Now we are about to remove it.

However, we would prefer to keep (3). It makes catching spinlock deadlocks very
easy. Don't you think that makes sence?

Thanks

Re: I request inclusion of reiser4 in the mainline kernel

[Nikita Danilov]

Vladimir V. Saveliev writes:
 > Hello
 > 
 > Christoph Hellwig wrote:
 > > ...
 > > Looking at the actual code all these point to the spin lock obsufcation
 > > SPIN_LOCK_FUNCTIONS/RW_LOCK_FUNCTIONS from spin_macros.h which I told
 > > to get rid of in the first round of reviews. 
 > > ...
 > 
 > reiser4 spinlock macros provide following functionality:
 > 
 >     (1) encapsulation of locks: instead of writing spin_lock(&obj->lock),
 >     where obj is object of type foo, one writes spin_lock_foo(obj).
 > 
 >     (2) keeping information about number of locks of particular type currently
 > held by thread
 > 
 >     (3) checking that locks are acquired in the proper order.
 > 
 >     (4) collection of spin lock contention statistics
 > 
 > 
 > I agree that (1) is not very necessary. (2) and (4) helped a lot in early
 > debugging. Now we are about to remove it.

This was already discussed during earlier attempts to merge reiser4. The
proper solution purportedly is to make useful features of reiser4
spin-lock code generic and merge them so that the rest of kernel can
enjoy their superiority.

 > 
 > However, we would prefer to keep (3). It makes catching spinlock deadlocks very
 > easy. Don't you think that makes sence?

Lock-ordering monitoring was _immensely_ useful. For one thing it forces
one to have complete and up to date description of lock ordering.

 > 
 > Thanks

Nikita.

Re: I request inclusion of reiser4 in the mainline kernel

[Christoph Hellwig]

On Fri, Sep 30, 2005 at 04:10:09PM +0400, Vladimir V. Saveliev wrote:
> > Looking at the actual code all these point to the spin lock obsufcation
> > SPIN_LOCK_FUNCTIONS/RW_LOCK_FUNCTIONS from spin_macros.h which I told
> > to get rid of in the first round of reviews. 
> > ...
> 
> reiser4 spinlock macros provide following functionality:
> 
>     (1) encapsulation of locks: instead of writing spin_lock(&obj->lock),
>     where obj is object of type foo, one writes spin_lock_foo(obj).
> 
>     (2) keeping information about number of locks of particular type currently
> held by thread
> 
>     (3) checking that locks are acquired in the proper order.
> 
>     (4) collection of spin lock contention statistics
> 
> 
> I agree that (1) is not very necessary. (2) and (4) helped a lot in early
> debugging. Now we are about to remove it.
> 
> However, we would prefer to keep (3). It makes catching spinlock deadlocks very
> easy. Don't you think that makes sence?

(4) is provided by the lockmeter patch which is or has been in -mm.
(1) is not just unessecary but actually considered obsfucation
(2) and (3) sounds useful and it would be cool if you could integrate
them into the core spinlock code.  I'd suggest to remove the feature
temporarily and reimplement it so further reiser4 progress isn't blocked
on that.

Re: I request inclusion of reiser4 in the mainline kernel

[Alexey Dobriyan]

On Mon, Sep 19, 2005 at 01:18:58PM +0400, Vladimir V. Saveliev wrote:
> Christoph Hellwig wrote:
> > I threw in your new codedrop into a compilation and the byte-order
> > mess is _still_ now sorted out.  
> 
> Did compiling notice this mess?
> We used to compile with C=1. Should we compile somehow else before sending code out?

Yes.

	make C=1 CF=-Wbitwise fs/reiser4/
or
	make C=1 CHECK="sparse -Wbitwise" fs/reiser4/

All coming from dformat.h