Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Ulrich Weigand]

Jeff Mahoney wrote:

> Are the S/390 asm/unaligned.h versions broken, or is the ReiserFS code doing 
> something not planned for? It's a 16-bit member, at a 16-bit alignment 
> in the structure. The structure itself need not be aligned in any 
> particular manner as it is read directly from disk, and is a packed structure.

One other point I overlooked before: using set_bit etc on a *16-bit* 
member is fundamentally broken on S/390 anyway (and I far as I know all 
other bigendian architectures as well).  set_bit assumes to operate on
a long (or an array of longs); if you use set_bit to set bit number 0
in the bitfield starting at address X, it will actually modify the byte 
located at address X+3 (or X+7 on 64-bit bigendian machines), because
this is where the bit with value 2^0 is located in a long.

If your bitfield is only 2 bytes long, this will obviously clobber
random memory after the field ...  (Note that the _unaligned variants
do not fix this problem, they will just cause it to clobber memory
*before* the field instead, if I interpret them correctly.)

Bye,
Ulrich

-- 
  Dr. Ulrich Weigand
  weigand@informatik.uni-erlangen.de

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Linus Torvalds]

In article <20010903.152443.59467554.davem@redhat.com>,
David S. Miller <davem@redhat.com> wrote:
>
>Oh thats different!  That won't even work %100 correctly on x86.  On
>x86 it will "execute", but it won't be atomic.

Actually, it will.  Intel definitely discourages it, but they'll lock
both cache-lines if the access is unaligned and crosses.  So while they
encourage natural alignment for atomic accesses, I think they also
guarantee that they always work - it ends up being only a performance
issue. 

I agree that it is bad practice, though, and I bet that the x86 is one
of the very few architectures that _will_ do this naturally.

		Linus

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Ulrich Weigand]

John Alvord wrote:

>> It is only *atomic* accesses (those implemented using the S/390
>> compare-and-swap instruction) that need to be word aligned; this
includes
>> the atomic bit operations that reiserfs appears to be using.
>
>Aren't their some other "must align" instructions like CVB? Or have they
>all been relaxed...

CVB doesn't have any alignment requirement (I'm not sure it ever had one).
Execpt for the 'atomic' operations (CS, CSG, CDS, CDSG, LPQ, STPQ, PLO)
I know only of two general-purpose instructions with operand alignment
requirement, and that's LAM and STAM.  As access registers are not
normally used in Linux this shouldn't be a problem.

There *is* a whole bunch of privileged system instructions that have
various aligment requirements; but here I'd say it's fair to require the
user to provide correct aligment in these special cases.


Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand
  Linux for S/390 Design & Development
  IBM Deutschland Entwicklung GmbH, Schoenaicher Str. 220, 71032 Boeblingen
  Phone: +49-7031/16-3727   ---   Email: Ulrich.Weigand@de.ibm.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[John Alvord]

On Tue, 4 Sep 2001, Ulrich Weigand wrote:

> 
> Jeff Mahoney wrote:
> 
> >    Are the S/390 asm/unaligned.h versions broken, or is the ReiserFS code
> doing
> >    something not planned for? It's a 16-bit member, at a 16-bit alignment
> >    in the structure.  The structure itself need not be aligned in any
> >    particular manner as it is read directly from disk, and is a packed
> structure.
> 
> The S/390 unaligned.h macros are just direct assignments because the
> S/390 hardware normally *allows* unaligned accesses just fine.
> 
> It is only *atomic* accesses (those implemented using the S/390
> compare-and-swap instruction) that need to be word aligned; this includes
> the atomic bit operations that reiserfs appears to be using.

Aren't their some other "must align" instructions like CVB? Or have they
all been relaxed...

john

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Richard B. Johnson]

On Tue, 4 Sep 2001, Ulrich Weigand wrote:
[SNIPPED...]

> 
> If these instructions really *need* to be atomic, then reiserfs should
> ensure they are performed on properly aligned data, or else there might
> be subtle bugs even on Intel, because the operations will not actually
> be atomic (even though they don't trap).
>

Regardless of alignment, locked instructions in Intel machines are
atomic. Any "hidden" read/modify/write operations are performed
by the hardware, under the lock, preventing access by any other
CPUs or DMA.
 
> If you say that reiserfs doesn't really need these operations to be
> atomic because they run under other locks anyway, then they should not
> be using atomic operations in the first place; this will only cause
> unnecessary slowdown even on Intel.
> 

Agreed that there are many "atomic_t" types on some drivers, and
"atomic" operations that don't need to be there. Unless a DMA operation
is in progress, anything executing under a spin-lock doesn't need
to be an "atomic" operation to make it, truly, atomic.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Ulrich Weigand]

Jeff Mahoney wrote:

>    Are the S/390 asm/unaligned.h versions broken, or is the ReiserFS code
doing
>    something not planned for? It's a 16-bit member, at a 16-bit alignment
>    in the structure.  The structure itself need not be aligned in any
>    particular manner as it is read directly from disk, and is a packed
structure.

The S/390 unaligned.h macros are just direct assignments because the
S/390 hardware normally *allows* unaligned accesses just fine.

It is only *atomic* accesses (those implemented using the S/390
compare-and-swap instruction) that need to be word aligned; this includes
the atomic bit operations that reiserfs appears to be using.

If these instructions really *need* to be atomic, then reiserfs should
ensure they are performed on properly aligned data, or else there might
be subtle bugs even on Intel, because the operations will not actually
be atomic (even though they don't trap).

If you say that reiserfs doesn't really need these operations to be
atomic because they run under other locks anyway, then they should not
be using atomic operations in the first place; this will only cause
unnecessary slowdown even on Intel.


Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand
  Linux for S/390 Design & Development
  IBM Deutschland Entwicklung GmbH, Schoenaicher Str. 220, 71032 Boeblingen
  Phone: +49-7031/16-3727   ---   Email: Ulrich.Weigand@de.ibm.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Jeff Mahoney]

On Tue, Sep 04, 2001 at 11:44:30AM +0200, Andi Kleen wrote:
> Jeff Mahoney <jeffm@suse.com> writes:
> 
> 
> >     I did kick around the idea of making those macros the default accessors for
> >     the deh_state member (which is the only place they're used), but it unfairly
> >     penalizes arches that don't need them.
> 
> On archs that don't need them {get,put}_unaligned should be just normal
> assignments. They are certainly on i386.

    Sorry, I guess I wasn't clear.

    When I mentioned "make those macros the default ..", I was referring to the
    reiserfs-defined macros, not the asm/unaligned.h macros.

    In my previous message, I had mentioned that the get/put _unaligned macros
    from asm/unaligned.h don't work in all cases. Specifically, the S/390 (and
    S/390x) versions won't work with ReiserFS, since they're nothing more than
    a normal access/mutate put into a compatible macro. Through testing on the
    S/390{,x}, I found that using the reiserfs-defined unaligned macros did the
    trick. The only place these reiserfs-defined macros are used is to
    access/mutate the reiserfs_de_head->deh_state member, which contains flags
    for the on-disk directory entry representation.
 
    Are the S/390 asm/unaligned.h versions broken, or is the ReiserFS code doing
    something not planned for? It's a 16-bit member, at a 16-bit alignment
    in the structure.  The structure itself need not be aligned in any
    particular manner as it is read directly from disk, and is a packed structure.

    -Jeff

-- 
Jeff Mahoney           |   "Bill Gates is a monocle and a Persian cat away
jeffm@suse.com         |    from being the villain in a James Bond movie."
jeffm@csh.rit.edu      |                   -- Dennis Miller

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[David S. Miller]

   From: Andi Kleen <ak@suse.de>
   Date: Tue, 4 Sep 2001 12:25:24 +0200

   On Tue, Sep 04, 2001 at 03:04:54AM -0700, David S. Miller wrote:
   > I can also almost guarentee you that the x86 will sometimes not
   > execute these bitops atomically on SMP.
   
   It's not needed when you have another lock to protect and don't have
   interrupt threads.

I agree completely, so therefore if moving to non-atomic bitops
is an option that would be a great way to fix this reiserfs bug.

Later,
David S. Miller
davem@redhat.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Andi Kleen]

On Tue, Sep 04, 2001 at 03:04:54AM -0700, David S. Miller wrote:
>    From: Andi Kleen <ak@suse.de>
>    Date: 04 Sep 2001 11:44:30 +0200
> 
>    Jeff Mahoney <jeffm@suse.com> writes:
>    
>    >     I did kick around the idea of making those macros the default accessors for
>    >     the deh_state member (which is the only place they're used), but it unfairly
>    >     penalizes arches that don't need them.
>    
>    On archs that don't need them {get,put}_unaligned should be just normal
>    assignments. They are certainly on i386.
> 
> I can also almost guarentee you that the x86 will sometimes not
> execute these bitops atomically on SMP.

It's not needed when you have another lock to protect and don't have
interrupt threads.

iirc near all of reiserfs still runs under the BKL. BKL/schedule are full
memory barriers.

> What's more, you will have less QA'ing to do, since this code will
> always be in use and thus tested.

I think for a fs it is best to go with bigger locks and avoid the issue
of atomic operations completely.

-Andi

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[David S. Miller]

   From: Andi Kleen <ak@suse.de>
   Date: 04 Sep 2001 11:44:30 +0200

   Jeff Mahoney <jeffm@suse.com> writes:
   
   >     I did kick around the idea of making those macros the default accessors for
   >     the deh_state member (which is the only place they're used), but it unfairly
   >     penalizes arches that don't need them.
   
   On archs that don't need them {get,put}_unaligned should be just normal
   assignments. They are certainly on i386.

I can also almost guarentee you that the x86 will sometimes not
execute these bitops atomically on SMP.

We had some obscure bug on SMP/x86 years ago, and Linus discovered
that removing an unaligned spinlock or bitop made the problem go away.

Reiserfs is broken and needs to be fixed.

If you make the unaligned accessors there the default for everyone,
you solve the problem _AND_ there is no penalization.  Look at what
the compiler makes of the code generated, it is going to be almost
entirely identical.  The compiler should be able to compute it all
via constants.  If not, oh you get 1 or 2 instructions here or
there, and that is MINISCULE compared to the cost of the atomic
operation itself.

What's more, you will have less QA'ing to do, since this code will
always be in use and thus tested.

FACT: Doing bitops on something not "long" aligned is a bug and
      will always be a bug.  You must fix it.

Later,
David S. Miller
davem@redhat.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Andi Kleen]

Jeff Mahoney <jeffm@suse.com> writes:


>     I did kick around the idea of making those macros the default accessors for
>     the deh_state member (which is the only place they're used), but it unfairly
>     penalizes arches that don't need them.

On archs that don't need them {get,put}_unaligned should be just normal
assignments. They are certainly on i386.


-Andi

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Jeff Mahoney]

On Mon, Sep 03, 2001 at 12:34:37AM +0200, Richard Zidlicky wrote:
> On Sun, Sep 02, 2001 at 07:57:17PM +0200, thunder7@xs4all.nl wrote:
> > 
> > --- linux/include/linux/reiserfs_fs.h   Sun Sep  2 21:54:25 2001
> > +++ linux-new/include/linux/reiserfs_fs.h       Sun Sep  2 20:47:27 2001
> > @@ -924,7 +924,7 @@
> >  #define DEH_Visible 2
> > 
> >  /* 64 bit systems (and the S/390) need to be aligned explicitly -jdm */
> > -#if BITS_PER_LONG == 64 || defined(__s390__)
> > +#if BITS_PER_LONG == 64 || defined(__s390__) || defined(__hppa__)
> >  #   define ADDR_UNALIGNED_BITS  (3)
> >  #endif
> 
> couldn't reiserfs use asm/unaligned.h like anyone else?
> Seems at least sparc and mips may need the same treatment.

    I'll be the first to admit that having the #if followed by arch defines
    really sucks, and that asm/unaligned.h would be much cleaner if for no
    other reason than avoiding an unnecessary gotcha for someone porting a new
    arch.

    Unfortunately, this doesn't appear to work in all cases. I know from
    testing that the S/390 case will fail using the {get,put)_unaligned macros,
    since they are identical to a normal, unassisted assignment.

    I did kick around the idea of making those macros the default accessors for
    the deh_state member (which is the only place they're used), but it unfairly
    penalizes arches that don't need them.

    I'm open to suggestions.

    -Jeff

-- 
Jeff Mahoney           |   "Bill Gates is a monocle and a Persian cat away
jeffm@suse.com         |    from being the villain in a James Bond movie."
jeffm@csh.rit.edu      |                   -- Dennis Miller

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[David S. Miller]

   From: "Ulrich Weigand" <Ulrich.Weigand@de.ibm.com>
   Date: Mon, 3 Sep 2001 14:08:43 +0200
   
   >From what I recall when we were looking into reiserfs on S/390,
   the core problem was that reiserfs tried to do *atomic* operations
   on non-aligned words.  This isn't supported by the hardware on
   S/390 (normal non-aligned accesses just work).
   
   I don't really see how this can be fixed in a trap handler; how
   would the handler guarantee atomicity?

Oh thats different!  That won't even work %100 correctly on x86.  On
x86 it will "execute", but it won't be atomic.

Why the F*CK does it need to do this?  It sounds like sloppy
programming to me.

Later,
David S. Miller
davem@redhat.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Ralf Baechle]

On Mon, Sep 03, 2001 at 02:08:43PM +0200, Ulrich Weigand wrote:

> >From what I recall when we were looking into reiserfs on S/390,
> the core problem was that reiserfs tried to do *atomic* operations
> on non-aligned words.  This isn't supported by the hardware on
> S/390 (normal non-aligned accesses just work).
> 
> I don't really see how this can be fixed in a trap handler; how
> would the handler guarantee atomicity?

Spinlocks.  Now that'd so infinitly ugly that I'd rather fix Reiserfs.
It's another proof that reiserfs design was done without too much
consideration of portability so I speculate we'll continue to see such
bugs.

  Ralf

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Ulrich Weigand]

David Miller wrote:

>   From: Richard Zidlicky
<Richard.Zidlicky@stud.informatik.uni-erlangen.de>
>   Date: Mon, 3 Sep 2001 00:34:37 +0200
>
>   On Sun, Sep 02, 2001 at 07:57:17PM +0200, thunder7@xs4all.nl wrote:
>   >  /* 64 bit systems (and the S/390) need to be aligned explicitly -jdm
*/
>   > -#if BITS_PER_LONG == 64 || defined(__s390__)
>   > +#if BITS_PER_LONG == 64 || defined(__s390__) || defined(__hppa__)
>   >  #   define ADDR_UNALIGNED_BITS  (3)
>   >  #endif
>
>   couldn't reiserfs use asm/unaligned.h like anyone else?
>   Seems at least sparc and mips may need the same treatment.
>
>Sparc will act correctly for unaliagned accesses.
>
>It will trap and run very slowly, but it wont' OOPS and
>it will give correct results.
>
>This is actually required behavior, I don't know why parisc
>is acting differently.

>From what I recall when we were looking into reiserfs on S/390,
the core problem was that reiserfs tried to do *atomic* operations
on non-aligned words.  This isn't supported by the hardware on
S/390 (normal non-aligned accesses just work).

I don't really see how this can be fixed in a trap handler; how
would the handler guarantee atomicity?



Mit freundlichen Gruessen / Best Regards

Ulrich Weigand

--
  Dr. Ulrich Weigand
  Linux for S/390 Design & Development
  IBM Deutschland Entwicklung GmbH, Schoenaicher Str. 220, 71032 Boeblingen
  Phone: +49-7031/16-3727   ---   Email: Ulrich.Weigand@de.ibm.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[David S. Miller]

   From: Richard Zidlicky <Richard.Zidlicky@stud.informatik.uni-erlangen.de>
   Date: Mon, 3 Sep 2001 00:34:37 +0200

   On Sun, Sep 02, 2001 at 07:57:17PM +0200, thunder7@xs4all.nl wrote:
   >  /* 64 bit systems (and the S/390) need to be aligned explicitly -jdm */
   > -#if BITS_PER_LONG == 64 || defined(__s390__)
   > +#if BITS_PER_LONG == 64 || defined(__s390__) || defined(__hppa__)
   >  #   define ADDR_UNALIGNED_BITS  (3)
   >  #endif
   
   couldn't reiserfs use asm/unaligned.h like anyone else?
   Seems at least sparc and mips may need the same treatment.

Sparc will act correctly for unaliagned accesses.

It will trap and run very slowly, but it wont' OOPS and
it will give correct results.

This is actually required behavior, I don't know why parisc
is acting differently.

Later,
David S. Miller
davem@redhat.com

Re: [SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[Richard Zidlicky]

On Sun, Sep 02, 2001 at 07:57:17PM +0200, thunder7@xs4all.nl wrote:
> 
> --- linux/include/linux/reiserfs_fs.h   Sun Sep  2 21:54:25 2001
> +++ linux-new/include/linux/reiserfs_fs.h       Sun Sep  2 20:47:27 2001
> @@ -924,7 +924,7 @@
>  #define DEH_Visible 2
> 
>  /* 64 bit systems (and the S/390) need to be aligned explicitly -jdm */
> -#if BITS_PER_LONG == 64 || defined(__s390__)
> +#if BITS_PER_LONG == 64 || defined(__s390__) || defined(__hppa__)
>  #   define ADDR_UNALIGNED_BITS  (3)
>  #endif

couldn't reiserfs use asm/unaligned.h like anyone else?
Seems at least sparc and mips may need the same treatment.

Bye
Richard

[SOLVED + PATCH]: documented Oops running big-endian reiserfs on parisc architecture

[thunder7]

On Sun, Sep 02, 2001 at 03:00:23PM +0100, Matthew Wilcox wrote:
> On Sun, Sep 02, 2001 at 10:55:38AM +0200, thunder7@xs4all.nl wrote:
> > ReiserFS version 3.6.25
> > bonnie[163]: Unaligned data reference 28
> 
> > which makes the error somewhere around here in 
> > fs/reiserfs/namei.c, function reiserfs_add_entry, after call to
> > padd_item, before call to reiserfs_find_entry:
> > 
> >     padd_item ((char *)(deh + 1), ROUND_UP (namelen), namelen);
> > 
> >     /* entry is ready to be pasted into tree, set 'visibility' and 'stat data in entry' attributes */
> >     mark_de_without_sd (deh);
> >     visible ? mark_de_visible (deh) : mark_de_hidden (deh);
> > 
> >     /* find the proper place for the new entry */
> >     memset (bit_string, 0, sizeof (bit_string));
> >     de.de_gen_number_bit_string = (char *)bit_string;
> >     retval = reiserfs_find_entry (dir, name, namelen, &path, &de);
> 
> I suspect mark_de_without_sd is an inlined function/macro and this will
> be where the unaligned data reference is happening.
> 
Correct. And the comments just above there about alignment are very
enlightening; it seems that the IBM/S390 architecture has some special
needs, and I just tested that my PA-RISC kernel has the same needs.
Thus I am able to present a real bugfix.

This patch allows me to run bonnie on a reiserfs partition with pa-risc
linux.

--- linux/include/linux/reiserfs_fs.h   Sun Sep  2 21:54:25 2001
+++ linux-new/include/linux/reiserfs_fs.h       Sun Sep  2 20:47:27 2001
@@ -924,7 +924,7 @@
 #define DEH_Visible 2

 /* 64 bit systems (and the S/390) need to be aligned explicitly -jdm */
-#if BITS_PER_LONG == 64 || defined(__s390__)
+#if BITS_PER_LONG == 64 || defined(__s390__) || defined(__hppa__)
 #   define ADDR_UNALIGNED_BITS  (3)
 #endif

This applies to linux-2.4.9-pa13 with
endian-safe-reiserfs-for-2.4.8.patch and to 2.4.9-ac5.

Please apply,
Jurriaan
-- 
It is well to remember, my son, that the entire population of the
universe, with one trifling exception, is composed of others.
        John Andrew Holmes
GNU/Linux 2.4.9-ac5 SMP/ReiserFS 2x1402 bogomips load av: 0.98 0.83 0.37