Proposal for a userspace "architecture portability" library

Proposal for a userspace "architecture portability" library

[Paul Mackerras]

Some of our kernel headers implement generally useful abstractions
across all of the architectures we support.  I would like to make an
"architecture portability" library, based on the kernel headers but as
a separate project from the kernel, and intended for use in userspace.

The headers that I want to base this on are:

atomic.h
bitops.h
byteorder.h
rwsem.h
semaphore.h
spinlock.h
system.h (for mb et al., xchg, cmpxchg)
unaligned.h

There are some others that may also be useful: cache.h, checksum.h,
io.h, xor.h.

Now, clearly I can do this under the GPL.  However, I think it would
be more useful to have the library under the LGPL, which requires
either getting the permission of the authors of the kernel files, or
rewriting them from scratch.

Linus (and other kernel copyright holders) - would you be willing to
relicense such of the above files that have your copyright under the
LGPL for this purpose?

I'm looking for volunteers to help with porting and testing on various
architectures.  I can do x86, ppc and ppc64, and I know sparc{,64} and
m68k assembler, but for the rest I'll need help.

My hope is that distributions will be able to use this to replace some
of the headers in /usr/include/asm, and thus reduce the desire for
applications to include kernel headers.

Paul.

Re: Proposal for a userspace "architecture portability" library

[Robert Love]

On Sun, 2004-12-05 at 11:53 +1100, Paul Mackerras wrote:

> Some of our kernel headers implement generally useful abstractions
> across all of the architectures we support.  I would like to make an
> "architecture portability" library, based on the kernel headers but as
> a separate project from the kernel, and intended for use in userspace.

I think that this is an _awesome_ idea.  Might want to check out what
overlap there is with existing glibc interfaces.  For example, I presume
that glibc implements at least some of the atomic operations (but I also
think having a full suite of atomic operations available is useful).

Some of the stuff, like semaphores, isn't really going to port very well
to user-space.  At least not directly, I would not think.

But on numerous occasions I have wanted the kernel's barriers, atomic
operations, bitwise operations, or some of the compiler things we
implement (likely, unlikely, fixes) in user-space.

> Now, clearly I can do this under the GPL.  However, I think it would
> be more useful to have the library under the LGPL, which requires
> either getting the permission of the authors of the kernel files, or
> rewriting them from scratch.

FWIW, you have my permission.  I've touched spinlock.h a bunch.

	Robert Love

Re: Proposal for a userspace "architecture portability" library

[Paul Mackerras]

Robert Love writes:

> I think that this is an _awesome_ idea.  Might want to check out what
> overlap there is with existing glibc interfaces.  For example, I presume
> that glibc implements at least some of the atomic operations (but I also
> think having a full suite of atomic operations available is useful).

I don't think glibc exports any atomic operations.  As for the
semaphores and spinlocks, clearly you can use the pthread_* functions,
but hopefully the kernel versions are a bit lighter-weight.

> Some of the stuff, like semaphores, isn't really going to port very well
> to user-space.  At least not directly, I would not think.

No, for semaphores and rwsems I was going to use futexes.  Or maybe we
don't need the kernel's semaphores, rwsems and spinlocks in userspace
at all.  I'm open to suggestions.

> FWIW, you have my permission.  I've touched spinlock.h a bunch.

Thanks.

Paul.

Re: Proposal for a userspace "architecture portability" library

[Kevin P. Fleming]

Paul Mackerras wrote:

> No, for semaphores and rwsems I was going to use futexes.  Or maybe we
> don't need the kernel's semaphores, rwsems and spinlocks in userspace
> at all.  I'm open to suggestions.

I think that _all_ of these items can find use in userspace, if they are 
usable with glibc's threading implementation and the other userspace 
issues that would be involved.

I for one would love to be able to use lightweight rwsems and spinlocks 
in an application I'm working on, but it's entirely userspace and it 
would be impossible for _me_ to pick out this code from the kernel tree 
and make it work on any architecture that I don't have here (which is 
all of them except x86 <G>).

Re: Proposal for a userspace "architecture portability" library

[H. Peter Anvin]

Followup to:  <16818.23575.549824.733470@cargo.ozlabs.ibm.com>
By author:    Paul Mackerras <paulus@samba.org>
In newsgroup: linux.dev.kernel
> 
> Now, clearly I can do this under the GPL.  However, I think it would
> be more useful to have the library under the LGPL, which requires
> either getting the permission of the authors of the kernel files, or
> rewriting them from scratch.
> 
> Linus (and other kernel copyright holders) - would you be willing to
> relicense such of the above files that have your copyright under the
> LGPL for this purpose?
> 
> I'm looking for volunteers to help with porting and testing on various
> architectures.  I can do x86, ppc and ppc64, and I know sparc{,64} and
> m68k assembler, but for the rest I'll need help.
> 

I can certainly pitch in.  I would personally prefer to see this under
the BSD license, (a) because it's not really that much code, and (b)
because it would let me add it to klibc.

	-hpa

Re: Proposal for a userspace "architecture portability" library

[Benjamin Herrenschmidt]

On Sun, 2004-12-05 at 11:53 +1100, Paul Mackerras wrote:

> I'm looking for volunteers to help with porting and testing on various
> architectures.  I can do x86, ppc and ppc64, and I know sparc{,64} and
> m68k assembler, but for the rest I'll need help.
> 
> My hope is that distributions will be able to use this to replace some
> of the headers in /usr/include/asm, and thus reduce the desire for
> applications to include kernel headers.

Interesting ... note also that it goes well with my intend of having
some of these (atomics, locks, ...) be provided by the kernel via the
vDSO library mapped by the kernel in userland on ppc. That library would
abstract that nicely. (That way, the kernel can take care of providing
the best implementation for a given processor, dealing with CPU errata
that often happen around areas of locks & atomics, etc...)

Ben.

Re: Proposal for a userspace "architecture portability" library

[Roland McGrath]

> I don't think glibc exports any atomic operations.

That is true.  But it does have implementations in bits/atomic.h for many
processors, and that is under the LGPL.

> As for the semaphores and spinlocks, clearly you can use the pthread_*
> functions, but hopefully the kernel versions are a bit lighter-weight.

The pthread_spin_* functions are not inlines, but are otherwise minimal.

Re: Proposal for a userspace "architecture portability" library

[Kevin P. Fleming]

Benjamin Herrenschmidt wrote:

> Interesting ... note also that it goes well with my intend of having
> some of these (atomics, locks, ...) be provided by the kernel via the
> vDSO library mapped by the kernel in userland on ppc. That library would
> abstract that nicely. (That way, the kernel can take care of providing
> the best implementation for a given processor, dealing with CPU errata
> that often happen around areas of locks & atomics, etc...)

Another thought... the Apache APR library already attempts to provide 
some of this functionality (atomic operations and locks, among others). 
This would fit nicely, as it would provide the underlying core for these 
operations, and allow APR to be extremely well optimized when built for 
a Linux platform.

Re: Proposal for a userspace "architecture portability" library

[Alan Cox]

On Sul, 2004-12-05 at 00:53, Paul Mackerras wrote:
> Now, clearly I can do this under the GPL.  However, I think it would
> be more useful to have the library under the LGPL, which requires
> either getting the permission of the authors of the kernel files, or
> rewriting them from scratch.

Why not use the mozilla ones ? Some of them are kernel ones reused with
permission and others are reimplementations but it has a fair set of
functions.

Re: Proposal for a userspace "architecture portability" library

[Joseph Seigh]

On Sun, 5 Dec 2004 11:53:43 +1100, Paul Mackerras <paulus@samba.org> wrote:

> Some of our kernel headers implement generally useful abstractions
> across all of the architectures we support.  I would like to make an
> "architecture portability" library, based on the kernel headers but as
> a separate project from the kernel, and intended for use in userspace.
>
> The headers that I want to base this on are:
>
> atomic.h
> bitops.h
> byteorder.h
> rwsem.h
> semaphore.h
> spinlock.h
> system.h (for mb et al., xchg, cmpxchg)
> unaligned.h
>

The atomics would be a good idea especially if they were cleaned up a  
little.
A list of functions that would be useful

atomic_set  -- atomically fetches a value
atomic_read -- atomically stores a value

These two exist but one should be renamed for consistancy, atomic_write or  
atomic_get.

fetch_and_add -- atomically add to an integer and return result.
fetch_and_increment -- atomically increment by one and return result
fetch_and_decrement -- atomically decrement by one and return result.

fetch_and_add could replace the numerous atomic_(add,sub,inc,dec)_xxxx  
which either don't
test for the right condition or don't test the result at all.   Also, get  
rid of atomic_t.  It doesn't
appear to be useful and is a pain to work around.  We probably need a way  
to define what
operands are atomic but that's not the way to do it.   The fetch and  
increment/decrement
aren't really needed but would allow optimization for some platforms.

Generic interlocked instructions such as compare and swap.  I prefer IBM  
style which returns
a sucessful/not successful w/ update of compare value over the Microsoft  
style which returns
the old value.   For hardware with LL/SC, compare and swap can be  
simulated.  Also
double wide compare and swap.  Not all platforms have it but it's  
imporatant enough to
provide even if you have to simulate it on the ones that don't.   Also an  
xchg/swap atomic
op would be a nice optimization for those platforms that have it.  You  
don't use it
a lot but when you do it's a nice optimization over simulating it using  
compare and swap.

Memory barriers also.  At a minimum, rmb(), wmb(), and mb().  If you're  
ambitious finer
granularity membars such as those on sparc.  You can use stronger membars  
on architectures
that don't have that granularity.   Also the dependent load barrier  
read_barrier_depends().
This would be used by many lock-free algorithms, not just RCU, which  
itself can have
multiple implementations for preemptive user threads.  I've done 3  
implementations of
RCU for preemtive user threads so far.

Read/write locks and semaphores already exist in userspace.  Spinlocks I  
personally
would avoid as they can have rather significant negative performance  
impact.  You can
usually get what you want with lock-free algorithms without the negative  
consequences
that spin-locks have.

These are fairly advanced interfaces and you really need to know what  
you're doing.
While defining these interfaces could be considered dangerous by enabling  
the the
unqualified to use them, not providing them won't stop them from doing  
dangerous
and stupid things which they'll do anyway.  With official headers, you  
could provide
man pages and header comments with appropiate warnings and pointers to  
faqs.
A common error is the incorrect impementation of DCL (double checked  
locking)
which has its own DangerWillRobisonDanger web page here
http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html
which has had little effect in dissuading people from using it.  If you  
had an official
load w/ read_barrier_depends() macro similar to RCU's, it would be hard  
for people
to ignore the fact of its existence when informed of such.  It'd be  
"official".

And these interfaces will help immensly those of us working with lock-free  
algorithms
who don't have the time or resources to implement yet another atomic  
operation on
every single platform.

Joe Seigh

Re: Proposal for a userspace "architecture portability" library

[Joel Becker]

On Sun, Dec 05, 2004 at 11:53:43AM +1100, Paul Mackerras wrote:
> Some of our kernel headers implement generally useful abstractions
> across all of the architectures we support.  I would like to make an
> "architecture portability" library, based on the kernel headers but as
> a separate project from the kernel, and intended for use in userspace.

	I'd love to see this also.  We've wrapped kernel-style bitops
from ext2 in
http://oss.oracle.com/projects/ocfs2-tools/src/trunk/libocfs2/include/bitops.h
(code is very similar to the libext2fs version).  We've also done the
kernel endian macros in userspace (glibc) in
http://oss.oracle.com/projects/ocfs2-tools/src/trunk/libocfs2/include/byteorder.h.
These might be useful (or not).  Also, I'd suggest list.h and
rbtree.[ch].  We use them extensively, as they are nice implementations
with little to no dependancies on anything else.
	Again, we've taken the approach of not including kernel headers,
instead making the kernel interface work via userspace methods.  I would
love to leave that work up to this library, and then just use it.

Joel

-- 

	Pitchers and catchers report.

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: Proposal for a userspace "architecture portability" library

[Adrian 'Dagurashibanipal' von Bidder]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

On Sunday 05 December 2004 01.53, Paul Mackerras wrote:
> Some of our kernel headers implement generally useful abstractions
> across all of the architectures we support.  I would like to make an
> "architecture portability" library, based on the kernel headers but as
> a separate project from the kernel, and intended for use in userspace.

Doesn't apr cover some of this already? Should such an effort be merged into 
apr?

Just a thought.
-- vbi

-- 
Beware of the FUD - know your enemies. This week
    * The Alexis de Toqueville Institue *
http://fortytwo.ch/opinion/adti

[-- Attachment #2: Type: application/pgp-signature, Size: 388 bytes --]

Re: Proposal for a userspace "architecture portability" library

[Dan Kegel]

Roland McGrath <roland () redhat ! com> wrote:
 >> I don't think glibc exports any atomic operations.
 >
 >That is true.  But it does have implementations in bits/atomic.h for many
 >processors, and that is under the LGPL.

Interesting.  This seems to be new as of glibc-2.3.3.
(glibc-2.3.2 had implementations of all sorts of things,
spinlocks even, but they were all internal.)

gcc's libstdc++ also exports an atomicity.h
(in e.g. /usr/include/c++/3.4.2/bits/atomicity.h).

gcc's libjava also has its own set of lock primitives
(buried in a file named locks.h).

It would be quite the engineering feat to demonstrate
a gcc/glibc toolchain actually using your proposed
portability layer and demonstrate zero loss of performance.
Even that might not be enough to convince the glibc
maintainer to use it...
- Dan

-- 
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html