From: Kyle Moffett <mrmacman_g4@mac.com>
To: Avi Kivity <avi@argo.co.il>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>, linux-kernel@vger.kernel.org
Subject: Re: Compiling C++ modules
Date: Tue, 25 Apr 2006 15:22:02 -0400 [thread overview]
Message-ID: <4C4500F3-3A8E-4992-82FD-6E16257676CC@mac.com> (raw)
In-Reply-To: <444E524A.10906@argo.co.il>
On Apr 25, 2006, at 12:46:02, Avi Kivity wrote:
> Kyle Moffett wrote:
>> Except making exceptions work in the kernel is exceptionally
>> nontrivial (sorry about the pun).
>
> My experience with exceptions in kernel-like code (a distributed
> filesystem) was excellent.
Well from all of the discussions about it that occurred on this list,
the biggest problem with exceptions in the kernel was the overhead to
keep track of the exception state and the try/catch stack. The other
problem was handling exceptions in atomic contexts, in the guts of
the scheduler, and in a host of other hot-paths.
>> Which of the following shows the flow of code better?
>
> Once you accept the idea that an exception can occur (almost) anywhere
Except they can't. Lots and lots of bits of kernel code explicitly
depend on the fact that certain operations _cannot_ fail, and they
make that obvious through the fact that those functions don't have
any way of returning error conditions.
> the C++ code shows you what the code does in the normal case
> without obfuscating it with error handling. Pretend that after
> every semicolon there is a comment of the form:
> /* possible exceptional return */
As I pointed out before and seem doomed to point out again, this is
an _implicit_ piece of code, and in kernels implicit code is very
very bad. Because of the varying contexts and close hardware
dependence, you want to explicitly state everything that happens.
>> First of all, that extra TakeLock object chews up stack, at least
>> 4 or 8 bytes of it, depending on your word size.
>
> No, it's optimized out. gcc notices that &lock doesn't change and
> that 'l' never escapes the function.
GCC does not notice that when you use out-of-line functions. Let me
remind you that many of the kernel's spinlocks and other functions
are out-of-line, inlining them has significant performance penalties.
>> Secondly with standard integer error returns you have one or two
>> easily-predictable assembly instructions at each step of the way,
>> whereas with exceptions you trade the absence of error handling in
>> the rest of the code for a lot of extra instructions at the
>> exception throw and catch points.
>
> The extra code is out of line (not even an if (unlikely())). So
> yes, total code grows, but the exceptional paths can be in
> a .text.exception section and not consume cache or TLB space.
Total bull. Let me give you an example. The following C++ code:
{
Foo my_foo;
function_that_throws_exception();
}
Is turned by the compiler into a slight variant on this C code:
{
Foo my_foo = Foo();
jmp_buf exception;
if (setjmp(exception))
goto out;
function_that_throws_exception();
out:
~Foo(&my_foo);
}
There is nothing about the above that the compiler can trivially "out-
of-line", not to mention the fact that you just royally screwed the
CPUs chances of getting branch prediction right. There's the fact
that setjmp and longjmp are kind of hard in kernelspace. Finally,
the stack usage is significantly increased, I think even in
userspace, jmp_buf occupies 8 longs (32 bytes) just for critical
registers and instruction pointer of the exception handler. Also,
the simplest case of throwing an exception has turned from the three
instructions "cmp bnz ret" (compare result or pointer, branch to
exception code, return) to this mess:
{
longjmp((int)pointer);
}
The longjmp function is simple, but not _that_ simple, and it still
breaks branch prediction in a bad way.
>> This is a really _really_ bad idea for a kernel. Having simple
>> declaration statements have big side effects (like the common
>> TakeLock object example I gave above) is bound to lead to people
>> screwing up and forgetting about the side effects. In C it's
>> impossible to miss the side effects of a statement; function calls
>> are obvious, as is global memory modification.
>
> In C++ you just have to treat declarations as executable
> statements. Just as you can't compile the code with a C compiler,
> you can't read it with a C mindset. Once you get used to it, it
> isn't surprising at all.
That's all well and good, until you assume that "some_type foo = 3;"
is just declaring an integer through a typedef instead of declaring
an object with side effects. The thing about the linux kernel is
that basically _nobody_ understands all of it, and as a result each
and every bit of code must stand on its own and be fairly obvious as
to side effects and such. In C++, most of the language features are
designed to _hide_ those side effects and as a result it's a terrible
fit for the kernel.
>> Let me point out _again_ how unobvious and fragile the flow of
>> code there is. Not to mention the fact that the C++ compiler can
>> easily notice that item1 and item2 are never used and optimize
>> them out entirely.
>
> Excellent! If there are no side effects, I want it out. If there
> are side effects, it won't optimize them out.
How can it tell? You aren't writing all your member functions
inline, are you?
>> You also totally missed the "int flags" argument you're supposed
>> to pass to object specifying allocation parameters,
>
> There is no allocation here (both the C and the C++ code allocate
> on the stack.
Let's look in the kernel. How often do you find non-trivial
constructor functions that _don't_ allocate memory, hm?
> Should you want to allocate from the heap, try this:
>
> {
> spinlock_t::guard g(some_lock);
> auto_ptr<Foo> item(new (gfp_mask) Foo); /* or pass a
> kmem_cache_t */
> item->do_something();
> item->do_something_else();
> return item.release();
> }
I think this code speaks for itself about its lack of readability.
> if ((r = foo_do_something(item))) {
Your kernel-idiomatic C is terrible. Please don't go around writing
much kernel code in this style, that's disgusting. Your multiply-
duplicated return statements were also bad form, see my example for a
much clearer way of doing it.
>> Yeah, sure, yours is 3 lines when you omit the following:
>> (1) Handling allocation flags like GFP_KERNEL
> done
And unreadable afterwards
>> (4) Reference counting, garbage collection, or another way to
>> selectively free the allocated objects based on success or failure
>> of other code.
> Reference counting is ridiculously to do in C++. I'll spare you the
> details.
I'll assume you mean "ridiculously easy" there. The problem is that
with the exception handling system you add refcounts to a lot of
objects that don't need them. Here's an example that doesn't need a
refcount. I'm registering the "item" with a subsystem, and if that
fails the object is immediately freed.
{
int result;
struct foo *item = kmalloc(sizeof(*item), GFP_KERNEL);
if (unlikely(!item))
return ERR_PTR(-ENOMEM);
spin_lock(&item_lock);
result = item_init(item, GFP_KERNEL);
if (unlikely(result))
goto free;
result = item_register(subsystem, item);
if (unlikely(result))
goto destroy;
out:
spin_unlock(&item_lock);
return result;
/* Error handling */
destroy:
item_destroy(item);
free:
kfree(item);
goto out;
}
Please note that the assembly into which this optimizes is quite
efficient, the compiler can trivially order the fast-path and place
all of the exception code after the function or could theoretically
even put it in a different section. The side-effects are quite
obvious, and it's also simple to identify exactly how many
instructions this function costs; there's a memory allocation, a
lock, and two function calls. Interspersed in there are exactly 3
conditional jumps to exception-handling code. Direct stack usage for
this function is around 8 or 12 bytes, depending on word size.
The biggest advantage to this method is that we can tell C exactly
what exceptions we expect from the various functions. Not only that,
but we already know what type they are, how likely they are to occur,
and exactly how to handle each kind of exception path. It's even
fairly easy to read through the fast-path based on how it's laid out.
Let me point out a few problems with the C++ ways you've described of
doing the same things:
(1) You can't easily allocate and initialize an object in 2
different steps. In the kernel you want to be able to sleep in
kmalloc to increase chances of getting the memory you need, but you
may need to take a spinlock before actually initializing the data
structure (say it's on a linked list). If you split up the actual
initialization into another function then you lose all of the
advantages of C++ constructors.
(2) Your code either adds a refcount for "item" or unconditionally
releases it at the end of the function. Yes that's fixable, but not
in a way that preserves the exception-handling properties you're
espousing so much. When you get an exception, how does the code tell
which objects to free and which ones not to? (Answer: it can't,
that's a semantic decision made by the programmer with "if" statements).
(3) You still haven't explained how adding all sorts of implicit
side effects is a good thing in an operating system kernel.
Cheers,
Kyle Moffett
next prev parent reply other threads:[~2006-04-25 19:22 UTC|newest]
Thread overview: 200+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-04-24 19:16 Compiling C++ modules Gary Poppitz
2006-04-24 19:27 ` Greg KH
2006-04-24 20:02 ` C++ pushback Gary Poppitz
2006-04-24 20:15 ` Christoph Hellwig
2006-04-24 20:16 ` Greg KH
2006-04-24 20:18 ` Martin Mares
2006-04-24 21:36 ` Jeff V. Merkey
2006-04-24 21:28 ` J.A. Magallon
2006-04-24 21:43 ` Harald Arnesen
2006-04-24 21:52 ` Alan Cox
2006-04-24 22:16 ` J.A. Magallon
2006-04-25 0:05 ` Harald Arnesen
2006-04-25 0:46 ` Diego Calleja
2006-04-25 9:12 ` Harald Arnesen
2006-04-25 1:30 ` linux-os (Dick Johnson)
2006-04-25 2:58 ` marty fouts
2006-04-27 22:55 ` Bill Davidsen
2006-05-02 15:58 ` Randy.Dunlap
2006-05-02 20:36 ` David Schwartz
2006-04-25 8:15 ` Xavier Bestel
2006-04-25 8:42 ` Avi Kivity
2006-04-25 8:52 ` Martin Mares
2006-04-25 9:00 ` Avi Kivity
2006-04-25 9:05 ` Martin Mares
2006-04-25 9:13 ` Avi Kivity
2006-04-25 9:22 ` Xavier Bestel
2006-04-25 20:20 ` J.A. Magallon
2006-04-25 20:31 ` Barry Kelly
2006-04-25 9:09 ` Nikita Danilov
2006-04-25 20:10 ` J.A. Magallon
2006-04-25 18:02 ` Geert Uytterhoeven
2006-04-27 9:09 ` Alexander E. Patrakov
2006-04-24 22:39 ` Willy Tarreau
2006-04-24 22:57 ` Jeff V. Merkey
2006-04-24 23:02 ` David Schwartz
2006-04-25 8:55 ` Martin Mares
2006-04-25 8:59 ` Jan Engelhardt
2006-04-25 14:37 ` David Schwartz
2006-04-25 19:50 ` Martin Mares
2006-04-26 2:33 ` David Schwartz
2006-04-26 3:42 ` Matthew Frost
2006-04-26 19:25 ` David Schwartz
2006-04-26 20:01 ` Jan-Benedict Glaw
2006-04-26 20:09 ` Linus Torvalds
2006-04-26 20:19 ` Al Viro
2006-04-26 21:37 ` Sam Ravnborg
2006-04-28 9:23 ` Avi Kivity
2006-04-28 12:00 ` linux-os (Dick Johnson)
2006-04-28 12:46 ` Jan-Benedict Glaw
2006-04-26 20:25 ` Jan-Benedict Glaw
2006-04-26 20:43 ` David Schwartz
2006-04-26 23:00 ` Roman Kononov
2006-04-27 0:38 ` Kyle Moffett
2006-04-27 2:05 ` Roman Kononov
2006-04-27 3:37 ` Kyle Moffett
2006-04-27 5:37 ` Roman Kononov
2006-04-27 13:58 ` Michael Buesch
2006-04-27 14:22 ` linux-os (Dick Johnson)
2006-04-27 8:07 ` Avi Kivity
2006-04-27 13:55 ` Denis Vlasenko
2006-04-27 14:27 ` Avi Kivity
2006-04-27 14:56 ` Denis Vlasenko
2006-04-27 15:54 ` Bob Copeland
2006-04-27 16:03 ` Avi Kivity
2006-04-27 15:00 ` Martin Mares
2006-04-27 15:31 ` Avi Kivity
2006-04-27 15:38 ` Martin Mares
2006-04-28 8:16 ` Avi Kivity
2006-04-28 8:30 ` Avi Kivity
2006-04-28 15:47 ` Jan Engelhardt
2006-04-28 15:51 ` Jan Engelhardt
2006-04-28 16:51 ` Avi Kivity
2006-04-27 14:50 ` Sam Ravnborg
2006-04-27 8:50 ` Martin Mares
2006-04-27 3:57 ` Willy Tarreau
2006-04-27 5:53 ` Roman Kononov
2006-04-27 7:55 ` Jan-Benedict Glaw
2006-04-27 17:20 ` C++ pushback (when does this religious thread end?) Leonard Peterson
2006-04-30 17:48 ` C++ pushback Jan Harkes
2006-04-30 20:55 ` David Schwartz
2006-04-26 20:05 ` linux-os (Dick Johnson)
2006-04-26 20:09 ` Xavier Bestel
2006-04-26 20:44 ` Randy.Dunlap
2006-05-02 20:09 ` C++ pushback + sparse Randy.Dunlap
2006-04-27 7:49 ` C++ pushback Jiri Kosina
2006-04-26 21:05 ` Martin Mares
2006-04-25 7:33 ` Avi Kivity
2006-04-25 7:47 ` Nick Piggin
2006-05-13 16:21 ` Esben Nielsen
2006-04-24 20:36 ` Thiago Galesi
2006-04-24 21:38 ` Kurt Wall
2006-04-27 16:17 ` Roman Kononov
2006-04-27 21:59 ` Grant Coady
2006-04-27 22:09 ` Bill Davidsen
2006-04-27 23:19 ` Jan Knutar
2006-04-24 19:30 ` Compiling C++ modules Al Viro
2006-04-24 19:40 ` linux-os (Dick Johnson)
2006-04-24 20:54 ` Geert Uytterhoeven
2006-04-24 19:42 ` Alexey Dobriyan
2006-04-24 20:30 ` Daniel Barkalow
2006-04-24 20:35 ` C++ is in US [Re: Compiling C++ modules] Jiri Slaby
2006-04-24 20:45 ` Compiling C++ modules Alan Cox
2006-04-24 21:03 ` Avi Kivity
2006-04-24 21:23 ` Joshua Hudson
2006-04-24 21:29 ` Kyle Moffett
2006-04-24 21:50 ` marty fouts
2006-04-24 22:09 ` Martin Mares
2006-04-24 22:30 ` Willy Tarreau
2006-04-24 22:32 ` Joshua Hudson
2006-04-24 22:45 ` marty fouts
2006-04-25 15:32 ` Michael Buesch
2006-04-25 7:08 ` Avi Kivity
2006-04-25 10:23 ` James Courtier-Dutton
2006-04-25 15:59 ` Kyle Moffett
2006-04-25 16:46 ` Avi Kivity
2006-04-25 17:10 ` Dmitry Torokhov
2006-04-25 17:19 ` Avi Kivity
2006-04-25 17:28 ` Dmitry Torokhov
2006-04-25 17:53 ` Avi Kivity
2006-04-25 18:04 ` Dmitry Torokhov
2006-04-25 18:08 ` Valdis.Kletnieks
2006-04-25 18:26 ` Avi Kivity
2006-04-25 18:38 ` Avi Kivity
2006-04-25 18:52 ` Michael Poole
2006-04-25 19:13 ` Avi Kivity
2006-04-27 15:10 ` Denis Vlasenko
2006-04-27 20:15 ` Willy Tarreau
2006-04-27 21:08 ` Davi Arnaut
2006-04-28 9:33 ` Avi Kivity
2006-04-28 10:03 ` Avi Kivity
2006-04-28 11:27 ` Sergei Organov
2006-04-28 11:03 ` Martin Mares
2006-04-28 11:30 ` Avi Kivity
2006-04-28 15:56 ` Jan Engelhardt
2006-04-28 17:02 ` Avi Kivity
2006-04-28 17:38 ` linux-os (Dick Johnson)
2006-04-29 2:50 ` Christer Weinigel
2006-05-01 17:46 ` Dave Neuer
2006-05-01 20:21 ` Jan Engelhardt
2006-05-01 23:53 ` David Schwartz
2006-05-02 5:12 ` Willy Tarreau
2006-05-02 10:32 ` Avi Kivity
2006-05-02 11:15 ` Martin Mares
2006-05-02 11:26 ` Avi Kivity
2006-05-02 11:40 ` linux-os (Dick Johnson)
2006-05-02 12:42 ` David Woodhouse
2006-05-02 16:27 ` Christer Weinigel
2006-05-02 12:48 ` Martin Mares
2006-05-02 13:52 ` Avi Kivity
2006-05-02 14:13 ` Al Viro
2006-05-02 14:54 ` Avi Kivity
2006-05-02 16:16 ` Brian Beattie
2006-05-02 16:21 ` Avi Kivity
2006-05-02 13:21 ` Willy Tarreau
2006-05-02 14:41 ` Avi Kivity
2006-05-02 22:25 ` Diego Calleja
2006-05-02 13:34 ` Al Viro
2006-05-02 14:02 ` Avi Kivity
2006-05-02 14:34 ` Al Viro
2006-05-02 15:04 ` Avi Kivity
2006-05-02 15:15 ` Al Viro
2006-05-02 15:19 ` Avi Kivity
2006-05-02 15:27 ` Kyle Moffett
2006-05-02 15:30 ` Avi Kivity
2006-05-02 15:28 ` Al Viro
2006-05-02 15:51 ` Avi Kivity
2006-05-02 15:24 ` Kyle Moffett
2006-05-03 13:13 ` Mark Lord
2006-05-03 20:51 ` David Schwartz
2006-04-30 21:15 ` Eric W. Biederman
2006-04-25 17:54 ` linux-os (Dick Johnson)
2006-04-26 8:30 ` Jan Engelhardt
2006-04-26 11:36 ` linux-os (Dick Johnson)
2006-04-25 19:22 ` Kyle Moffett [this message]
2006-04-25 19:54 ` Michael Buesch
2006-04-25 20:24 ` Avi Kivity
2006-04-25 20:11 ` Bongani Hlope
2006-04-25 20:26 ` Avi Kivity
2006-04-25 21:02 ` Valdis.Kletnieks
2006-04-25 21:15 ` Avi Kivity
[not found] ` <71a0d6ff0604251646g4fc90b3dr30a03b8606360e7f@mail.gmail.com>
2006-04-26 4:39 ` Avi Kivity
2006-04-25 17:55 ` Geert Uytterhoeven
2006-04-24 21:58 ` Alan Cox
2006-04-25 7:20 ` Avi Kivity
2006-04-25 9:06 ` Matt Keenan
2006-04-25 20:29 ` Bongani Hlope
2006-04-25 20:37 ` Avi Kivity
2006-04-25 21:08 ` Bongani Hlope
2006-04-25 4:17 ` Martin J. Bligh
2006-04-25 5:30 ` Avi Kivity
2006-04-25 8:58 ` Sam Ravnborg
2006-04-25 7:56 ` Jakob Oestergaard
2006-04-25 9:03 ` Jan Engelhardt
2006-04-24 21:36 ` J.A. Magallon
[not found] <65eLE-GJ-21@gated-at.bofh.it>
[not found] ` <65zwH-61W-51@gated-at.bofh.it>
[not found] ` <65zZH-6Bw-23@gated-at.bofh.it>
[not found] ` <66grR-2DK-27@gated-at.bofh.it>
2006-04-28 0:03 ` Robert Hancock
2006-04-28 9:37 Khushil Dep
2006-05-02 18:21 Al Boldi
2006-05-02 20:28 ` J.A. Magallon
2006-05-02 23:55 ` Peter Williams
2006-05-03 8:09 ` Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4C4500F3-3A8E-4992-82FD-6E16257676CC@mac.com \
--to=mrmacman_g4@mac.com \
--cc=alan@lxorguk.ukuu.org.uk \
--cc=avi@argo.co.il \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.