[QUESTION] which kernel debugger is "best"?

[QUESTION] which kernel debugger is "best"?

[Jeremy Jackson]

What are people using?  neither kdb or kgdb appear to support
2.5.7 (kdb does 2.5.5)... or do real men debug with prink() ?

Re: [QUESTION] which kernel debugger is "best"?

[Andrew Morton]

Jeremy Jackson wrote:
> 
> What are people using?

kgdb.  Tried kdb and (sorry, Keith), it's not in the same
league.  Not by miles.

>  neither kdb or kgdb appear to support
> 2.5.7 (kdb does 2.5.5)...

General answer to this is to go for a foray in
http://www.zip.com.au/~akpm/linux/patches/

Which turns up
	http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.7/kgdb.patch

> or do real men debug with prink() ?

I have done it both ways, extensively, for long periods.
The printk method is comically inefficient.  The amount
of transparency whch kgdb gives to kernel internals is
extraordinary.

-

Re: [QUESTION] which kernel debugger is "best"?

[Keith Owens]

On Fri, 29 Mar 2002 19:18:39 -0800, 
Andrew Morton <akpm@zip.com.au> wrote:
>Jeremy Jackson wrote:
>> 
>> What are people using?
>
>kgdb.  Tried kdb and (sorry, Keith), it's not in the same
>league.  Not by miles.

kdb and kgdb are aimed at different debugging environments.  kgdb
requires a second machine containing the kernel compiled with -g, kdb
lets you debug directly on the machine that failed, with or without
compiling with -g.  Almost all the differences flow from that design
decision.

Another important niggle to me is that kgdb requires the kernel to be
compiled with frame pointers, because that is all that gdb understands.
On ix86 the extra register pressure from dedicating ebp to frame
pointers can cause Heisenbugs.  kdb works with and without frame
pointers.

Can kgdb handle the special hard wired calls that do not add frame
pointers, such as __down_failed?  I doubt that gdb knows how to handle
those.

I am not knocking kgdb, it has its place.  I see a spectrum of
debugging tools from UML through kgdb to kdb, each tool is aimed at a
different debugging environment.  Pick the right tool.

Re: [QUESTION] which kernel debugger is "best"?

[Keith Owens]

On Fri, 29 Mar 2002 18:38:50 -0800, 
"Jeremy Jackson" <jerj@coplanar.net> wrote:
>What are people using?  neither kdb or kgdb appear to support
>2.5.7 (kdb does 2.5.5)... or do real men debug with prink() ?

I just uploaded kdb patches for 2.5.7 to
ftp://oss.sgi.com/projects/kdb/download/v2.1.  They compile but have not
been booted, I don't have much time to work on 2.5 kernels.  I have no
idea if it will work with a preemptible kernel or not.

Re: [QUESTION] which kernel debugger is "best"?

[Andrew Morton]

Keith Owens wrote:
> 
> On Fri, 29 Mar 2002 19:18:39 -0800,
> Andrew Morton <akpm@zip.com.au> wrote:
> >Jeremy Jackson wrote:
> >>
> >> What are people using?
> >
> >kgdb.  Tried kdb and (sorry, Keith), it's not in the same
> >league.  Not by miles.
> 
> <good points deleted>
> ..
> Pick the right tool.

I guess the distinction here is that I use kgdb for "development",
not for "debugging".

Displaying data structures, values of variables.  Seeing what
state all tasks in the system are in, where they're sleeping,
where they're spending CPU, etc.

When adding ad-hoc inxtrumentation to the kernel, you don't
need to bother printing it out - just increment the counters
and go in take a look when desired.

And yes, kgdb mucks up call chains across down() because of the
lack of a frame pointer - backtraces don't display who called
down() - it loses the innermost frame.  That's irritating,
but not enough to have motivated me to soil my hands with
x86 assembly yet.

I haven't had any problems with -fno-omit-frame-pointer at
any time.

I *have* had problems with -fno-inline.  I'd very much like
to be able to turn that on, but the presence of `extern inline'
functions causes a link failure with `-fno-inline'.  I'd suggest
that this is a gcc shortcoming.  I actually had a poke yesterday
at teaching gcc to convert extern inline to static inline if 
flag_no_inline, but it didn't work out.

kgdb is damned inconvenient.  You have to set up a cross-build
machine, serial cable and generally get organised to use it.
In reality, this would take an hour or so but it is some friction.

I would like to see kdb shipped in the mainline kernel, so that
we can get better diagnostic reports from users/testers.


-

Re: [QUESTION] which kernel debugger is "best"?

[David S. Miller]

   From: Andrew Morton <akpm@zip.com.au>
   Date: Fri, 29 Mar 2002 20:24:05 -0800
   
   I *have* had problems with -fno-inline.  I'd very much like
   to be able to turn that on, but the presence of `extern inline'
   functions causes a link failure with `-fno-inline'.

Feel free to submit the patch that converts the remaining extern
inline into static inline.  That is the correct solution.

GCC has every right not to inline and expect the function name to be
referencable externally if you say extern inline, so this is another
reason to fix the remaining extern inline instances.

Re: [QUESTION] which kernel debugger is "best"?

[Peter Wächtler]

Andrew Morton wrote:
> 
> I would like to see kdb shipped in the mainline kernel, so that
> we can get better diagnostic reports from users/testers.
> 

Whoops, I think the same. And also something like a crash dump
utility would be nice in the mainline kernels.

Without them it's hard to get qualified bug reports from
production machines...

Re: [QUESTION] which kernel debugger is "best"?

[Amit S. Kale]

Hi,

It's time we have a webpage comparing the two debuggers.
I put-up one at http://kgdb.sourceforge.net/whichdebugger.html

Please let me know if I have missed anything.

Peter Wächtler wrote:
> 
> Andrew Morton wrote:
> >
> > I would like to see kdb shipped in the mainline kernel, so that
> > we can get better diagnostic reports from users/testers.
> >
> 
> Whoops, I think the same. And also something like a crash dump
> utility would be nice in the mainline kernels.
> 
> Without them it's hard to get qualified bug reports from
> production machines...
-- 
Amit Kale
Veritas Software India ( http://www.veritasindia.com/ )

Re: [QUESTION] which kernel debugger is "best"?

[Philip R. Auld]

Andrew Morton wrote:
> 

[other stuff cut]

> 
> I would like to see kdb shipped in the mainline kernel, so that
> we can get better diagnostic reports from users/testers.
> 

Here, Here! That and crash dumps are important to providing 
high quality support to enterprise customers.



Phil



-- 
Philip R. Auld, Ph.D.                  Technical Staff 
Egenera, Inc.                        pauld@egenera.com
165 Forest St., Marlboro, MA 01752       (508)786-9444