How do _you_ read the linux source?

How do _you_ read the linux source?

[r00nk at simplecpu.com]

The following question gets asked a lot, "I know C, but reading the
kernel source is hard, what should I do?" and the common response is "ctags."
It's a lot like asking "how can I build a house?" and receiving the response
"screwdriver."
	There is obviously more to it then learning C and installing ctags.
As a newbie myself, I recently had to overcome this problem, Here's what I
did:

	First, I asked myself 'who understands the linux kernel?'. Linus was
the obvious answer, so I hunted down anything that he wrote. I found that he
had an autobiography, so I picked up a copy and read it. In it, I learned
about the foundational 'open, close, read, write, fork, exec' system calls.
	Second, I decided to try to write my own kernel. In writing it, I
learned how central the filesystem was to unix, I learned the significance of
interrupts, I wrote a few drivers, and I ran into (and sometimes got past) a
few nasty deadlocks.
	Third, I subscribed to the mailing lists and begun to read through them.
I got caught up on the way things are currently done, and I found and read
the
way linux approached the problems that I had already become familar with.

	Ok, lets take that and break it down;

	First; Find if someone has done this before. If they have, find out
how they did it.
	Second; If you can, try praticing it or solving said problem yourself.
	Third; Now that you have the foundations, figure out how its currently
being done.

	Now for the important part. It's an algorithmn. See:

	if(someone has done this before)
		figure out how they did it;
	if(we can try)
		try to solve it ourselves;
	figure out how it's currently being done;

	And that's how I usually solve a problem. _However_, I didn't come up
with that algorithm myself. I had a programmer friend who was very
skilled. He
always seemed to do what others thought to be impossible.  One day I asked
him
how he did it, and he said "well, I look if some one has done it before,
then I
try it, then I look at how people do it now." I imitated this, and it lead me
to become a much better programmer.
	My point is this, I learned how to do things by learning the thought
process of someone who could. I'm not saying that that example is the best
way,
or even a good one. I'm saying that learning the way others think and
approach
problems is key to success.

	The problem a lot of newbies are having is in 'separating the trunk
from the leaves.' So my question is this: Experienced kernel developers, how
do _you_ read source code? How do you separate the trunk from the leaves?
What do you do when you read code you're not familiar with? How do you learn?
What's your algorithm?

	I mean obviously it might change on the context, or you might have
more then one way. But still, any peek into the way you've learned to
approach problems is incredibly valuable.

	Anyways, thanks for your time.

How do _you_ read the linux source?

[nick]

On 2015-04-19 09:57 PM, r00nk at simplecpu.com wrote:
> The following question gets asked a lot, "I know C, but reading the
> kernel source is hard, what should I do?" and the common response is "ctags."
> It's a lot like asking "how can I build a house?" and receiving the response
> "screwdriver."
> 	There is obviously more to it then learning C and installing ctags.
> As a newbie myself, I recently had to overcome this problem, Here's what I
> did:
> 
> 	First, I asked myself 'who understands the linux kernel?'. Linus was
> the obvious answer, so I hunted down anything that he wrote. I found that he
> had an autobiography, so I picked up a copy and read it. In it, I learned
> about the foundational 'open, close, read, write, fork, exec' system calls.
> 	Second, I decided to try to write my own kernel. In writing it, I
> learned how central the filesystem was to unix, I learned the significance of
> interrupts, I wrote a few drivers, and I ran into (and sometimes got past) a
> few nasty deadlocks.
> 	Third, I subscribed to the mailing lists and begun to read through them.
> I got caught up on the way things are currently done, and I found and read
> the
> way linux approached the problems that I had already become familar with.
> 
> 	Ok, lets take that and break it down;
> 
> 	First; Find if someone has done this before. If they have, find out
> how they did it.
> 	Second; If you can, try praticing it or solving said problem yourself.
> 	Third; Now that you have the foundations, figure out how its currently
> being done.
> 
> 	Now for the important part. It's an algorithmn. See:
> 
> 	if(someone has done this before)
> 		figure out how they did it;
> 	if(we can try)
> 		try to solve it ourselves;
> 	figure out how it's currently being done;
> 
> 	And that's how I usually solve a problem. _However_, I didn't come up
> with that algorithm myself. I had a programmer friend who was very
> skilled. He
> always seemed to do what others thought to be impossible.  One day I asked
> him
> how he did it, and he said "well, I look if some one has done it before,
> then I
> try it, then I look at how people do it now." I imitated this, and it lead me
> to become a much better programmer.
> 	My point is this, I learned how to do things by learning the thought
> process of someone who could. I'm not saying that that example is the best
> way,
> or even a good one. I'm saying that learning the way others think and
> approach
> problems is key to success.
> 
> 	The problem a lot of newbies are having is in 'separating the trunk
> from the leaves.' So my question is this: Experienced kernel developers, how
> do _you_ read source code? How do you separate the trunk from the leaves?
> What do you do when you read code you're not familiar with? How do you learn?
> What's your algorithm?
> 
> 	I mean obviously it might change on the context, or you might have
> more then one way. But still, any peek into the way you've learned to
> approach problems is incredibly valuable.
> 
> 	Anyways, thanks for your time.
> 
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> 
There were a few things I did when starting to learn the kernel
1. Read Robert Love's Linux Kernel Development, I don't care how much you think
you known about the kernel read and trace the actual kernel code with this book!!
2.Read a book on device drivers and the Linux networking stack, I
read Linux networking internals for this
3. Build Linux from Scratch, this helps you understand how the system gets build for a distro
and how things get connected to the kernel/tool chain either directly or through the c library/
other libraries
4. Read Linux Virtual Memory Management as this helps with understanding the key areas of 
the virtual memory and memory management subsystems(somewhat outdated in terms of reporting
on newer kernel releases but the concepts are explained well)
5.Read The Linux Programming Interface to get used to the system calls/APIs for user space the kernel 
supports and how it is done(Chapter 5 of Linux Kernel Development also helps with this)
5. Read the Kernel wikis and documentation in the kernel tree related to areas your interested in
6. Find a area or a few your interested in and play around with them(working on this as of late)
7. Learn the best way for you to debug/trace bugs in the kernel, I learned the hard way this 
only comes with experience. This is not user space where bugs are much easier to trace,basically
understand your system calls,threading theory,how to use gdb and memory analysis toolkit,maybe
tool chain issues if working with embedded systems. The kernel is very complex in debugging everything
from deadlock to registers being written from being wrong on exact hardware can happen. Further more due
to the kernel being only a layer above hardware we very easily get bugs  that can only be reduplicated  
on exact hardware.(working on this too)
8. This a add on to 7,learn assembly for tracing bugs as this is very helpful for reading register
trace backs of oops/kernel panics if necessary for debugging.(another thing on my kernel learning list)
Hope this Helps,
Nick
 
 

How do _you_ read the linux source?

[Christoffer Holmstedt]

2015-04-20 5:11 GMT+02:00 nick <xerofoify@gmail.com>:
> There were a few things I did when starting to learn the kernel
> 1. Read Robert Love's Linux Kernel Development, I don't care how much you think
> you known about the kernel read and trace the actual kernel code with this book!!
> 2.Read a book on device drivers and the Linux networking stack, I
> read Linux networking internals for this

Thank you Nick, I'm just getting started with linux kernel development
and have been looking around for books both more general and specific
to networking. One thing in common several books I've found have is
that they are based on the 2.6 version of the kernel (or older). Some
parts have changed but are entire chapters in the above mentioned
books too old to make sense when working on version linux kernel 4.1
and beyond?

How do _you_ read the linux source?

[Robert P. J. Day]

On Sun, 19 Apr 2015, nick wrote:

> On 2015-04-19 09:57 PM, r00nk at simplecpu.com wrote:
> > The following question gets asked a lot, "I know C, but reading the
> > kernel source is hard, what should I do?" and the common response is "ctags."
> > It's a lot like asking "how can I build a house?" and receiving the response
> > "screwdriver."
> > 	There is obviously more to it then learning C and installing ctags.
> > As a newbie myself, I recently had to overcome this problem, Here's what I
> > did:

... snip ...

> > 	The problem a lot of newbies are having is in 'separating the trunk
> > from the leaves.' So my question is this: Experienced kernel developers, how
> > do _you_ read source code? How do you separate the trunk from the leaves?
> > What do you do when you read code you're not familiar with? How do you learn?
> > What's your algorithm?

  *sigh* ... this is the wrong question, in the same way that asking,
"how do i start writing kernel code?" is the wrong question. someone
once made the brilliant analogy that asking how to start contributing
to the kernel is akin to asking, "i want to write a book ... what
should i write about?" if you don't already know what interests you,
no one else can help you start writing.

  in the same way, asking "how do i read kernel code?" is akin to
walking into a library and asking, "i want to read something ... where
do i start?" if you don't already know what part of the kernel
interests you, then you should be asking yourself why you should be
reading kernel code in the first place.

  it seems that there is a regular supply of posters whose opening
question is something like, "i'm interested in the kernel ... how do i
get started?" i submit that it's not the mandate of this list to
answer that question. rather, i submit that it's the mandate of this
list to answer questions you have *after* you get started and have
shown that you've invested the time to dig in on your own.

  unless you've demonstrated the interest and ability to do at least a
*little* homework and research before asking questions, you're
probably not going to get much help here, and you might ask yourself
why you're interested in the first place.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

How do _you_ read the linux source?

[nick]

On 2015-04-20 01:45 AM, Christoffer Holmstedt wrote:
> 2015-04-20 5:11 GMT+02:00 nick <xerofoify@gmail.com>:
>> There were a few things I did when starting to learn the kernel
>> 1. Read Robert Love's Linux Kernel Development, I don't care how much you think
>> you known about the kernel read and trace the actual kernel code with this book!!
>> 2.Read a book on device drivers and the Linux networking stack, I
>> read Linux networking internals for this
> 
> Thank you Nick, I'm just getting started with linux kernel development
> and have been looking around for books both more general and specific
> to networking. One thing in common several books I've found have is
> that they are based on the 2.6 version of the kernel (or older). Some
> parts have changed but are entire chapters in the above mentioned
> books too old to make sense when working on version linux kernel 4.1
> and beyond?
> 
What you need to do is look at the source code and see what has changed this writing
of the book. This is what I do when reading kernel programming books or documentation.
Further more around way is to read the logs of the networking mailing lists for the
kernel to see what has changed recently.
Nick

How do _you_ read the linux source?

[Greg KH]

On Sun, Apr 19, 2015 at 06:57:49PM -0700, r00nk at simplecpu.com wrote:
> 	The problem a lot of newbies are having is in 'separating the trunk
> from the leaves.' So my question is this: Experienced kernel developers, how
> do _you_ read source code? How do you separate the trunk from the leaves?
> What do you do when you read code you're not familiar with? How do you learn?
> What's your algorithm?

I print out the source code, using a huge font as to take up lots of
space, bind it all together, and relax in a bathtub full of warm bubbles
and drink wine while reading the code and scribbling comments in the
margins with a colored soap shard.  After falling asleep due to the wine
and warm bath, wake up in a few hours freezing cold, surrounded by soggy
pieces of paper with all of the ink washed off.  But my brain has
absorbed it all and I can then resume coding from where I left off.

</sarcasm>

I use vgrep[1], that's it, no ctags needed.

Reality isn't romantic, there is no magic solution to hard tasks.

greg k-h

[1] https://github.com/vrothberg/vgrep

How do _you_ read the linux source?

[Valdis.Kletnieks at vt.edu]

On Mon, 20 Apr 2015 07:45:37 +0200, Christoffer Holmstedt said:

> Thank you Nick, I'm just getting started with linux kernel development
> and have been looking around for books both more general and specific
> to networking. One thing in common several books I've found have is
> that they are based on the 2.6 version of the kernel (or older). Some
> parts have changed but are entire chapters in the above mentioned
> books too old to make sense when working on version linux kernel 4.1
> and beyond?

The details have changed.  The basic concepts haven't.  In fact, the *basic*
concepts change so slowly that reading Bach's book on the SYSV kernel or
McKusic & Liefler on the BSD kernel will teach you a lot.

In fact, I'll go out on a limb and say that reading how *other* systems have
solved the same issues is a Good Thing. Knowing just "Linux solved it this way"
is one thing, but knowing "Linux solved it this way, but BSD took a different
approach because X, and IBM's VM/SP did Y instead to avoid the issue entirely"
is a whole different level of understanding.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20150420/fed99190/attachment.bin 

How do _you_ read the linux source?

[Jeff Haran]

> -----Original Message-----
> From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
> bounces at kernelnewbies.org] On Behalf Of r00nk at simplecpu.com
> Sent: Sunday, April 19, 2015 6:58 PM
> To: kernelnewbies at kernelnewbies.org
> Subject: How do _you_ read the linux source?
> 
> The following question gets asked a lot, "I know C, but reading the kernel
> source is hard, what should I do?" and the common response is "ctags."
> It's a lot like asking "how can I build a house?" and receiving the response
> "screwdriver."

I'll put in another plug for Coverity's source code browser here.

https://scan.coverity.com/

To use it you need to sign-up for an account, but that is free and it's much better for browsing large projects than ctags, cscope, grep, etc., IMHO.

Jeff Haran

How do _you_ read the linux source?

[Ruben Safir]

On 04/20/2015 12:32 PM, Jeff Haran wrote:
> I'll put in another plug for Coverity's source code browser here.
> 
> https://scan.coverity.com/
> 
> To use it you need to sign-up for an account, but that is free and it's much better for browsing large projects than ctags, cscope, grep, etc., IMHO.


if you want to learn nothing and sit behind a proprietary software wall
to access your free software.

How do _you_ read the linux source?

[Milton Krutt]

> On Sun, Apr 19, 2015 at 06:57:49PM -0700, r00nk at simplecpu.com wrote:
> > 	The problem a lot of newbies are having is in 'separating the trunk
> > from the leaves.' So my question is this: Experienced kernel developers, how
> > do _you_ read source code? How do you separate the trunk from the leaves?
> > What do you do when you read code you're not familiar with? How do you learn?
> > What's your algorithm?

Maybe it could help to firstly focus on data structures/types rather than functions;
and I would discourage to read code like a book, I mean from left to right and
from top to bottom. And, take a subsystem/part (even if it's very small) of interest
and just focus on it. For instance, I guess there is plenty of documentation on how
linux boots up: read it, and search through the source where what you have read is done.

compile your own kernel, if you haven't done it yet!

HTH

How do _you_ read the linux source?

[Ruben Safir]

On Tue, Apr 21, 2015 at 02:16:49AM +0200, Milton Krutt wrote:
> > On Sun, Apr 19, 2015 at 06:57:49PM -0700, r00nk at simplecpu.com wrote:
> > > 	The problem a lot of newbies are having is in 'separating the trunk
> > > from the leaves.' So my question is this: Experienced kernel developers, how
> > > do _you_ read source code? How do you separate the trunk from the leaves?
> > > What do you do when you read code you're not familiar with? How do you learn?
> > > What's your algorithm?
> 
> Maybe it could help to firstly focus on data structures/types rather than functions;
> and I would discourage to read code like a book, I mean from left to right and
> from top to bottom. And, take a subsystem/part (even if it's very small) of interest
> and just focus on it. For instance, I guess there is plenty of documentation on how
> linux boots up: read it, and search through the source where what you have read is done.
> 


Booting is so messed up that it might not be the place to start.  Asking
how to read the code is a non-sensible question.  Its like asking how do
you speak French.  The answer is, I speak it.  You don't know how to
speak it?  Well staring at incomrehesible code aint helping you any.

It requires one to interact with it, and that requires background.  

So the question is, I just do and if I can't understand it I roll up my
sleeves and research, ask, discuss, and work with it.

There is no forrest... there is no trees.  

> compile your own kernel, if you haven't done it yet!
> 
> HTH
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

-- 
So many immigrant groups have swept through our town
that Brooklyn, like Atlantis, reaches mythological
proportions in the mind of the world - RI Safir 1998
http://www.mrbrklyn.com 

DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002
http://www.nylxs.com - Leadership Development in Free Software
http://www2.mrbrklyn.com/resources - Unpublished Archive 
http://www.coinhangout.com - coins!
http://www.brooklyn-living.com 

Being so tracked is for FARM ANIMALS and and extermination camps, 
but incompatible with living as a free human being. -RI Safir 2013

How do _you_ read the linux source?

[r00nk at simplecpu.com]

> On Sun, 19 Apr 2015, nick wrote:
>
>> On 2015-04-19 09:57 PM, r00nk at simplecpu.com wrote:
>> > 	The problem a lot of newbies are having is in 'separating the trunk
>> > from the leaves.' So my question is this: Experienced kernel
>> developers, how
>> > do _you_ read source code? How do you separate the trunk from the
>> leaves?
>> > What do you do when you read code you're not familiar with? How do you
>> learn?
>> > What's your algorithm?
>
>   *sigh* ... this is the wrong question, in the same way that asking,
> "how do i start writing kernel code?" is the wrong question...

How to start isn't what I'm trying to ask. Like, if this were
mountain climbing, I'm not asking how to start climbing. I've
gotten to the point where I've developed my own technique to climb,
now I'm asking others how they climb, so I can learn/improve my own
technique based on theirs.

>   it seems that there is a regular supply of posters whose opening
> question is something like, "i'm interested in the kernel ... how do i
> get started?" i submit that it's not the mandate of this list to
> answer that question. rather, i submit that it's the mandate of this
> list to answer questions you have *after* you get started and have
> shown that you've invested the time to dig in on your own.

Right, I'm not asking how to get started, I'm asking how you (not I)
go about digging, to see if your method is better then mine. I have
already done a fair amount of digging myself (knock on wood).

I think that meta-cognition is a important part of any skill set.
I'm trying to learn the way skilled kernel developers think, because
learning how skilled people think generally leads into valuable
insights for that said skill.

In my (admittedly small) personal experience, trying to get an idea
of how kernel developers think is difficult. There are a few places
that sort of do it (Documentation/CodingStyle, Documentation/*, lkml),
but those are a bit more geared to management then skill.
I figured rather then trying to parse those and come up with a model
that probably isn't true, I'd just directly ask. I dunno. Maybe its
not an appropriate question for this mailing list.

How do _you_ read the linux source?

[Miles Fidelman]

Robert P. J. Day wrote:
> On Sun, 19 Apr 2015, nick wrote:
>
>> On 2015-04-19 09:57 PM, r00nk at simplecpu.com wrote:
>>> The following question gets asked a lot, "I know C, but reading the
>>> kernel source is hard, what should I do?" and the common response is "ctags."
>>> It's a lot like asking "how can I build a house?" and receiving the response
>>> "screwdriver."
>>> 	There is obviously more to it then learning C and installing ctags.
>>> As a newbie myself, I recently had to overcome this problem, Here's what I
>>> did:
> ... snip ...
>
>>> 	The problem a lot of newbies are having is in 'separating the trunk
>>> from the leaves.' So my question is this: Experienced kernel developers, how
>>> do _you_ read source code? How do you separate the trunk from the leaves?
>>> What do you do when you read code you're not familiar with? How do you learn?
>>> What's your algorithm?
>    *sigh* ... this is the wrong question, in the same way that asking,
> "how do i start writing kernel code?" is the wrong question. someone
> once made the brilliant analogy that asking how to start contributing
> to the kernel is akin to asking, "i want to write a book ... what
> should i write about?" if you don't already know what interests you,
> no one else can help you start writing.

<snip>

Maybe it's the wrong question, but it's sure stimulating a lot of good 
(as in informative) answers.

Miles Fidelman



-- 
In theory, there is no difference between theory and practice.
In practice, there is.   .... Yogi Berra

How do _you_ read the linux source?

[John de la Garza]

On Mon, Apr 20, 2015 at 05:46:57PM -0700, r00nk at simplecpu.com wrote:
> I figured rather then trying to parse those and come up with a model
> that probably isn't true, I'd just directly ask. I dunno. Maybe its
> not an appropriate question for this mailing list.

I appreciate the question and responses.