will be able to load new kernel without restarting?

will be able to load new kernel without restarting?

[Gabe Foobar]

Hi!

Just a simple question. When I will be able to load new
kernel without restarting the system? Working anybody on
this problem?

If answer is "yes", the question is: when?
If answer is "no", the question is: why not?

Thank you.

gabe

Re: will be able to load new kernel without restarting?

[Jurriaan]

From: Gabe Foobar <foobar.gabe@freemail.hu>
Date: Sat, May 03, 2003 at 09:24:22PM +0200
> Hi!
> 
> Just a simple question. When I will be able to load new
> kernel without restarting the system? Working anybody on
> this problem?
> 
> If answer is "yes", the question is: when?

Check the archives for 'kexec'. Some success messages were posted even
today.

HTH,
Jurriaan
-- 
I am Grammy of Borg. And the assimilation goes to...
Debian (Unstable) GNU/Linux 2.5.68-bk9 3940 bogomips load av: 0.06 0.15 0.09

Re: will be able to load new kernel without restarting?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]

On Sat, 03 May 2003 22:56:56 +0200, Jurriaan said:

> > Just a simple question. When I will be able to load new
> > kernel without restarting the system? Working anybody on
> > this problem?

> Check the archives for 'kexec'. Some success messages were posted even
> today.
> 

As I understand it, that's still restarting, just skipping the usual detour
through the BIOS and lilo/grub/whatever.

What he wants is the sort of on-the-fly upgrading that bellheads have grown to
know and love, and which is NOT likely to be implemented for the entire Linux
kernel anytime soon.  Large sections can do it now with the "module" stuff, so
you can rmmod the old one and insmod the new one.. but I don't see any easy way
to implement rmmmod/insmod semantics for things like kernel/schedule.c (how
would you get your next timeslice?).  There's also issues with changing the
API for something - it's hard to drop a 2.5.71 kernel/signals.c into a 2.5.70
kernel if the API is different.  Googling around will probably cough up
lots of references to how the telcos do software upgrades - it basically
involves LOTS of up-front design work to make sure all the details are
addressed in the basic design of the system.

Bottom line - you can do it now for things that can be built as modules,
*if* it's something you can effectively rmmod and insmod.  If it's not a module,
or if it's the driver for something you can't rmmod (a disk or network driver,
etc), you can't do it on-the-fly.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: will be able to load new kernel without restarting?

[Eric W. Biederman]

Valdis.Kletnieks@vt.edu writes:

> On Sat, 03 May 2003 22:56:56 +0200, Jurriaan said:
> 
> > > Just a simple question. When I will be able to load new
> > > kernel without restarting the system? Working anybody on
> > > this problem?
> 
> > Check the archives for 'kexec'. Some success messages were posted even
> > today.
> > 
> 
> As I understand it, that's still restarting, just skipping the usual detour
> through the BIOS and lilo/grub/whatever.
> 
> What he wants is the sort of on-the-fly upgrading that bellheads have grown to
> know and love, and which is NOT likely to be implemented for the entire Linux
> kernel anytime soon.  Large sections can do it now with the "module" stuff, so
> you can rmmod the old one and insmod the new one.. but I don't see any easy way
> to implement rmmmod/insmod semantics for things like kernel/schedule.c (how
> would you get your next timeslice?).  There's also issues with changing the
> API for something - it's hard to drop a 2.5.71 kernel/signals.c into a 2.5.70
> kernel if the API is different.  Googling around will probably cough up
> lots of references to how the telcos do software upgrades - it basically
> involves LOTS of up-front design work to make sure all the details are
> addressed in the basic design of the system.
> 
> Bottom line - you can do it now for things that can be built as modules,
> *if* it's something you can effectively rmmod and insmod.  If it's not a module,
> 
> or if it's the driver for something you can't rmmod (a disk or network driver,
> etc), you can't do it on-the-fly.

If you can checkpoint user space you can use kexec to load the new kernel.
So at this point we are approaching half way there.   I don't know
where all of the work is for checkpointing but I do know there is a lot of interest
in it, and several partial implementations.

When replacing the entire kernel at least you have a stable ABI which
makes a number of things at least theoretically easier.

Eric

Re: will be able to load new kernel without restarting?

[rmoser]

You silly boys.  Thinking everything is impossible.  The old kernel
is just a shadow of the new one... just a shadow............  I'll try
spitting out a basic sketch.  This I'm sure you've done but if not,
fix it.  I'm an idiot, so this won't work as-is.

For starters, freeze the system.  halt EVERYTHING.

Now load the new kernel.

Now you have this thing in RAM.  Fine.  It's not running but it's in
RAM.

Start feeding data over to it.  In a manner it can understand.  You know,
make all the modules work with a standard named stream (check the
Advanced Tracker planning file at the very end:
http://advancedtracker.sourceforge.net/at/__at.plan.txt
I did copy it below, so you don't have to go there).  This is the
hard part--Module compatibility through defined names.  This would
bloat the kernel, so think about it and maybe you'll figure out a good
way.

Good.  Ask the new kernel if it has understood completely the data.
If yes, then jump to its resume routine.  This will make it unload the
old running kernel.  Free the old kernel, move yourself where you want
to be in RAM.  Now start up IPC again, and BAM!  Running new kernel
switched on-the-fly.

That last step is CRITICAL!  If the kernel--every single part--does NOT
understand EVERY piece of data sent to it, then stop.  Free the new
kernel.  Thank it kindly for its time.  Blarg to the user about loading an
incompatible kernel.  If the kernel acutally HAD the on-the-fly switching
support, then you can actually tell where it failed.  If not, say it's an old
kernel which does not understand OTFS.  Either way, turn everything
back on. Keep running.  No problem.

Please note you may need a standard, non-changing jump table in the
same exact place in RAM for all of the kernel syscalls, so that running
applicatioins don't b0rk out when the addresses of the syscalls change.

--Bluefox Icy

=======================
What is a named stream?
=======================
  A named stream is a named stream.  It separates out segments with clear names
to pass varying data.  Unknown names are usually discarded.  It looks like
this:

TYPE   SIZE    DESC
string    n    A string that tells the name of the next data.  Null terminated.
DWORD     4    32 bit length of the following data.  If 0, there is no data.
stream    x    A stream of bytes as long as the preceding dword gave.

*********** REPLY SEPARATOR  ***********

On 5/3/2003 at 6:52 PM Valdis.Kletnieks@vt.edu wrote:

>On Sat, 03 May 2003 22:56:56 +0200, Jurriaan said:
>
>> > Just a simple question. When I will be able to load new
>> > kernel without restarting the system? Working anybody on
>> > this problem?
>
>> Check the archives for 'kexec'. Some success messages were posted even
>> today.
>>
>
>As I understand it, that's still restarting, just skipping the usual detour
>through the BIOS and lilo/grub/whatever.
>
>What he wants is the sort of on-the-fly upgrading that bellheads have
>grown to
>know and love, and which is NOT likely to be implemented for the entire
>Linux
>kernel anytime soon.  Large sections can do it now with the "module"
>stuff, so
>you can rmmod the old one and insmod the new one.. but I don't see any
>easy way
>to implement rmmmod/insmod semantics for things like kernel/schedule.c (how
>would you get your next timeslice?).  There's also issues with changing the
>API for something - it's hard to drop a 2.5.71 kernel/signals.c into a
>2.5.70
>kernel if the API is different.  Googling around will probably cough up
>lots of references to how the telcos do software upgrades - it basically
>involves LOTS of up-front design work to make sure all the details are
>addressed in the basic design of the system.
>
>Bottom line - you can do it now for things that can be built as modules,
>*if* it's something you can effectively rmmod and insmod.  If it's not a
>module,
>or if it's the driver for something you can't rmmod (a disk or network
>driver,
>etc), you can't do it on-the-fly.
>
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.2.1 (GNU/Linux)
>Comment: Exmh version 2.5 07/13/2001
>
>iD8DBQE+tEgWcC3lWbTT17ARAipeAKDXI7/xjQaVzg9HQW7+pQpJRHkH8wCg0mAF
>p+dGwFcpwR1X8ayWfGQ738I=
>=CHbq
>-----END PGP SIGNATURE-----
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/

Re: will be able to load new kernel without restarting?

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2378 bytes --]

On Sun, 2003-05-04 12:00:00 -0400, rmoser <mlmoser@comcast.net>
wrote in message <200305041200000380.00B553E1@smtp.comcast.net>:
> You silly boys.  Thinking everything is impossible.  The old kernel
> is just a shadow of the new one... just a shadow............  I'll try
> spitting out a basic sketch.  This I'm sure you've done but if not,
> fix it.  I'm an idiot, so this won't work as-is.
> 
> For starters, freeze the system.  halt EVERYTHING.
> 
> Now load the new kernel.
> 
> Now you have this thing in RAM.  Fine.  It's not running but it's in
> RAM.
> 
> Start feeding data over to it.  In a manner it can understand.  You know,
> make all the modules work with a standard named stream (check the
> Advanced Tracker planning file at the very end:
> http://advancedtracker.sourceforge.net/at/__at.plan.txt
> I did copy it below, so you don't have to go there).  This is the
> hard part--Module compatibility through defined names.  This would
> bloat the kernel, so think about it and maybe you'll figure out a good
> way.

Sounds like being a nice feature (esp. for 24/7 uptime), but it also
sounds like a *lot* of bloat. You'll have to encode (and to decode) any
single bit if data. But - where to put all that? See, it you're having
some Gigabytes of ram, you probably end with > 100MB only describing all
those single pages (4..8K) of RAM. Encode that into a named stream - you
end up with > 1GB of data. Then, you also need to encode the whole state
of any drivers.

What do you do if drivers/mm/you-name-it changed and now uses different
data structures? In fact, any two choosen kernels may potentially use
different data structures. So for the newly-to-load kernel, you need
stream decoders for all possible older kernels. Bloat again.

So if I would need something like that, I think kexec is the way to go,
possibly with a second machine for failover (to even close the kexec
booting gap).

However - if you like this feature so much you'd die for: nobody stops
you to implement a proof-of-concept:)

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: will be able to load new kernel without restarting?

[Felipe Alfaro Solana]

On Sat, 2003-05-03 at 21:24, Gabe Foobar wrote:
> Hi!
> 
> Just a simple question. When I will be able to load new
> kernel without restarting the system? Working anybody on
> this problem?

There's an experimental Linux kernel patch called "kexec" that allows
you to load another kernel (not only a Linux kernel) witout rebooting in
the sense of what this term really means: e.g. you can load another
Linux kernel without doing a full POST using the BIOS.
-- 
Please AVOID sending me WORD, EXCEL or POWERPOINT attachments.
See http://www.fsf.org/philosophy/no-word-attachments.html
Linux Registered User #287198

Re: will be able to load new kernel without restarting?

[rmoser]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 3935 bytes --]



*********** REPLY SEPARATOR  ***********

On 5/4/2003 at 7:09 PM Jan-Benedict Glaw wrote:

>On Sun, 2003-05-04 12:00:00 -0400, rmoser <mlmoser@comcast.net>
>wrote in message <200305041200000380.00B553E1@smtp.comcast.net>:
>> You silly boys.  Thinking everything is impossible.  The old kernel
>> is just a shadow of the new one... just a shadow............  I'll try
>> spitting out a basic sketch.  This I'm sure you've done but if not,
>> fix it.  I'm an idiot, so this won't work as-is.
>> 
>> For starters, freeze the system.  halt EVERYTHING.
>> 
>> Now load the new kernel.
>> 
>> Now you have this thing in RAM.  Fine.  It's not running but it's in
>> RAM.
>> 
>> Start feeding data over to it.  In a manner it can understand.  You know,
>> make all the modules work with a standard named stream (check the
>> Advanced Tracker planning file at the very end:
>> http://advancedtracker.sourceforge.net/at/__at.plan.txt
>> I did copy it below, so you don't have to go there).  This is the
>> hard part--Module compatibility through defined names.  This would
>> bloat the kernel, so think about it and maybe you'll figure out a good
>> way.
>
>Sounds like being a nice feature (esp. for 24/7 uptime), but it also
>sounds like a *lot* of bloat. You'll have to encode (and to decode) any
>single bit if data. But - where to put all that? See, it you're having
>some Gigabytes of ram, you probably end with > 100MB only describing all
>those single pages (4..8K) of RAM. Encode that into a named stream - you
>end up with > 1GB of data. Then, you also need to encode the whole state
>of any drivers.
>
>What do you do if drivers/mm/you-name-it changed and now uses different
>data structures? In fact, any two choosen kernels may potentially use
>different data structures. So for the newly-to-load kernel, you need
>stream decoders for all possible older kernels. Bloat again.
>

Bloat yes.  But the idea is you keep the transit data structures the same.
It's doable.  It's not easy, but the modules, even with changing datastructures,
could understand what the stream means.  The data structures in the stream
need to be identified by module and function.  Then whatever a new module
has discarded, it discards.  Whatever it needs that's not there, it figures out
somehow.  If it can't, it tries to unload, and if it can unload, it notes all this.

When you're all done, ask the modules which work, which don't, and such.
See if it can go really.  If it can, do it.  If it can but some modules need to
reload, which aren't in use, reload them.  Feh, if some modules can't correlate
see if you can use the old kernel version!

>So if I would need something like that, I think kexec is the way to go,
>possibly with a second machine for failover (to even close the kexec
>booting gap).
>

Isn't that boot-without-bios?

>However - if you like this feature so much you'd die for: nobody stops
>you to implement a proof-of-concept:)
>

Do I look like I can code?  I'm a normal user--I write C, C++, but not
kernel-C.  And I can learn.  But if I crash something.  No.  I have one
machine, not something I can play with.  Well I can.  But I don't want
to blow it up.

>MfG, JBG
>
>-- 
>   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
>   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen
>Krieg
>    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
>      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));
>
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.2.1 (GNU/Linux)
>
>iD8DBQE+tUlbHb1edYOZ4bsRAmTEAJ9nSPZCVtsbAVRogpBBFUCruvZaVACfb12C
>vZp7TCxxq7XbyYgeCGY8aK4=
>=+rM+
>-----END PGP SIGNATURE-----
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/

Re: will be able to load new kernel without restarting?

[Timothy Miller]

Valdis.Kletnieks@vt.edu wrote:
  but I don't see any easy way
> to implement rmmmod/insmod semantics for things like kernel/schedule.c (how
> would you get your next timeslice?). 

This isn't such a silly idea, really.

How much code would be required for a default, completely brainless 
scheduler?  That's built-in.  When you load a new scheduler, the default 
one doesn't get kicked out; it just gets turned off.  When you unload, 
the default takes over.

The only major issue is that the data structures used to manage 
processes would be different from one scheduler to the next.  One 
possible answer would be to have an unloading driver translate all of 
its process information into the default scheduler's format.  A newly 
loaded one would translate it to its own format.  Things that would be 
lost in the translation include interactivity information, etc.

Please forgive my total ignorance about how processes are represented in 
data structures in the linux kernel.

Re: will be able to load new kernel without restarting?

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

On Mon, 05 May 2003 14:49:15 EDT, Timothy Miller said:

> The only major issue is that the data structures used to manage 
> processes would be different from one scheduler to the next.  One 
> possible answer would be to have an unloading driver translate all of 
> its process information into the default scheduler's format.  A newly 
> loaded one would translate it to its own format.  Things that would be 
> lost in the translation include interactivity information, etc.

We just had a similar battle regarding LSM extended attributes on files,
the biggest problem being what a module should do if it doesn't understand
the formats and semantics of the previous module...

It's certainly *doable* - the telcos have been doing this since whichever
of the SSn systems was first programmable rather than hardwire logic.  The
big question is whether it will still look like Linux if you implement it.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]