bzip2 support against 2.4.18

bzip2 support against 2.4.18

[Christian Ludwig]

Hey,

I think it's time to have bzip2 support in the kernel. I know the discussion
about the speed and memory issues that are around with this. But everything
in this patch is optional. You may use these new features if you want, you
do not have to use them...

This is a testing version of the patch. Only apply this if you really want
to play around with it a little bit. I know too less persons, who have the
time/fancy to test it (including myself). If you find errors in it feel free
to go on developing the patch yourself! Just CC a copy to me.
This patch consists actually of two parts:

1. A kernel bzip2 compression patch. The kernel will be compressed with
bzip. Therefore you have to type "make bz2bzImage" at the prompt after the
kernel configuration. This part is architecture dependent and was
implemented only for i386 based PCs right now.

2. A ramdisk bzip2 compression support patch. The ramdisk/initrd recongnises
now bzip2 compressed ramdisk images, loads and decompresses them. You can
choose between gzip and bzip2 (or even both) in the kernel configuration.

These two parts cannot be split up, because both are using the same
decompression code in "linux/lib/bzip2_inflate.c".

I have adapted this kernel hack by Tom Oehser (tom@toms.net). He wrote this
for kernel 2.2 and I ported it to 2.4.18 and cleaned up the code.

You will find the diff here:
    http://chrissicool.piranho.com/patch-2.4.x-bzip2-i386

Known bugs:
    - gzip crc support was corrupted in file "rd.c", function
"flush_window()"
      [maybe it can be fixed, but time is money...]
    - too less testing time was invested

Best regards,

    - Christian

bzip2 patent status query

[jbradford]

Is bzip2 *definitely* patent-unencumbered?

It claims to be on it's home page, but I found this from the OpenBSD people:

http://www.openbsd.org/2.8_packages/m68k/bzip-0.21.tgz-long.html

> I think it's time to have bzip2 support in the kernel. I know the discussion
> about the speed and memory issues that are around with this. But everything
> in this patch is optional. You may use these new features if you want, you
> do not have to use them...

Re: bzip2 patent status query

[Albert D. Cahalan]

jbradford@dial.pip writes:

> Is bzip2 *definitely* patent-unencumbered?

You could ask a lawyer and hope he's right...

> It claims to be on it's home page, but I found this from the OpenBSD people:
> 
> http://www.openbsd.org/2.8_packages/m68k/bzip-0.21.tgz-long.html

...but that's not bzip2. It's the original bzip, which used
an arithmetic encoding in the final stage. The bzip2 program
won't even read files created with bzip.

Re: bzip2 patent status query

[Alan Cox]

> Is bzip2 *definitely* patent-unencumbered?

Who knows. Move out of the USA if it worries you

> It claims to be on it's home page, but I found this from the OpenBSD people:
> http://www.openbsd.org/2.8_packages/m68k/bzip-0.21.tgz-long.html

bzip != bzip2. bzip is known to be not usable in the USSA

Re: bzip2 patent status query

[Mark Mielke]

On Wed, Jul 10, 2002 at 04:54:51PM +0100, jbradford@dial.pipex.com wrote:
> Is bzip2 *definitely* patent-unencumbered?
> It claims to be on it's home page, but I found this from the OpenBSD people:
> http://www.openbsd.org/2.8_packages/m68k/bzip-0.21.tgz-long.html

If you read a little further on the bzip pages, you would find that the
reason bzip2 and bzip are incompatible, is because bzip was found to be
patent-encumbered. The search for a better compression algorithm, that
wasn't covered by a patent began...

I actually remember when the original sources for the block sorter
compression program were posted to comp.sources.misc or somewhere
similar. The original impression most people had was 'this algorithm
is a hoax... it can't compress...' And so, I believe, nobody bothered
to patent it. :-)

mark

-- 
mark@mielke.cc/markm@ncf.ca/markm@nortelnetworks.com __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/

Re: bzip2 patent status query

[jbradford]

> ...but that's not bzip2. It's the original bzip, which used
> an arithmetic encoding in the final stage. The bzip2 program
> won't even read files created with bzip.

Right, I see...  Sorry about that waste of bandwith, I should have looked at it a bit more closely, I'll try to exit L-user mode before posting in future.

John.

Re: bzip2 patent status query

[Rik van Riel]

On Wed, 10 Jul 2002 jbradford@dial.pipex.com wrote:

> Is bzip2 *definitely* patent-unencumbered?

As long as bitflipping and xor are patented, I guess there
must be _something_ covering bzip2 ;)

Rik
-- 
Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Re: bzip2 support against 2.4.18

[Christian Ludwig]

I actually did not mesásured how much smaller bz2bzImages are definitely.
Want to say that I haven't made a complete battery of tests yet.

The only thing I found out empirically is that my boot+root floppy (2MB
ext2fs root, totally overcrowded of course ;( and with kernel/ramdisk dd'ed
straight on it without lilo) is about 1.42MB with the standard 2.4.18 kernel
alltogether. Using bz2bzImage and compressing the root image with bzip -9
the overall size went down to 1.3MB with exactly the same configuration.

I also have tested it with a 4MB RAM 486DX-100 PC, but this crashed after
loading the kernel, so the decompression failed.
Putting in another 4MB into that machine (thus it has 8MB now), made the
kernel boot, but ramdisk decompression failed. All in all you will need at
least 12MB to boot correctly, if you are using a bz2bzImage of about 700kB
and a 2MB compressed ramdisk image.
But I think nowadays ram shouldn't be that problem anymore, as log as we are
speaking of 12MB. For anybody who does not have that "much" ram the old
method is still available...

Have fun...

    - Christian

---------------------------------
Computers get faster every day.
Linux gets better every day.
I won't use Windows in no way.

Re: bzip2 support against 2.4.18

[Daniel Phillips]

On Thursday 11 July 2002 09:21, Christian Ludwig wrote:
> Putting in another 4MB into that machine (thus it has 8MB now), made the
> kernel boot, but ramdisk decompression failed. All in all you will need at
> least 12MB to boot correctly, if you are using a bz2bzImage of about 700kB
> and a 2MB compressed ramdisk image.

Good stuff, but why take this opportunity to make an ugly name even uglier?
How about bz2Image, or, more natural in my mind, bz2linux.

-- 
Daniel

Re: bzip2 support against 2.4.18

[Christian Ludwig]

On 11.07.2001 - 19:21 Daniel Phillips wrote:
> How about bz2Image, or, more natural in my mind, bz2linux.

bzImage stands for "big zipped Image". Zipped, in this case, means that it
is gzipped. Perhaps Linus never wants to support other compression formats
for the kernel.
Sure "bz2bzImage" is a bit ugly. I personally would prefer bzImage.bz2,
although it is some kind of self-extracting executable, thus *.bz2 is also
not correct. But it would imply better which sort of compression you are
using. But that also means that the standard kernel has to be called
"bzImage.gz". I did not want to mess up the standard names...

But the question is: who is responsible for all those naming conventions?
Does anyone has an idea?

Have fun,

    - Christian

Re: bzip2 patent status query

[Christian Ludwig]

Rik van Riel wrote on Wednesday, July 10:
> As long as bitflipping and xor are patented, I guess there
> must be _something_ covering bzip2 ;)

Perhaps it is so, even the author does not know it exactly (but why doesn't
even he know it?).
Bzip2 is included in nearly every distribution I know. BUT, if there would
be a patent on bzip2, I think there would have been somebody already who
sued at least all bigger distributions. If I were the one having a patent on
bzip2, or only parts of it, I would not have let the chance to get rich go
by.

Have fun.

    - Christian

Re: bzip2 support against 2.4.18

[Daniel Phillips]

On Friday 12 July 2002 08:36, Christian Ludwig wrote:
> On 11.07.2001 - 19:21 Daniel Phillips wrote:
> > How about bz2Image, or, more natural in my mind, bz2linux.
> 
> bzImage stands for "big zipped Image".

Yes, I know, and I also know that one can make many foolish arguments in
the name of consistency, but this doesn't change the fact that bz2bzImage
is one of the ugliest symbols we've yet seen, and much worse, it's one
that normal, unsuspecting users are going to come in contact with, if
this code gets into the kernel.

So I'm suggesting it's time to break with tradition try for something a
little less offensive to the eyes.

> Zipped, in this case, means that it
> is gzipped. Perhaps Linus never wants to support other compression formats
> for the kernel.
> Sure "bz2bzImage" is a bit ugly.

"A bit" doesn't begin to describe it.

> I personally would prefer bzImage.bz2,
> although it is some kind of self-extracting executable, thus *.bz2

Exactly, which is why I didn't suggest it.

> is also
> not correct. But it would imply better which sort of compression you are
> using. But that also means that the standard kernel has to be called
> "bzImage.gz". I did not want to mess up the standard names...

There is nothing standard about this name, it exists in exactly one place:
the Linux build process.

> But the question is: who is responsible for all those naming conventions?
> Does anyone has an idea?

You are, it's your patch.  And I've taken upon myself the responsibility
of heaving the decaying vegetables deserved by your first attempt.

Actually, what is the use of even including 'bz2' in the name?  Nobody
besides we geeks needs to know the thing is compressed with bzip2.  It
would be nice to see the word 'linux' in there.  How about bzlinux?
Just think of the hundreds of cases of carpal tunnel syndrome you'd
prevent by eliminating the shifted character.

-- 
Daniel

Re: bzip2 support against 2.4.18

[Christian Ludwig]

Daniel wrote on Friday, July 12, 2002:
> On Friday 12 July 2002 08:36, Christian Ludwig wrote:
> > But the question is: who is responsible for all those naming
conventions?
> > Does anyone has an idea?
>
> You are, it's your patch.  And I've taken upon myself the responsibility
> of heaving the decaying vegetables deserved by your first attempt.
>
> Actually, what is the use of even including 'bz2' in the name?  Nobody
> besides we geeks needs to know the thing is compressed with bzip2.  It
> would be nice to see the word 'linux' in there.  How about bzlinux?
> Just think of the hundreds of cases of carpal tunnel syndrome you'd
> prevent by eliminating the shifted character.

First, thanks for your help!
Surely it is better not to have a capital letter. My idea to have that 'bz2'
in the name was that you could also have some more kernel compression
algorithms some day. For all of these you would need new names. To make it
at least a little bit easier there should be that 'bz2' in the name. So
'bz2linux' would be a goal. But if we do this we also could change 'bzImage'
to 'gzlinux'.
On the other hand I also had the idea to let the name 'bzImage' be for both,
gzip and bzip2. The problem is that I can neither overload the name nor
choose the kernel compression at configuration time [I do not know how to
make it at least].

Have fun.

    - Christian

Re: bzip2 support against 2.4.18

[Daniel Phillips]

On Friday 12 July 2002 10:32, Christian Ludwig wrote:
> Daniel wrote on Friday, July 12, 2002:
> > On Friday 12 July 2002 08:36, Christian Ludwig wrote:
> > > But the question is: who is responsible for all those naming
> conventions?
> > > Does anyone has an idea?
> >
> > You are, it's your patch.  And I've taken upon myself the responsibility
> > of heaving the decaying vegetables deserved by your first attempt.
> >
> > Actually, what is the use of even including 'bz2' in the name?  Nobody
> > besides we geeks needs to know the thing is compressed with bzip2.  It
> > would be nice to see the word 'linux' in there.  How about bzlinux?
> > Just think of the hundreds of cases of carpal tunnel syndrome you'd
> > prevent by eliminating the shifted character.
> 
> First, thanks for your help!
> Surely it is better not to have a capital letter. My idea to have that 'bz2'
> in the name was that you could also have some more kernel compression
> algorithms some day. For all of these you would need new names.

Do you really?  Why?  Exactly what purpose does it serve to know how your
kernel was compressed, considering that it knows how to uncompress itself?

> To make it
> at least a little bit easier there should be that 'bz2' in the name. So
> 'bz2linux' would be a goal. But if we do this we also could change 'bzImage'
> to 'gzlinux'.

You can feel pretty confident in thinking the name bzImage is never going
to change, if only because too many fingers know how to type the stupid
thing by reflex action.

> On the other hand I also had the idea to let the name 'bzImage' be for both,
> gzip and bzip2. The problem is that I can neither overload the name nor
> choose the kernel compression at configuration time [I do not know how to
> make it at least].

Now that you mention it, bzImage should continue to serve perfectly well,
so long as you have some other way of configuring the kernel compression
method than via the make target.  Why not just make the compression method
a config option?  If it had been done this way from the beginning, we'd 
never have acquired the b or the z.

This way you avoid the entire controversy of chosing a new name for the
kernel image, and anyway, it's a nicer interface than via the make
target.

/me thinks for a moment about the idea of encoding every single config
option in the name of the name of the image file and shudders

-- 
Daniel

Re: bzip2 support against 2.4.18

[Christian Ludwig]

Daniel Phillips wrote on Friday, July 12, 2002 10:52 AM:

> On Friday 12 July 2002 10:32, Christian Ludwig wrote:
> > To make it
> > at least a little bit easier there should be that 'bz2' in the name. So
> > 'bz2linux' would be a goal. But if we do this we also could change
'bzImage'
> > to 'gzlinux'.
>
> You can feel pretty confident in thinking the name bzImage is never going
> to change, if only because too many fingers know how to type the stupid
> thing by reflex action.

Right.

> > On the other hand I also had the idea to let the name 'bzImage' be for
both,
> > gzip and bzip2. The problem is that I can neither overload the name nor
> > choose the kernel compression at configuration time [I do not know how
to
> > make it at least].
>
> Now that you mention it, bzImage should continue to serve perfectly well,
> so long as you have some other way of configuring the kernel compression
> method than via the make target.  Why not just make the compression method
> a config option?  If it had been done this way from the beginning, we'd
> never have acquired the b or the z.
>
> This way you avoid the entire controversy of chosing a new name for the
> kernel image, and anyway, it's a nicer interface than via the make
> target.

That came into my mind, too. Let's see what I can do about it...
It won't probably be ready before August, because I still have some exams.

Have fun.

    - Christian

Re: bzip2 support against 2.4.18

[jw schultz]

[sniped a back and forth about make bzimage bz2image
bz2bzimage etc.]
On Fri, Jul 12, 2002 at 12:12:18PM +0200, Christian Ludwig wrote:
> Daniel Phillips wrote on Friday, July 12, 2002 10:52 AM:
> 
> > Now that you mention it, bzImage should continue to serve perfectly well,
> > so long as you have some other way of configuring the kernel compression
> > method than via the make target.  Why not just make the compression method
> > a config option?  If it had been done this way from the beginning, we'd
> > never have acquired the b or the z.
> >
> > This way you avoid the entire controversy of chosing a new name for the
> > kernel image, and anyway, it's a nicer interface than via the make
> > target.
> 
> That came into my mind, too. Let's see what I can do about it...
> It won't probably be ready before August, because I still have some exams.

Ditto.  A config option is where this belongs.  The
filename stays the same.  This avoids the issue of
forgetting which compression you are using.  It gets saved
in the config and make install will cover it.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: bzip2 support against 2.4.18

[Thunder from the hill]

Hi,

On Fri, 12 Jul 2002, Christian Ludwig wrote:
> Sure "bz2bzImage" is a bit ugly. I personally would prefer bzImage.bz2,

What about bbz2image? b{z}Image is for g{z}ipped, and b{bz2}image ... 
well, you know.

							Regards,
							Thunder
-- 
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

Re: bzip2 support against 2.4.18

[Thunder from the hill]

Hi,

On Fri, 12 Jul 2002, Daniel Phillips wrote:
> Actually, what is the use of even including 'bz2' in the name?  Nobody
> besides we geeks needs to know the thing is compressed with bzip2.  It
> would be nice to see the word 'linux' in there.  How about bzlinux?
> Just think of the hundreds of cases of carpal tunnel syndrome you'd
> prevent by eliminating the shifted character.

What's your suggestion for e.g. sparc then? bzvmlinux? vmlinubz?

I would leave i386 at *Image and the rest of the world use vmlinuz. 
Whatever we call _that_ one.

							Regards,
							Thunder
-- 
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

Re: bzip2 support against 2.4.18

[jbradford]

> bzImage stands for "big zipped Image". Zipped, in this case, means that it
> Does anyone has an idea?

bbimage

I.E. "big bzip2ed image"

John.

Re: bzip2 support against 2.4.18

[Mark Mielke]

On Fri, Jul 12, 2002 at 08:36:41AM +0200, Christian Ludwig wrote:
> On 11.07.2001 - 19:21 Daniel Phillips wrote:
> > How about bz2Image, or, more natural in my mind, bz2linux.
> bzImage stands for "big zipped Image". Zipped, in this case, means that it
> is gzipped. Perhaps Linus never wants to support other compression formats
> for the kernel.

> Sure "bz2bzImage" is a bit ugly. I personally would prefer bzImage.bz2,
> although it is some kind of self-extracting executable, thus *.bz2 is also
> not correct. But it would imply better which sort of compression you are
> using. But that also means that the standard kernel has to be called
> "bzImage.gz". I did not want to mess up the standard names...

I would suggest keeping bzImage as the actual kernel name, and the
compression format to be a CONFIG parameter. This leaves all the
installation notes correct. As the executable is self-extracting,
there is no need for the type to be specified outside of the image.

> But the question is: who is responsible for all those naming conventions?
> Does anyone has an idea?

Not me... probably Linus... :-)

mark

-- 
mark@mielke.cc/markm@ncf.ca/markm@nortelnetworks.com __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/

Re: bzip2 support against 2.4.18

[Christian Ludwig]

Mark Mielke wrote on Friday, July 12, 2002 3:24 PM +0200:
> I would suggest keeping bzImage as the actual kernel name, and the
> compression format to be a CONFIG parameter. This leaves all the
> installation notes correct. As the executable is self-extracting,
> there is no need for the type to be specified outside of the image.

That's the point to have the discussion about a new name for *Image has to
end. I will code the kernel compression format as a CONFIG parameter in
future releases. I think that's the place where it belongs. So nobody has to
lern a new name and all are happy.

Over and out.

Have fun further on,

    - Christian

Re: bzip2 support against 2.4.18

[Tom Oehser]

> > How about bz2Image, or, more natural in my mind, bz2linux.

> Sure "bz2bzImage" is a bit ugly. I personally would prefer bzImage.bz2,

I chose the name bz2bzImage and have been using it on tomsrtbt since 2.0.0,
the reason I chose it was to make as clear as possible that it is still a
big/compressed image and that the bz2 is additional/different.  I get a lot
of confusion from users who assume that bzImage *already* has something to do
with bzip2.  I tried to convey that it was a "Bzip2-Big-Compressed-image"
rather than a "normal" "Big-Compressed-image".  I still prefer it to either
bz2Image or bzImage.? or bzip2Image.  But I don't really care.

Note, I have gotten it to work fine with a 4MB machine on 2.2.x, so 2.4.x
will probably work on 4MB also in some smaller kernel configurations.  Also,
the speed penalty was not problematic even on 486 machines.  See my post a
few months ago for details.  But, overall, it is fine on an 8MB 486, and I
think it is useful enough in embedded and floppy and flash environments that
it would be worthwhile.

Does your mod of my patch support configuring the normal vs. "small" option?
Also, does it support choosing the compression-level-number?  Does it support
using gzip/bzip2 one for the kernel and the other for the ramdisk in either
combination?  My original patch was only "small", disabled gzip, and I think
used "-9" compression for both the kernel and the ramdisk.

-Tom

Re: bzip2 support against 2.4.18

[Tom Oehser]

> Do you really?  Why?  Exactly what purpose does it serve to know how your
> kernel was compressed, considering that it knows how to uncompress itself?

I already use the name in scripts for tomsrtbt to decide whether the ramdisk
should be compressed with bzip2 or gzip, since the kernel compression method
(in my original patch) determines the required ramdisk compression.

-Tom

Re: bzip2 support against 2.4.18

[Mark Mielke]

On Fri, Jul 12, 2002 at 10:25:37AM -0400, Tom Oehser wrote:
> > Do you really?  Why?  Exactly what purpose does it serve to know how your
> > kernel was compressed, considering that it knows how to uncompress itself?
> I already use the name in scripts for tomsrtbt to decide whether the ramdisk
> should be compressed with bzip2 or gzip, since the kernel compression method
> (in my original patch) determines the required ramdisk compression.

This doesn't sound like the proper way to do this. Naming conventions are
notoriously inaccurate and limited, when it comes to detecting capabilities.

It would be far better to have a set of flags at the beginning of the image,
that detailed the capabilities. For example, what if the kernel supported
bzip2 or gzip initrd images? You would be unable to detect whether gzip
initrd images were supported in a bzip2 kernel.

The 'bzImage' name should only change, after an architectural decision
is made to simplify the single name. The 'bzImage' does not define how
the image is compressed, only that it _is_ compressed.

mark

-- 
mark@mielke.cc/markm@ncf.ca/markm@nortelnetworks.com __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/

Re: bzip2 support against 2.4.18

[Mike Touloumtzis]

On Thu, Jul 11, 2002 at 07:22:29PM +0200, Daniel Phillips wrote:
> On Thursday 11 July 2002 09:21, Christian Ludwig wrote:
> > Putting in another 4MB into that machine (thus it has 8MB now), made the
> > kernel boot, but ramdisk decompression failed. All in all you will need at
> > least 12MB to boot correctly, if you are using a bz2bzImage of about 700kB
> > and a 2MB compressed ramdisk image.
> 
> Good stuff, but why take this opportunity to make an ugly name even uglier?
> How about bz2Image, or, more natural in my mind, bz2linux.

Dare I say it... "linux.bz2"?  Why are Linux images so special?
After all, the first Unix kernel images were just called "unix".

cheerfully,
miket

Re: bzip2 support against 2.4.18

[john slee]

On Fri, Jul 12, 2002 at 09:30:29AM +0200, Daniel Phillips wrote:
> Actually, what is the use of even including 'bz2' in the name?  Nobody
> besides we geeks needs to know the thing is compressed with bzip2.  It
> would be nice to see the word 'linux' in there.  How about bzlinux?
> Just think of the hundreds of cases of carpal tunnel syndrome you'd
> prevent by eliminating the shifted character.

why not just call it 'linux'?  file(1) exists for a reason, and the 'vm'
prefix is a bit redundant these days

also i've never really understood why the binary format of the kernel is
selected via make.  why not just make it a regular kernel option with a
sane default?  surely your average kernel compiling person picks
something that works (zImage? bzImage? Image?) and sticks to it...

j.

-- 
toyota power: http://indigoid.net/

Re: bzip2 support against 2.4.18

[Daniel Phillips]

On Saturday 13 July 2002 15:56, john slee wrote:
> On Fri, Jul 12, 2002 at 09:30:29AM +0200, Daniel Phillips wrote:
> > Actually, what is the use of even including 'bz2' in the name?  Nobody
> > besides we geeks needs to know the thing is compressed with bzip2.  It
> > would be nice to see the word 'linux' in there.  How about bzlinux?
> > Just think of the hundreds of cases of carpal tunnel syndrome you'd
> > prevent by eliminating the shifted character.
> 
> why not just call it 'linux'?  file(1) exists for a reason, and the 'vm'
> prefix is a bit redundant these days
> 
> also i've never really understood why the binary format of the kernel is
> selected via make.  why not just make it a regular kernel option with a
> sane default?  surely your average kernel compiling person picks
> something that works (zImage? bzImage? Image?) and sticks to it...

Unix geeks like tradition and they like oddball names that can only be
explained by knowing history; this creates a kind of lore to be passed 
down from senior to junior programmers, a kind of bonding.

This gets increasingly silly as time goes by.  However, it's not yet
gotten so silly that 'bzImage' is likely to be replaced by 'linux' any
time soon.  At least, we don't have to make it any worse and if you
read the whole thread you'll see the consensus is strongly in favor
of pushing the desired image format into the config.

Note that Jeff Dike sensibly called his make target 'linux'.

-- 
Daniel

Re: bzip2 support against 2.4.18

[Thunder from the hill]

Hi,

On Sat, 13 Jul 2002, john slee wrote:
> why not just call it 'linux'?  file(1) exists for a reason, and the 'vm'
> prefix is a bit redundant these days

I used to type make vmunix for years, vmlinux is just fine for me. Same 
probably with lots of lots of other people.

							Regards,
							Thunder
-- 
(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

Re: bzip2 support against 2.4.18

[Tomas Szepe]

> This gets increasingly silly as time goes by.  However, it's not yet
> gotten so silly that 'bzImage' is likely to be replaced by 'linux' any
> time soon.  At least, we don't have to make it any worse and if you
> read the whole thread you'll see the consensus is strongly in favor
> of pushing the desired image format into the config.

... which is how things work w/ kbuild 2.5.

T.

Re: bzip2 support against 2.4.18

[Christian Ludwig]

Tom Oehser wrote on Friday, July 12, 2002 4:21 PM:
> I chose the name bz2bzImage and have been using it on tomsrtbt since
2.0.0,
> the reason I chose it was to make as clear as possible that it is still a
> big/compressed image and that the bz2 is additional/different.  I get a
lot
> of confusion from users who assume that bzImage *already* has something to
do
> with bzip2.  I tried to convey that it was a "Bzip2-Big-Compressed-image"
> rather than a "normal" "Big-Compressed-image".  I still prefer it to
either
> bz2Image or bzImage.? or bzip2Image.  But I don't really care.

Fine, I am just working on a new version of the patch, where the make target
'bz2bzImage' disappears. The kernel compression will be a CONFIG option,
too. This is possible, because in the original patch you only changed
'misc.c' to support bzip2. I will release the new version of the patch
within this week (I hope).

The ramdisk compression is already a CONFIG option. You can choose between
gzip/bzip2 ramdisk compression, or you can even select both.

> Note, I have gotten it to work fine with a 4MB machine on 2.2.x, so 2.4.x
> will probably work on 4MB also in some smaller kernel configurations.
Also,
> the speed penalty was not problematic even on 486 machines.  See my post a
> few months ago for details.  But, overall, it is fine on an 8MB 486, and I
> think it is useful enough in embedded and floppy and flash environments
that
> it would be worthwhile.

Kernel 2.4 is much bigger than kernel 2.2. With very small configurations
this should work on 4MB machines as well. You have to try. My kernel was
580kB bzip2 compressed, this one didn't work on 4MB.

> Does your mod of my patch support configuring the normal vs. "small"
option?
> Also, does it support choosing the compression-level-number?  Does it
support
> using gzip/bzip2 one for the kernel and the other for the ramdisk in
either
> combination?  My original patch was only "small", disabled gzip, and I
think
> used "-9" compression for both the kernel and the ramdisk.

Choosing a compression level is (still) not supported, but choosing any
combination of gzip/bzip2 is already implemented. I wonder if it is useful
to have compression-level-numbers? The patch uses '-9' compression in any
case (gzip/bzip2 kernel/ramdisk).

Have fun.

    - Christian

Re: bzip2 support against 2.4.18

[Tom Oehser]

>>>Does your mod of my patch support configuring the normal
>>>vs. "small" option?  Also, does it support choosing the
>>>compression-level-number?  Does it support using gzip/bzip2 one
>>>for the kernel and the other for the ramdisk in either combination?
>>>My original patch was only "small", disabled gzip, and I think used
>>>"-9" compression for both the kernel and the ramdisk.

> Choosing a compression level is (still) not supported, but choosing any
> combination of gzip/bzip2 is already implemented. I wonder if it is useful
> to have compression-level-numbers? The patch uses '-9' compression in any
> case (gzip/bzip2 kernel/ramdisk).

It is definitely useful to be able to control the compression
number, as it has drastic effects on decompression memory
requirements.  Especially between, say, -6 and -9, there is a
lot of memory usage increase without a lotta more compression,
controlling this may let your kernel decompress on a 4MB machine
it might not otherwise work on.  Consider:

                  Compress   Decompress   Decompress   Corpus
           Flag     usage      usage       -s usage     Size

            -1      1200k       500k         350k      914704
            -2      2000k       900k         600k      877703
            -3      2800k      1300k         850k      860338
            -4      3600k      1700k        1100k      846899
            -5      4400k      2100k        1350k      845160
            -6      5200k      2500k        1600k      838626
            -7      6100k      2900k        1850k      834096
            -8      6800k      3300k        2100k      828642
            -9      7600k      3700k        2350k      828642

Leaving it at -s only is probably OK, also, you would need major rework
of the patch to support the fast option, as I removed all that code
completely, it has alternate routines for much of bzip2, but supporting
control of the compression level will be really easy, since it only
affects the actual bzip2ing in the Makefile.

-Tom

Re: bzip2 support against 2.4.18

[Horst von Brand]

"Christian Ludwig" <cl81@gmx.net> said:

[...]

> Surely it is better not to have a capital letter. My idea to have that 'bz2'
> in the name was that you could also have some more kernel compression
> algorithms some day. For all of these you would need new names. To make it
> at least a little bit easier there should be that 'bz2' in the name. So
> 'bz2linux' would be a goal. But if we do this we also could change 'bzImage'
> to 'gzlinux'.

What for? The kernel is compressed and unzips itself on boot, which exact
compression mechanism is used is completely irrelevant to the booter, so it
has no place in the name.

Also, bzip2 is not used because it needs around 1MiB for buffers when
uncompressing, RAM which just isn't there when booting (it has to work in
the mythical PC 640KiB, IIRC). Or am I missing something here?
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

[PATCH] bzip2 compression for kernel 2.4 and ramdisk

[Christian Ludwig]

Hi,

I have released the new version 1.4 of the bzip2 support patch.
You can find it at http://chrissicool.piranho.com/linux (sorry for the ads.)

This patch consists actually of two independant parts, which do _not_ belong
together in any case. The only reason why they cannot be split up is that
both are using the same decompression code for bzip2 at the same location.
The two parts are:

1. A kernel bzip2 compression support patch.
The kernel will be compressed with bzip2, if you choose the appropriate
option in the "General options" menu of the kernel configuration. Choosing
gzip compression is still possible. You can also choose the compression
level in nine steps, from very poor compression (level 1), which is not very
memory and speed intensive. A very strong compression (level 9) makes the
bzImage smaller, but uses a large amount of RAM for decompression and takes
longer. This part is architecture dependent and was implemented for i386
based PCs.

2. A ramdisk bzip2 compression support patch.
You can enable gzip or bzip2 compression (or even both) for the ramdisk in
the kernel configuration. The ramdisk driver will test the image, which
should be loaded. If it recognises a valid (and supported) ramdisk image, it
will load and decompress it. The ramdisk compression is optional. You even
can turn off compressed ramdisk support at all.

If you find bugs, please mail me.

Have fun.

    - Christian Ludwig

Re: bzip2 support against 2.4.18

[Tom Oehser]

> Also, bzip2 is not used because it needs around 1MiB for buffers when
> uncompressing, RAM which just isn't there when booting (it has to work in
> the mythical PC 640KiB, IIRC). Or am I missing something here?

The reason bzip2 has not been thought desirable is as much the slowerness
as it is the biggerness, not something to do with the 640, after all, the
difference betwixt zImage and bzImage already is that bzImage loads high.

Note, it does not require "around 1MiB", it requires 350K if -1 -s is used,
and 3700K if -9 is used.  Reasonable use would be, say, 1600K for -6 -s, it
could certainly make a kernel that would boot in 4MB into one that requires
8MB, but, in many situations, that just isn't an issue, for example, if you
are going to be loading ramdisks, anyway, or an X server.  -Tom