Re: Deprecating .gz format on kernel.org

Re: Deprecating .gz format on kernel.org

[Andrzej Krzysztofowicz]

> From: Pavel Machek <pavel@ucw.cz>
> To: Pascal Schmidt <der.eremit@email.de>
> Cc: linux-kernel@vger.kernel.org
> 
> > > So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?
> > My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.
> 
> I've seen 7(or so) on i386sx/25. Alan seen <1.

I've seen ~1 bm on i386sx/16 with add-on ISA memory card (150 ns chips).
But it no longer works :(

Andrzej

-- 
=======================================================================
  Andrzej M. Krzysztofowicz               ankry@mif.pg.gda.pl
  phone (48)(58) 347 14 61
Faculty of Applied Phys. & Math.,   Gdansk University of Technology

Re: Deprecating .gz format on kernel.org

[Alan Cox]

On Fri, 2003-03-21 at 11:28, Andrzej Krzysztofowicz wrote:
> > From: Pavel Machek <pavel@ucw.cz>
> > To: Pascal Schmidt <der.eremit@email.de>
> > Cc: linux-kernel@vger.kernel.org
> > 
> > > > So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?
> > > My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.
> > 
> > I've seen 7(or so) on i386sx/25. Alan seen <1.
> 
> I've seen ~1 bm on i386sx/16 with add-on ISA memory card (150 ns chips).
> But it no longer works :(
> 

An original IBM 4.77MHz PC reports 0.7 bogomips running Linux 8086, but
can still run a webserver!

Re: Deprecating .gz format on kernel.org

[H. Peter Anvin]

Followup to:  <1048261179.5718.34.camel@irongate.swansea.linux.org.uk>
By author:    Alan Cox <alan@lxorguk.ukuu.org.uk>
In newsgroup: linux.dev.kernel
> 
> An original IBM 4.77MHz PC reports 0.7 bogomips running Linux 8086, but
> can still run a webserver!
> 

I miss my 0.59 bogomips 386/16 (no cache, memory on the ISA bus.)
That was a truly braindamaged piece of engineering, and held the
bogomips antirecord for a long time until ELKS came along and ruined
it all :(

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: Deprecating .gz format on kernel.org

[Jörn Engel]

On Tue, 25 March 2003 10:59:54 -0500, Bill Davidsen wrote:
> On Thu, 20 Mar 2003, [iso-8859-1] J?rn Engel wrote:
> 
> > And I have a hard time to think of a real-world application where you
> > don't want to unpack but need to verify the signature.
> 
> My real world includes downloading a bunch of files and burning a CD to
> move them to a test environment which is completely private and has no
> external connections of any kind. I don't do all files that way, of
> course, but that is the way at least half of the 2.5 kernels I've used
> were moved to a machine which was non-production.

Real world always wins over imagination. I'll shut up now. :)

Jörn

-- 
When in doubt, use brute force.
-- Ken Thompson

Re: Deprecating .gz format on kernel.org

[Stephen Frost]

[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]

* Bill Davidsen (davidsen@tmr.com) wrote:
> On Thu, 20 Mar 2003, Thomas Duffy wrote:
> 
> > On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > > So, who can beat his 15.10 bogomips?
> > 
[...]
> > bogomips        : 12.44
> 
> At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
> (IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
> still see spam to glacial.tmr.com today. The name was NOT because it was
> so cool ;-)
> 
> I may still have that board, but I'm not about to put it back in service
> to measure speed. Your firewall is the slowest "real machine" I've seen,
> emulation and embedded machines are not really general purpose.

If we're really curious...

sfrost@ns2:/home/sfrost> cat /proc/cpuinfo
processor       : 0
vendor_id       : unknown
cpu family      : 4
model           : 0
model name      : 486
stepping        : unknown
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : no
cpuid level     : -1
wp              : yes
flags           :
bogomips        : 9.42

This is my secondary name server.  This was also after an upgrade from
a 386 because bind9 is a bloody pig. :)  To be honest I've thought about
putting the 386 back in service as something else because unlike my web
server and primary name server there's no chance a CPU fan on it is
going to die causing a CPU to fry and the system to crash.  At one point
the 386 had a 630 day uptime, running 2.2.16.

	Stephen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On Tue, 2003-03-25 11:04:51 -0500, Bill Davidsen <davidsen@tmr.com>
wrote in message <Pine.LNX.3.96.1030325110027.1437B-100000@gatekeeper.tmr.com>:
> On Thu, 20 Mar 2003, Thomas Duffy wrote:
> 
> > On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > > So, who can beat his 15.10 bogomips?
> > 
> > my firewall:
> 
> At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
> (IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
> still see spam to glacial.tmr.com today. The name was NOT because it was
> so cool ;-)
> 
> I may still have that board, but I'm not about to put it back in service
> to measure speed. Your firewall is the slowest "real machine" I've seen,

I do have such a system, too (but only 4..8MB RAM) and I'm going to
install it this evening (or tomorrow). I've just got back a 40GB HDD
from IBM^WHitachi:) That box should have just about 1.2 BogoMips...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[Bill Davidsen]

On Thu, 20 Mar 2003, Thomas Duffy wrote:

> On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > So, who can beat his 15.10 bogomips?
> 
> my firewall:
> 
> [tduffy@crackho ~]$ more /proc/cpuinfo
> processor       : 0
> vendor_id       : unknown
> cpu family      : 4
> model           : 0
> model name      : 486
> stepping        : unknown
> fdiv_bug        : no
> hlt_bug         : no
> sep_bug         : no
> f00f_bug        : no
> coma_bug        : no
> fpu             : no
> fpu_exception   : no
> cpuid level     : -1
> wp              : yes
> flags           :
> bogomips        : 12.44
> 
> -- 
> YOO-ESS-AYE! YOO-ESS-AYE!

At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
(IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
still see spam to glacial.tmr.com today. The name was NOT because it was
so cool ;-)

I may still have that board, but I'm not about to put it back in service
to measure speed. Your firewall is the slowest "real machine" I've seen,
emulation and embedded machines are not really general purpose.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Deprecating .gz format on kernel.org

[Bill Davidsen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=US-ASCII, Size: 774 bytes --]

On Thu, 20 Mar 2003, [iso-8859-1] Jörn Engel wrote:

> That shouldn't matter, most of the times. If you want to build the
> code, you have to [bg]unzip anyway, so there is no extra cost.
> And I have a hard time to think of a real-world application where you
> don't want to unpack but need to verify the signature.

My real world includes downloading a bunch of files and burning a CD to
move them to a test environment which is completely private and has no
external connections of any kind. I don't do all files that way, of
course, but that is the way at least half of the 2.5 kernels I've used
were moved to a machine which was non-production.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Deprecating .gz format on kernel.org

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1861 bytes --]

On Thu, 2003-03-20 11:51:41 -0600, Eli Carter <eli.carter@inet.com>
wrote in message <3E79FFAD.3040904@inet.com>:
> Mike Dresser wrote:
> >On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
> >
> >>jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> >
> >
> >>bogomips        : 15.10
> 
> So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?

I've tested an Intel i386SX, 16MHz these days. It reached 1.19 Bogomips
IIRC using 2.4.x.

As you may see, I'm currently collecting very different hardware (very
old ia32 based systems as well as anything != ia32 capable of running
Linux). My goal is to build an (automated) compile & boot farm of
machines to detect major breakages _early_. In the past few years, we
had quite some ups (and some really hard downs...) which I want to see:

- Remember SCSI-eh changes? Some drivers still need to be ported (or am
  I wrong here?)
- module-init-tools took quite some time...
- sparc32 unfortunately is not yet really running 2.5.x. 2.4.x doesn't
  do SMP for me (Level 15 Interrupt). However, here I do know when this
  was introduced and I'll have a look. Somewhen...
- ...

Things are easier to sort out as long as the breakages are _new_. If
they're there for a year and nobody cares about fixing them, non-ia32
architectures might again be near being dropped (sun4?).

With some luck, I'll get some room (incl. power and IP connectivity)
near Halle/Westf. (Germany). If this fails, I'd be _really_ interested
in other offers here... "CRT - Compile and Run a Test"

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[nick]

I can get down to somewhere around 2... (Mips R5k w/ no l2
cache... painfull) why?  I appear to have missed the start of this thread
	Nick

On 24 Mar 2003, Miles Bader wrote:

> Eli Carter <eli.carter@inet.com> writes:
> > So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?
> 
> The normal systems I develop for have 4-6 bogomips (NEC V850E/MA1 @50MHz).
> 
> For debugging I often run linux on gdb's simulator, which probably has
> somewhere under 1 bogomips (the number reported is not correct because I
> have to lie to the kernel about the clock interrupt rate, as it can't
> handle the real value).  It's still quite usable interactively though
> (actually 2.5.x feels noticably better than 2.4.x in this situation)...
> 
> -Miles
> -- 
> 97% of everything is grunge
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Deprecating .gz format on kernel.org

[Miles Bader]

Miles Bader <miles@lsi.nec.co.jp> writes:
> The normal systems I develop for have 4-6 bogomips (NEC V850E/MA1 @50MHz).

BTW, this system has no cache, and I think about 3 wait-states on normal
memory access.

-Miles
-- 
"Whatever you do will be insignificant, but it is very important that
 you do it."  Mahatma Ghandi

Re: Deprecating .gz format on kernel.org

[Miles Bader]

Eli Carter <eli.carter@inet.com> writes:
> So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?

The normal systems I develop for have 4-6 bogomips (NEC V850E/MA1 @50MHz).

For debugging I often run linux on gdb's simulator, which probably has
somewhere under 1 bogomips (the number reported is not correct because I
have to lie to the kernel about the clock interrupt rate, as it can't
handle the real value).  It's still quite usable interactively though
(actually 2.5.x feels noticably better than 2.4.x in this situation)...

-Miles
-- 
97% of everything is grunge

Re: Deprecating .gz format on kernel.org

[Krzysztof Halasa]

Pavel Machek <pavel@ucw.cz> writes:

> I've seen 7(or so) on i386sx/25. Alan seen <1.

Hmm. My 33 MHz i386 with 64 KB cache had 6.8 IIRC. One hour was more than
enough for the kernel to compile I think.

25 MHz i386 with no cache but with pipelined RAM had less bogomips, but
don't remember how much less.

Anyway, I don't use .gz from kernel.org anymore.
-- 
Krzysztof Halasa
Network Administrator

Re: Deprecating .gz format on kernel.org

[Hank Leininger]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 21 Mar 2003, Ville Herva wrote:

> On Thu, Mar 20, 2003 at 06:14:53PM -0500, you [Hank Leininger] wrote:
> >
> > Right, but if the uncompressed file is what's signed, then you must
> > waste either CPU uncompressing twice (once to verify, once to untar) or
> > waste disk (to store the uncompressed file, then verify, then untar).
>
> bzip2 -d < foo.tar.bz2 | tee >(md5sum) | tar xf
> or
> bzip2 -d < foo.tar.bz2 | tee >(gpg --verify foo.tar.bz2.sig) | tar xf

Yup, but (besides the tar tpyo you corrected later) this still isn't
safe.

1) gpg --verify won't be able to complete until it's seen all the unpacked
   tar file.

2) During that time tar -xf - will be unpacking and writing.

3) If the signature is bad, too late you've already unpacked it:
   -At best you need to blow away what you just unpacked.
   -Worse, it may have just (over)written real files in pwd other than
     the ones you think it should have.
   -Worst, if the tarfile is maliciously crafted to exploit tar (..'ing
     archive, symlink-following archive, or bad data which overflows
     tar), who-knows-what damage is already done.  This might sound
     far-fetched, except that it already happens.

...But it sounds like the whole discussion is dead anyway.  It would be
at least slightly less off-topic on security-audit, perhaps we should
move it there (http://lsap.org/mail.html).  Or to alt.tinfoil.hat.

Hank Leininger <hlein@progressive-comp.com>
E407 AEF4 761E D39C D401  D4F4 22F8 EF11 861A A6F1
-----BEGIN PGP SIGNATURE-----

iD8DBQE+er6+IvjvEYYapvERAqVKAJ9Z2sJ6pcib2+la0NqKYCeanuZwHwCdHEwt
9dSMcChXaa2G9sihxav6t0M=
=aCPQ
-----END PGP SIGNATURE-----

RE: Deprecating .gz format on kernel.org

[Eric Sandall]

Vlad@geekizoid.com said:
<snip>
> But, WinRAR doesn't handle symlinks.

True...hence an argument how OpenSource software is better.  ;)  Not that
I need to argue that here, and not all OS is better than closed, but it is
easier to get what you want.

Perhaps someone with the knowledge (certainly not I) could write a GPL'd
unarchiver (with GUI, since Windows folks like those) that can handle .gz,
.bz2, .zip, .lzo, etc. files?

-Sandalle

-- 
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214

Eric Sandall                     |  Source Mage GNU/Linux Developer
eric@sandall.us                  |  http://www.sourcemage.org/
http://eric.sandall.us/          |  SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/  #196285  |  http://www.shock.wsu.edu/

Re: Deprecating .gz format on kernel.org

[Ville Herva]

On Fri, Mar 21, 2003 at 08:26:35AM +0200, you [Ville Herva] wrote:
> 
> bzip2 -d < foo.tar.bz2 | tee >(md5sum) | tar xf
> or
> bzip2 -d < foo.tar.bz2 | tee >(gpg --verify foo.tar.bz2.sig) | tar xf

Uhh sorry, "tar x" or "tar xf -" (if your tar so requires, most don't), not
"tar xf". Not enough coffee...


-- v --

v@iki.fi

Re: Deprecating .gz format on kernel.org

[Ville Herva]

On Thu, Mar 20, 2003 at 06:14:53PM -0500, you [Hank Leininger] wrote:
> 
> Right, but if the uncompressed file is what's signed, then you must
> waste either CPU uncompressing twice (once to verify, once to untar) or
> waste disk (to store the uncompressed file, then verify, then untar).

bzip2 -d < foo.tar.bz2 | tee >(md5sum) | tar xf
or
bzip2 -d < foo.tar.bz2 | tee >(gpg --verify foo.tar.bz2.sig) | tar xf
 

-- v --

v@iki.fi

RE: Deprecating .gz format on kernel.org

[Vlad@geekizoid.com]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org
> [mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Eric Sandall
> Sent: Thursday, March 20, 2003 6:10 PM
> To: linux-kernel@vger.kernel.org
> Subject: RE: Deprecating .gz format on kernel.org
> LA Walsh said:
> > ----
> > 	I asked WinZip for plugin support so users could add
> > arbitrary compressions formats (I specifically mentioned 'bz2').
> > I even offered to do it myself if they wanted to give me 
> NDA access to
> > the code....  and was told that they have thought about
> > extensibility but had no plans to support  it in the forseeable
> > future.  Consider it a benefit of a closed source product.
> > They control your file format, they control your business 
> and personal
> > decisions...nice.
> > 	Anyone up for writing a Free version of Winzip to replace
> > the "cooperative" Winzip authors' version (was was up for making
> > minor mod's, but not starting from scratch).
> > -linda
> FYI, WinRAR (http://www.rarsoft.com) supports BZ2 files.  ;)
> -Sandalle

But, WinRAR doesn't handle symlinks.

--

 /"\                         / For information and quotes, email us at
 \ /  ASCII RIBBON CAMPAIGN / info@lrsehosting.com
  X   AGAINST HTML MAIL    / http://www.lrsehosting.com/
 / \  AND POSTINGS        / vlad@lrsehosting.com
-------------------------------------------------------------------------

RE: Deprecating .gz format on kernel.org

[Eric Sandall]

LA Walsh said:
> ----
> 	I asked WinZip for plugin support so users could add
> arbitrary compressions formats (I specifically mentioned 'bz2').
> I even offered to do it myself if they wanted to give me NDA access to
> the code....  and was told that they have thought about
> extensibility but had no plans to support  it in the forseeable
> future.  Consider it a benefit of a closed source product.
> They control your file format, they control your business and personal
> decisions...nice.
>
> 	Anyone up for writing a Free version of Winzip to replace
> the "cooperative" Winzip authors' version (was was up for making
> minor mod's, but not starting from scratch).
>
> -linda

FYI, WinRAR (http://www.rarsoft.com) supports BZ2 files.  ;)

-Sandalle

-- 
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214

Eric Sandall                     |  Source Mage GNU/Linux Developer
eric@sandall.us                  |  http://www.sourcemage.org/
http://eric.sandall.us/          |  SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/  #196285  |  http://www.shock.wsu.edu/

Re: Deprecating .gz format on kernel.org

[H. Peter Anvin]

Followup to:  <200303202154.h2KLsDcT009516@marc2.theaimsgroup.com>
By author:    Hank Leininger <linux-kernel@progressive-comp.com>
In newsgroup: linux.dev.kernel
>
> On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> > On Thu, 20 March 2003 17:39:20 +0000, Jamie Lokier wrote:
> > > (b) On something as large as a .tar, decompressing a bz2 file to
> > > check the signature is really quite slow, compared with checking the
> > > signature of the compressed file.
> 
> > That shouldn't matter, most of the times. If you want to build the
> > code, you have to [bg]unzip anyway, so there is no extra cost.
> > And I have a hard time to think of a real-world application where you
> > don't want to unpack but need to verify the signature.
> 

Just to finish this debate: I have added support for generating .sign
files from .gz files, and those are currently being generated, but I
will not remove .gz.sign or .bz2.sign files.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: Deprecating .gz format on kernel.org

[Jörn Engel]

On Thu, 20 March 2003 18:14:53 -0500, Hank Leininger wrote:
> On Thu, 20 Mar 2003, [iso-8859-1] Jörn Engel wrote:
> 
> > "Come to mind" doesn't sound line "that'd break our environment." ;)
> 
> Well, no ;)  But I could see how they might break some, and/or would
> cause real problems even though they wouldn't be insurmountable ones.

might/could doesn't matter. does matters. ;)

> > > -To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
> > >   tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
> > >   to a tempfile or pipe to gpg, then verify, then untar.  Silly waste of
> > >   CPU and/or disk space.[*]
> >
> > Veryfy and use are two action. You need a script or a human, changing
> > either one won't be hard.
> 
> Right, but if the uncompressed file is what's signed, then you must
> waste either CPU uncompressing twice (once to verify, once to untar) or
> waste disk (to store the uncompressed file, then verify, then untar).

Waste the disk. You'll waste it anyway, once you start to compile and
by that time, the temporary tar is deleted.

> > real-world?
> 
> Sure.  You're net-connected only intermittantly, and want to verify
         ^^^
> downloads as soon as you get them, so you can re-get before
                       ^^^              ^^^
> disconnecting.  Or you're pulling files down to burn to CD, you'd like
                     ^^^                                      ^^^
> to know if they are bad before you've burned the CD and try to use it
                                 ^^^
> elsewhere; unpacking just to verify is a waste.

*I* don't have any of those problems. But it is good to know that you
are concerned about me. :)

> Hell, here's an easy example: the kernel.org mirror sites.  I don't know
> for sure if any of them --verify the tarball+sig files that they mirror,
> but it sure would make a lot of sense.  With signatures of the
> compressed tarballs, that would be trivial.  With signatures of the .tar
> files, it would be far more resource-intensive for them to implement.

Agreed, that would be a problem. 

> Another flavor I hit personally is: I want to hack on and/or upgrade my
> laptop to some new version of something while on a flight.  Before
> leaving I pull down the tarballs and sig files to my main workstation,
> verify the signatures, scp to the laptop.  It'd be a waste to unpack
> just to verify before pushing it, a pain to have to unpack and verify on
> the laptop right away, and a showstopper if I didn't verify until I was
> at 30,000 feet.

Malicious packages should be rare enough to ignore the showstopper
argument. If that problem bites you on a regular basis,...

> > > [*] ...Now if tar had a --sig option to chain gpg between gunzip and
> > >     untar... but that would just be Wrong.
> >
> > unzip && checksig && tar?
> 
> Yes, but all as one pipeline, not seperate commands with tempfiles.
> After all there is a reason the [Zzj] flags were added to tar.  Signing
> the intermediate .tar instead of the .tar.(gz|bs2) breaks that.
> 
> But that's inventing questionable features in GNU tar that won't be
> present in other tars or archive-managing tools, creating dependencies
> on GPG/PGP/tool of choice, etc.  Unnecessary complexity.
> 
> Actually, unlike gzip | tar -xf -, you can't start feeding the
> signature-checked tar file to tar -xf until all of it has been read in
> to be verified, so really, it can't be done in a single pass.

Yup, I'm convinced. signatures for plain tars would still be a nice
thing to have, but they cannot replace those for .gz and .bz2.

Jörn

-- 
Victory in war is not repetitious.
-- Sun Tzu

Re: Deprecating .gz format on kernel.org

[Hank Leininger]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 4330 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 20 Mar 2003, [iso-8859-1] Jörn Engel wrote:

> On Thu, 20 March 2003 16:54:13 -0500, Hank Leininger wrote:
> > On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> >
> > A few come to mind:
>
> "Come to mind" doesn't sound line "that'd break our environment." ;)

Well, no ;)  But I could see how they might break some, and/or would
cause real problems even though they wouldn't be insurmountable ones.

> > -To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
> >   tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
> >   to a tempfile or pipe to gpg, then verify, then untar.  Silly waste of
> >   CPU and/or disk space.[*]
>
> Veryfy and use are two action. You need a script or a human, changing
> either one won't be hard.

Right, but if the uncompressed file is what's signed, then you must
waste either CPU uncompressing twice (once to verify, once to untar) or
waste disk (to store the uncompressed file, then verify, then untar).

> > -Verifying downloads immediately, when they won't necessarily be needed /
> >   used right away; no need to unpack until it's needed, but would like to
> >   know the download is bad right away.
>
> real-world?

Sure.  You're net-connected only intermittantly, and want to verify
downloads as soon as you get them, so you can re-get before
disconnecting.  Or you're pulling files down to burn to CD, you'd like
to know if they are bad before you've burned the CD and try to use it
elsewhere; unpacking just to verify is a waste.

> > -Verifying something pulled down to one machine before scp'ing it elsewhere
> >   where it will actually be used.
>
> real-world?

Sure.  Fits same as above.  Say you pull stuff down to a somewhat
central internal repository and then push it out to several other boxes,
which may themselves have no generalized external connectivity (no
default routes, behind firewalls, running local firewalling, etc).
You'd want to verify it there and then (perhaps in addition to at the
final destination depending on how paranoid you are) and then push it
around, still compressed, or you're wasting work.

Hell, here's an easy example: the kernel.org mirror sites.  I don't know
for sure if any of them --verify the tarball+sig files that they mirror,
but it sure would make a lot of sense.  With signatures of the
compressed tarballs, that would be trivial.  With signatures of the .tar
files, it would be far more resource-intensive for them to implement.

Another flavor I hit personally is: I want to hack on and/or upgrade my
laptop to some new version of something while on a flight.  Before
leaving I pull down the tarballs and sig files to my main workstation,
verify the signatures, scp to the laptop.  It'd be a waste to unpack
just to verify before pushing it, a pain to have to unpack and verify on
the laptop right away, and a showstopper if I didn't verify until I was
at 30,000 feet.

> > -Verifying before [bg]unzip means you won't expose [bg]unzip to likely
> >   malicious data (think bugs in [bg]unzip which make them crash on bad
> >   compressed files).  Of course GPG/PGP is still subject to input-based
>
> Crashes don't matter. Exploits would, so that point is actually valid.

Well, indeed, that's what I was suggesting.

> > [*] ...Now if tar had a --sig option to chain gpg between gunzip and
> >     untar... but that would just be Wrong.
>
> unzip && checksig && tar?

Yes, but all as one pipeline, not seperate commands with tempfiles.
After all there is a reason the [Zzj] flags were added to tar.  Signing
the intermediate .tar instead of the .tar.(gz|bs2) breaks that.

But that's inventing questionable features in GNU tar that won't be
present in other tars or archive-managing tools, creating dependencies
on GPG/PGP/tool of choice, etc.  Unnecessary complexity.

Actually, unlike gzip | tar -xf -, you can't start feeding the
signature-checked tar file to tar -xf until all of it has been read in
to be verified, so really, it can't be done in a single pass.


Hank Leininger <hlein@progressive-comp.com>
E407 AEF4 761E D39C D401  D4F4 22F8 EF11 861A A6F1

-----BEGIN PGP SIGNATURE-----

iD8DBQE+ekttIvjvEYYapvERAtY1AJ9QX8suP6G6zHxQjHiM8yxGO8QtEwCeMKJL
+jgFyusy4ofF7esaP/mMu/w=
=9xEC
-----END PGP SIGNATURE-----

RE: Deprecating .gz format on kernel.org

[LA Walsh]

'Sam Ravnborg' wrote:
> It is a convinient feature that I can download the kernel now and then
> when at work (no linux) - and it makes it a bit simpler to use
> an archiver that has native support for the format used.
> Winzip, being the only allowed archiver at work, does not
> have native bz2 support.
----
	I asked WinZip for plugin support so users could add
arbitrary compressions formats (I specifically mentioned 'bz2').
I even offered to do it myself if they wanted to give me NDA access
to the code....  and was told that they have thought about
extensibility but had no plans to support  it in the forseeable
future.  Consider it a benefit of a closed source product.
They control your file format, they control your business and personal decisions...nice.

	Anyone up for writing a Free version of Winzip to replace
the "cooperative" Winzip authors' version (was was up for making
minor mod's, but not starting from scratch).

-linda

Re: Deprecating .gz format on kernel.org

[Jörn Engel]

On Thu, 20 March 2003 16:54:13 -0500, Hank Leininger wrote:
> On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> 
> > That shouldn't matter, most of the times. If you want to build the
> > code, you have to [bg]unzip anyway, so there is no extra cost.
> > And I have a hard time to think of a real-world application where you
                                         ^^^^^^^^^^
> > don't want to unpack but need to verify the signature.
> 
> A few come to mind:

"Come to mind" doesn't sound line "that'd break our environment." ;)

> -To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
>   tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
>   to a tempfile or pipe to gpg, then verify, then untar.  Silly waste of
>   CPU and/or disk space.[*]

Veryfy and use are two action. You need a script or a human, changing
either one won't be hard.

> -Verifying downloads immediately, when they won't necessarily be needed /
>   used right away; no need to unpack until it's needed, but would like to
>   know the download is bad right away.

real-world?

> -Verifying something pulled down to one machine before scp'ing it elsewhere
>   where it will actually be used.

real-world?

> -Verifying before [bg]unzip means you won't expose [bg]unzip to likely
>   malicious data (think bugs in [bg]unzip which make them crash on bad
>   compressed files).  Of course GPG/PGP is still subject to input-based 
>   bugs, but they are in any case; no need for the decompression tools to
>   be as well.

Crashes don't matter. Exploits would, so that point is actually valid.

> [*] ...Now if tar had a --sig option to chain gpg between gunzip and 
>     untar... but that would just be Wrong.

unzip && checksig && tar?

Jörn

-- 
More computing sins are committed in the name of efficiency (without
necessarily achieving it) than for any other single reason - including
blind stupidity.
-- W. A. Wulf 

Re: Deprecating .gz format on kernel.org

[Hank Leininger]

On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> On Thu, 20 March 2003 17:39:20 +0000, Jamie Lokier wrote:
> > (b) On something as large as a .tar, decompressing a bz2 file to
> > check the signature is really quite slow, compared with checking the
> > signature of the compressed file.

> That shouldn't matter, most of the times. If you want to build the
> code, you have to [bg]unzip anyway, so there is no extra cost.
> And I have a hard time to think of a real-world application where you
> don't want to unpack but need to verify the signature.

A few come to mind:
-To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
  tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
  to a tempfile or pipe to gpg, then verify, then untar.  Silly waste of
  CPU and/or disk space.[*]
-Verifying downloads immediately, when they won't necessarily be needed /
  used right away; no need to unpack until it's needed, but would like to
  know the download is bad right away.
-Verifying something pulled down to one machine before scp'ing it elsewhere
  where it will actually be used.
-Verifying before [bg]unzip means you won't expose [bg]unzip to likely
  malicious data (think bugs in [bg]unzip which make them crash on bad
  compressed files).  Of course GPG/PGP is still subject to input-based 
  bugs, but they are in any case; no need for the decompression tools to
  be as well.

[*] ...Now if tar had a --sig option to chain gpg between gunzip and 
    untar... but that would just be Wrong.

--
Hank Leininger <hlein@progressive-comp.com> 
  

Re: Deprecating .gz format on kernel.org

[Jörn Engel]

On Thu, 20 March 2003 17:39:20 +0000, Jamie Lokier wrote:
> Eric Sandall wrote:
> > 
> > Why not get the signature from the .tar file, that way the compression
> > method doesn't matter?  This is how Source Mage does it's checking, we
> > create and md5sum (and soon GPG) signature based on the uncompressed .tar
> > file.  This way, you can use any compression you want, even changing
> > around the compression to your favourite one, and the signatures will
> > always match.  :)
> 
> (b) On something as large as a .tar, decompressing a bz2 file to check
>     the signature is really quite slow, compared with checking the
>     signature of the compressed file.

That shouldn't matter, most of the times. If you want to build the
code, you have to [bg]unzip anyway, so there is no extra cost.
And I have a hard time to think of a real-world application where you
don't want to unpack but need to verify the signature.

Jörn

-- 
If System.PrivateProfileString("",
"HKEY_CURRENT_USER\Software\Microsoft\Office\9.0\Word\Security", "Level") <>
"" Then  CommandBars("Macro").Controls("Security...").Enabled = False
-- from the Melissa-source

Re: Deprecating .gz format on kernel.org

[Pavel Machek]

Hi!

> > So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?
> My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.

I've seen 7(or so) on i386sx/25. Alan seen <1.
								Pavel
-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Re: Deprecating .gz format on kernel.org

[Pascal Schmidt]

On Thu, 20 Mar 2003 19:20:06 +0100, you wrote in linux.kernel:

> So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?
My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.

-- 
Ciao,
Pascal

Re: Deprecating .gz format on kernel.org

[Eric Sandall]

Jamie Lokier said:
> Eric Sandall wrote:
>> Jamie Lokier said:
>> <snip>
>> > Which is ok of course, but then the signatures don't match any more.
>> <snip>
>> > -- Jamie
>>
>> Why not get the signature from the .tar file, that way the compression
>> method doesn't matter?  This is how Source Mage does it's checking, we
>> create and md5sum (and soon GPG) signature based on the uncompressed
>> .tar file.  This way, you can use any compression you want, even
>> changing around the compression to your favourite one, and the
>> signatures will always match.  :)
>
> (a) I use .gz for the patches not the tar files.  But your point still
> applies.
>
> (b) On something as large as a .tar, decompressing a bz2 file to check
>     the signature is really quite slow, compared with checking the
> signature of the compressed file.
>
> -- Jamie

True...for large files it'd be nice to know if you have the correct
tarball _before_ spending all that CPU time decompressing it.  It's a
trade off, mostly, CPU time for more generic useage.  I know this isn't
the Right Way(TM), but since fast computers are becoming fairly cheap, I
say the signature on the .tar file is a good way to go.  However, Linux
also runs sufficiently well on slow machines, and this would just make
them slower.  Shall we just follow the path of least resistence and keep
it as is?

-Sandalle

-- 
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214

Eric Sandall                     |  Source Mage GNU/Linux Developer
eric@sandall.us                  |  http://www.sourcemage.org/
http://eric.sandall.us/          |  SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/  #196285  |  http://www.shock.wsu.edu/

re: Deprecating .gz format on kernel.org

[Tomas Szepe]

> [dana.lacoste@peregrine.com]
> 
> On Thu, 2003-03-20 at 12:48, Tomas Szepe wrote:
> > Well, you see, hpa probably made a mistake in his original proposal.
> 
> > Likely he meant to say he intended to deprecate bz2 and was about
> > to introduce lzo instead.  ;)
> 
> LZO rocks!
> 
> It's got my vote :)

Yeah, lzo{,p} makes my good old sparc32s feel like gigahertz monsters.  :)

-- 
Tomas Szepe <szepe@pinerecords.com>

Re: Deprecating .gz format on kernel.org

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2007 bytes --]

On Thu, 2003-03-20 09:23:34 -0800, Randy.Dunlap <rddunlap@osdl.org>
wrote in message <20030320092334.31ee2254.rddunlap@osdl.org>:
> On Thu, 20 Mar 2003 17:32:07 +0100 Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:
> 
> | However, please keep in mind that it's a *PITA* if you're working on a
> | machine with not > 500MHz and > 128MB RAM:
> | 
> | jbglaw@schnarchnase:/tmp$ ls -l linux-2.5.65.tar.*
> | -rw-r--r--    1 jbglaw   jbglaw   31889910 Mar 20 11:37
> | linux-2.5.65.tar.bz2
> | -rw-r--r--    1 jbglaw   jbglaw   39711645 Mar 20 11:44
> | linux-2.5.65.tar.gz
> | jbglaw@schnarchnase:/tmp$ time tar xjf linux-2.5.65.tar.bz2
> | 
> | real    194m21.665s
> | user    172m55.026s
> | sys     14m19.018s
> | jbglaw@schnarchnase:/tmp$ mv linux-2.5.65 linux-2.5.65xx
> | jbglaw@schnarchnase:/tmp$ time tar xzf linux-2.5.65.tar.gz
> | 
> | real    39m39.294s
> | user    22m32.306s
> | sys     13m56.524s
> | jbglaw@schnarchnase:/tmp$ free
> |              total       used       free     shared    buffers     cached
> | Mem:         10100       9792        308          0        952       5232
> ...
> | jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo 
> ...
> | bogomips        : 15.10
> | 
> | jbglaw@schnarchnase:/tmp$ uname -a
> | Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown unknown GNU/Linux
> 
> What kind of processor/system is that?

It's a UMC chip - a i486 clone without FPU. (I thought that this chip
was an i386 clone but uname tells me different so I believe it.)

However, I've got two early i386 (original Intel brand) laying around
and I'm definitely like installing Linux on them, just to see it
running:-)

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

On Thu, 2003-03-20 11:51:41 -0600, Eli Carter <eli.carter@inet.com>
wrote in message <3E79FFAD.3040904@inet.com>:
> Mike Dresser wrote:
> >On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
> >
> >
> >>jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> >
> >
> >>bogomips        : 15.10
> >
> >
> >out of curiosity, how long does the machine take to compile a kernel?
> >
> >Do you use a stopwatch or a calendar?
> 
> Well, going back to his original email,
> 
> jbglaw@schnarchnase:/tmp$ uname -a
> Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown 
> unknown GNU/Linux
> 
> And looking on kernel.org,
> Mar 17 22:29 linux-2.5.65.tar.gz
> 
> It takes him less than a week anyway*... ;)  (Timezone conversions left 
> as an exercise to the reader.)

Um, no. I compiled it on my Athlon (2x1.4GHz) and that's more like
5min... Letting the box compile a full 2.5.x (with mostly anything
useable choosen as a module), I think that would be at about 10..20 day.
So it's more like looking at a calendar than looking at a stopwatch:=

> So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?

No problem - start linux from within bochs:)
-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[Thomas Duffy]

On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> So, who can beat his 15.10 bogomips?

my firewall:

[tduffy@crackho ~]$ more /proc/cpuinfo
processor       : 0
vendor_id       : unknown
cpu family      : 4
model           : 0
model name      : 486
stepping        : unknown
fdiv_bug        : no
hlt_bug         : no
sep_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : no
fpu_exception   : no
cpuid level     : -1
wp              : yes
flags           :
bogomips        : 12.44

-- 
YOO-ESS-AYE! YOO-ESS-AYE!

re: Deprecating .gz format on kernel.org

[Dana Lacoste]

On Thu, 2003-03-20 at 12:48, Tomas Szepe wrote:
> Well, you see, hpa probably made a mistake in his original proposal.

> Likely he meant to say he intended to deprecate bz2 and was about
> to introduce lzo instead.  ;)

LZO rocks!

It's got my vote :)

--
Dana Lacoste
Ottawa, Canada

Re: Deprecating .gz format on kernel.org

[Eric Sandall]

Jamie Lokier said:
<snip>
> Which is ok of course, but then the signatures don't match any more.
<snip>
> -- Jamie

Why not get the signature from the .tar file, that way the compression
method doesn't matter?  This is how Source Mage does it's checking, we
create and md5sum (and soon GPG) signature based on the uncompressed .tar
file.  This way, you can use any compression you want, even changing
around the compression to your favourite one, and the signatures will
always match.  :)

-Sandalle

-- 
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214

Eric Sandall                     |  Source Mage GNU/Linux Developer
eric@sandall.us                  |  http://www.sourcemage.org/
http://eric.sandall.us/          |  SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/  #196285  |  http://www.shock.wsu.edu/

Re: Deprecating .gz format on kernel.org

[Eli Carter]

Mike Dresser wrote:
> On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
> 
> 
>>jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> 
> 
>>bogomips        : 15.10
> 
> 
> out of curiosity, how long does the machine take to compile a kernel?
> 
> Do you use a stopwatch or a calendar?

Well, going back to his original email,

jbglaw@schnarchnase:/tmp$ uname -a
Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown 
unknown GNU/Linux

And looking on kernel.org,
Mar 17 22:29 linux-2.5.65.tar.gz

It takes him less than a week anyway*... ;)  (Timezone conversions left 
as an exercise to the reader.)

So, who can beat his 15.10 bogomips?  Anyone run <1 bogomips?

*grin*

Eli

* Yes, I know he could be doing patches and incremental builds.
--------------------. "If it ain't broke now,
Eli Carter           \                  it will be soon." -- crypto-gram
eli.carter(a)inet.com `-------------------------------------------------

re: Deprecating .gz format on kernel.org

[Tomas Szepe]

> [jbglaw@lug-owl.de]
> 
> On Wed, 2003-03-19 12:19:42 -0800, H. Peter Anvin <hpa@zytor.com>
> wrote in message <3E78D0DE.307@zytor.com>:
> > Hello everyone,
> > 
> > At some point it probably would make sense to start deprecating .gz
> > format files from kernel.org.
> 
> However, please keep in mind that it's a *PITA* if you're working on a
> machine with not > 500MHz and > 128MB RAM:

Well, you see, hpa probably made a mistake in his original proposal.

Likely he meant to say he intended to deprecate bz2 and was about
to introduce lzo instead.  ;)

-- 
Tomas Szepe <szepe@pinerecords.com>

Re: Deprecating .gz format on kernel.org

[Jamie Lokier]

Eric Sandall wrote:
> Jamie Lokier said:
> <snip>
> > Which is ok of course, but then the signatures don't match any more.
> <snip>
> > -- Jamie
> 
> Why not get the signature from the .tar file, that way the compression
> method doesn't matter?  This is how Source Mage does it's checking, we
> create and md5sum (and soon GPG) signature based on the uncompressed .tar
> file.  This way, you can use any compression you want, even changing
> around the compression to your favourite one, and the signatures will
> always match.  :)

(a) I use .gz for the patches not the tar files.  But your point still applies.

(b) On something as large as a .tar, decompressing a bz2 file to check
    the signature is really quite slow, compared with checking the
    signature of the compressed file.

-- Jamie

Re: Deprecating .gz format on kernel.org

[Randy.Dunlap]

On Thu, 20 Mar 2003 17:32:07 +0100 Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:

| However, please keep in mind that it's a *PITA* if you're working on a
| machine with not > 500MHz and > 128MB RAM:
| 
| jbglaw@schnarchnase:/tmp$ ls -l linux-2.5.65.tar.*
| -rw-r--r--    1 jbglaw   jbglaw   31889910 Mar 20 11:37
| linux-2.5.65.tar.bz2
| -rw-r--r--    1 jbglaw   jbglaw   39711645 Mar 20 11:44
| linux-2.5.65.tar.gz
| jbglaw@schnarchnase:/tmp$ time tar xjf linux-2.5.65.tar.bz2
| 
| real    194m21.665s
| user    172m55.026s
| sys     14m19.018s
| jbglaw@schnarchnase:/tmp$ mv linux-2.5.65 linux-2.5.65xx
| jbglaw@schnarchnase:/tmp$ time tar xzf linux-2.5.65.tar.gz
| 
| real    39m39.294s
| user    22m32.306s
| sys     13m56.524s
| jbglaw@schnarchnase:/tmp$ free
|              total       used       free     shared    buffers     cached
| Mem:         10100       9792        308          0        952       5232
...
| jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo 
...
| bogomips        : 15.10
| 
| jbglaw@schnarchnase:/tmp$ uname -a
| Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown unknown GNU/Linux

What kind of processor/system is that?

I also see quite a difference in untarring gz vs. bz2.

On a Pentium classic with genuine f00f bug, 180 Mhz, 80 MB of RAM,
running Linux 2.5.64, I get:

[rddunlap@eeyore linsrc]$ time tar xzf linux-2.5.65.tar.gz
41.98user 38.15system 1:45.94elapsed 75%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (314major+97minor)pagefaults 0swaps
[rddunlap@eeyore linsrc]$ mv linux-2.5.65 linux-2565

[rddunlap@eeyore linsrc]$ time tar xjf linux-2.5.65.tar.bz2
243.99user 39.62system 5:29.39elapsed 86%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (324major+971minor)pagefaults 0swaps

--
~Randy

Re: Deprecating .gz format on kernel.org

[Mike Dresser]

On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:

> jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo

> bogomips        : 15.10

out of curiosity, how long does the machine take to compile a kernel?

Do you use a stopwatch or a calendar?

Mike

Re: Deprecating .gz format on kernel.org

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1992 bytes --]

On Wed, 2003-03-19 12:19:42 -0800, H. Peter Anvin <hpa@zytor.com>
wrote in message <3E78D0DE.307@zytor.com>:
> Hello everyone,
> 
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.

However, please keep in mind that it's a *PITA* if you're working on a
machine with not > 500MHz and > 128MB RAM:

jbglaw@schnarchnase:/tmp$ ls -l linux-2.5.65.tar.*
-rw-r--r--    1 jbglaw   jbglaw   31889910 Mar 20 11:37
linux-2.5.65.tar.bz2
-rw-r--r--    1 jbglaw   jbglaw   39711645 Mar 20 11:44
linux-2.5.65.tar.gz
jbglaw@schnarchnase:/tmp$ time tar xjf linux-2.5.65.tar.bz2

real    194m21.665s
user    172m55.026s
sys     14m19.018s
jbglaw@schnarchnase:/tmp$ mv linux-2.5.65 linux-2.5.65xx
jbglaw@schnarchnase:/tmp$ time tar xzf linux-2.5.65.tar.gz

real    39m39.294s
user    22m32.306s
sys     13m56.524s
jbglaw@schnarchnase:/tmp$ free
             total       used       free     shared    buffers     cached
Mem:         10100       9792        308          0        952       5232
-/+ buffers/cache:       3608       6492
Swap:       292816       1484     291332
jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo 
processor       : 0
vendor_id       : UMC UMC UMC 
cpu family      : 4
model           : 2
model name      : ff/02
stepping        : 3
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : no
fpu_exception   : no
cpuid level     : 1
wp              : yes
flags           :
bogomips        : 15.10

jbglaw@schnarchnase:/tmp$ uname -a
Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown unknown GNU/Linux


MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[Jon Portnoy]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 472 bytes --]

On Thu, 20 Mar 2003, Dagfinn Ilmari [iso-8859-1] Mannsåker wrote:

> Thierry Vignaud <tvignaud@mandrakesoft.com> writes:
> 
> > "Eric Sandall" <eric@sandall.us> writes:
> >
> >> > > > I can't speak for the others, but Red Hat Linux uses the .bz2
> >> > > > files in kernel rpms
> >> > >
> >> > > Conectiva too.
> >> >
> >> > ... and so does SuSE.
> >> 
> >> And Source Mage (http://www.sourcemage.org).  :)
> >
> > mandrake too ...
> 
> Debian as well.
> 

And Gentoo :-)

Re: Deprecating .gz format on kernel.org

[Juan Quintela]

>>>>> "tigran" == Tigran Aivazian <tigran@veritas.com> writes:

tigran> On Wed, 19 Mar 2003, H. Peter Anvin wrote:
>> i) Does this sound reasonable to everyone?  In particular, is there any
>> loss in losing the "original" compressed files?

tigran> No, there is at least one reason for the "original" .gz files. Here are 
tigran> the logical steps:

tigran> a) any Linux distribution contains their own "linux" package with the 
tigran> source base being "vanilla" Linux .tar.gz file

Mandrake also use .bz2 since ages ago.

Later, Juan.

-- 
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy

Re: Deprecating .gz format on kernel.org

[Richard B. Johnson]

On Thu, 20 Mar 2003, John Jasen wrote:

>
> Among the latest 2.4-release kernels (2.4.19 and 2.4.20), it seems that
> bz2 saves ~6MB.
>
> Downloads: 1.5MB DSL
>
> time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.gz`
> real    3m24.004s
> <snipped>
>
> time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.bz2`
> real    2m51.481s
> <snipped>
>
> Uncompression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
> time `gunzip linux-2.4.20.tar.gz`
> real    0m5.428s
> user    0m2.285s
> sys     0m1.096s
>
> time `bunzip2 linux-2.4.20.tar.bz2 `
> real    0m28.892s
> user    0m27.318s
> sys     0m1.363s
>
> Compression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
> time `gzip linux-2.4.20.tar`
> real    0m18.771s
> user    0m17.990s
> sys     0m0.674s
>
> time `gzip -9 linux-2.4.20.tar`
> real    0m42.032s
> user    0m40.725s
> sys     0m0.791s
>
> time `bzip2 linux-2.4.20.tar`
> real    1m50.411s
> user    1m49.197s
> sys     0m0.555s
>
> bz2 is about 18% of the size of the tarfile. gz is 22%. gzip -9 saved a
> whopping 310k compared to gzip.
>
> --
> -- John E. Jasen (jjasen@realityfailure.org)
> -- User Error #2361: Please insert coffee and try again.
>

Simple question. Has anybody provided a modified `tar` that
will properly extract `tar -xzf ...` without having to determine
ahead of time if it's a bz2 or gzip file? If not, you will
obsolete bz(2)illions of scripts that are in use for automatic
functions world-wide.

If tar will filter through past, present, and future compression
programs without human intervention, then you can get rid of
anything you want. But, until this is done, you need to leave
the "past" alone. There are many machines that get and install
stuff from tapes and home-brew CDs. They may not even have
'bz' stuff. Right now you have to modify scripts to use
tar -xjf after parsing the file-type to check its name and,
      |______ Dumb, doesn't make any sense.
some working systems don't even have bzip2 and are not connected
to the Internet.

Last year, I was in Kuala Lumpur, Malaysia (a nice modern city).
My company sent me a new CD to install a software update on
a 3-year old system. The compression was in "@$^_@%" bz format.
There was no such thing on that machine. I had to buy some
"Distribution" and waste time finding out how to extract the
bzip2 stuff from the CD/ROM. Not easy to do when the Distribution
CD/ROM was designed to install a complete operating system.

So my advise is to let the past slowly die away. Do not convert
any older distributions into something new. Leave them alone.
Make sure that any new distribution has the tools to handle the
old stuff and the next stuff. Then you can't go wrong.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: Deprecating .gz format on kernel.org

[John Jasen]

Among the latest 2.4-release kernels (2.4.19 and 2.4.20), it seems that 
bz2 saves ~6MB.

Downloads: 1.5MB DSL

time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.gz`
real    3m24.004s
<snipped>

time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.bz2`
real    2m51.481s
<snipped>

Uncompression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
time `gunzip linux-2.4.20.tar.gz`
real    0m5.428s
user    0m2.285s
sys     0m1.096s

time `bunzip2 linux-2.4.20.tar.bz2 `
real    0m28.892s
user    0m27.318s
sys     0m1.363s

Compression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
time `gzip linux-2.4.20.tar`
real    0m18.771s
user    0m17.990s
sys     0m0.674s

time `gzip -9 linux-2.4.20.tar`
real    0m42.032s
user    0m40.725s
sys     0m0.791s

time `bzip2 linux-2.4.20.tar`
real    1m50.411s
user    1m49.197s
sys     0m0.555s

bz2 is about 18% of the size of the tarfile. gz is 22%. gzip -9 saved a 
whopping 310k compared to gzip.

-- 
-- John E. Jasen (jjasen@realityfailure.org)
-- User Error #2361: Please insert coffee and try again.

Re: Deprecating .gz format on kernel.org

[Erik Hensema]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

H. Peter Anvin (hpa@zytor.com) wrote:
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.

[...]

I think it may be better to dump the .gz format for 2.5 and the forthcoming
2.6 series in favour of .bz2. Keep .gz for 2.[0-4] for those who need it (I
certainly don't).

- -- 
Erik Hensema <erik@hensema.net>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE+eZINuNWfqpFSu/cRAkJfAJ9LNq+2krlpaU9PTNE0m0pVgMtgngCg+L8G
FT6d5flYtelPMNCoEvoZQTY=
=MUhg
-----END PGP SIGNATURE-----

Re: Deprecating .gz format on kernel.org

[John Bradford]

> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.

Presumably the idea is to save bandwidth.  In that case, it might be
simpler to add:

<h1>.gz is depreciated and will be removed soon!</h1>
Please start using .bz2 immediately.  .gz downloads are rate limited
to 5kbps.

to index.html :-).

Seriously, there are occasions when you're patching a kernel on a 486
or something, that it's quicker to use the .gz than the .bz2, even
including the download time, (if you're on a fast connection).
However, there are probably a lot of people downloading .gz, because
they don't know what .bz2 is.

Likewise, http access to kernel.org is frequently slow, whereas ftp
access can use 100% of my link, simply because many more people use
http.

One other thing that occurs to me, the send your e-mail address as
anonymous password requirement means I can't use wget to access the
ftp site, (I think - the user:password@host syntax seems to confuse
it, when it's anonymous:me@mydomain@ftp.site).

John.

Re: Deprecating .gz format on kernel.org

[Dagfinn Ilmari Mannsåker]

Thierry Vignaud <tvignaud@mandrakesoft.com> writes:

> "Eric Sandall" <eric@sandall.us> writes:
>
>> > > > I can't speak for the others, but Red Hat Linux uses the .bz2
>> > > > files in kernel rpms
>> > >
>> > > Conectiva too.
>> >
>> > ... and so does SuSE.
>> 
>> And Source Mage (http://www.sourcemage.org).  :)
>
> mandrake too ...

Debian as well.

-- 
ilmari

Re: Deprecating .gz format on kernel.org

[Thierry Vignaud]

"Eric Sandall" <eric@sandall.us> writes:

> > > > I can't speak for the others, but Red Hat Linux uses the .bz2
> > > > files in kernel rpms
> > >
> > > Conectiva too.
> >
> > ... and so does SuSE.
> 
> And Source Mage (http://www.sourcemage.org).  :)

mandrake too ...

Re: Deprecating .gz format on kernel.org

[H. Peter Anvin]

Martin J. Bligh wrote:
>>OK, there seems to be enough resistance to this to put it off for a year
>>or two.  Since that means I don't have to do any work, that plan has
>>inherent appeal to me.
>>
>>In other words, the current setup will remain for now.  HOWEVER, I would
>>like to recommend that mirror sites start carrying .bz2 files if they
>>want to carry only one format.
> 
> 
> Can we at least switch the default upload format to bz2? I would think
> that's a reasonably simple change to the robot? For those of us
> uploading stuff over a slow modem link, the extra efficiency would be
> much appreciated.
> 

No, I don't want to muck with working scripts.  I suggest setting up a 
script to upload to the /staging area and convert automatically.  I have 
put a script called bz2togz in /usr/local/bin to help out.

	-hpa

Re: Deprecating .gz format on kernel.org

[Martin J. Bligh]

> OK, there seems to be enough resistance to this to put it off for a year
> or two.  Since that means I don't have to do any work, that plan has
> inherent appeal to me.
> 
> In other words, the current setup will remain for now.  HOWEVER, I would
> like to recommend that mirror sites start carrying .bz2 files if they
> want to carry only one format.

Can we at least switch the default upload format to bz2? I would think
that's a reasonably simple change to the robot? For those of us
uploading stuff over a slow modem link, the extra efficiency would be
much appreciated.

M.

Re: Deprecating .gz format on kernel.org

[Jamie Lokier]

On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> i) Does this sound reasonable to everyone?  In particular, is there any
> loss in losing the "original" compressed files?

Personally I fetch the .bz2 tar files for a few base kernel versions,
but I fetch the .gz patch files.

This is so that I can "zgrep" through the patch files looking for
which version changed some feature or API.

bzgrep exists, but it is way too slow.

So if there were only .bz2 patch files, I would fetch them and convert
them back into .gz files on my local mirror.

Which is ok of course, but then the signatures don't match any more.

Well, that's my really weak reason for liking .gz patches.

enjoy,
-- Jamie

Re: Deprecating .gz format on kernel.org

[David S. Miller]

   From: "H. Peter Anvin" <hpa@zytor.com>
   Date: Wed, 19 Mar 2003 12:19:42 -0800

   Now, the questions that come up are:
   
   i) Does this sound reasonable to everyone?  In particular, is there any
   loss in losing the "original" compressed files?
   
   ii) Assuming a yes on the previous question, what time frame would it
   make sense for this changeover to happen over?

I'm fine with this, and my personal feeling on time is the sooner
the better.

It's pretty hard to find a Linux system without bzip2 these days.

Re: Deprecating .gz format on kernel.org

[Eric Sandall]

Kurt Garloff said:
> Hi,
>
> On Wed, Mar 19, 2003 at 06:55:52PM -0300, Arnaldo Carvalho de Melo
> wrote:
>> Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
>> > I can't speak for the others, but Red Hat Linux uses the .bz2 files
>> in kernel rpms
>>
>> Conectiva too.
>
> ... and so does SuSE.

And Source Mage (http://www.sourcemage.org).  :)

-Sandalle

-- 
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214

Eric Sandall                     |  Source Mage GNU/Linux Developer
eric@sandall.us                  |  http://www.sourcemage.org/
http://eric.sandall.us/          |  SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/  #196285  |  http://www.shock.wsu.edu/

Re: Deprecating .gz format on kernel.org

[DervishD]

    Hi HPA :)

 H. Peter Anvin dixit:
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.

    Just one point: while with kernel sources the space saving is
notable using .bz2 instead of .gz, in my experience with different
files, formats, etc... the saving is negligible and the speed just
sucks, gzip beats bzip2 for me in general.

    Anyway I think bzip2 is standard nowadays.

> i) Does this sound reasonable to everyone?  In particular, is there any
> loss in losing the "original" compressed files?

    IMHO, just the decompression speed. I've made no tests, but if
the kernel sources behave like other archives I have, .gz is faster.
OTOH, you save a few megs for downloading, and this is *very*
important too.

    Raúl Núñez de Arenas Coronado

-- 
Linux Registered User 88736
http://www.pleyades.net & http://www.pleyades.net/~raulnac

Re: Deprecating .gz format on kernel.org

[H. Peter Anvin]

H. Peter Anvin wrote:
> Hello everyone,
> 
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
> 

OK, there seems to be enough resistance to this to put it off for a year
or two.  Since that means I don't have to do any work, that plan has
inherent appeal to me.

In other words, the current setup will remain for now.  HOWEVER, I would
like to recommend that mirror sites start carrying .bz2 files if they
want to carry only one format.

	-hpa

Re: Deprecating .gz format on kernel.org

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 527 bytes --]

Hi,

On Wed, Mar 19, 2003 at 06:55:52PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
> > I can't speak for the others, but Red Hat Linux uses the .bz2 files in
> > kernel rpms
> 
> Conectiva too.

... and so does SuSE.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers                 SuSE Labs (Head)
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Deprecating .gz format on kernel.org

[Arnaldo Carvalho de Melo]

Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
> On Wed, 2003-03-19 at 22:12, Tigran Aivazian wrote:
> > On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> > a) any Linux distribution contains their own "linux" package with the 
> > source base being "vanilla" Linux .tar.gz file
 
> I can't speak for the others, but Red Hat Linux uses the .bz2 files in
> kernel rpms

Conectiva too.

- Arnaldo

Re: Deprecating .gz format on kernel.org

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 546 bytes --]

On Wed, 2003-03-19 at 22:12, Tigran Aivazian wrote:
> On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> > i) Does this sound reasonable to everyone?  In particular, is there any
> > loss in losing the "original" compressed files?
> 
> No, there is at least one reason for the "original" .gz files. Here are 
> the logical steps:
> 
> a) any Linux distribution contains their own "linux" package with the 
> source base being "vanilla" Linux .tar.gz file

I can't speak for the others, but Red Hat Linux uses the .bz2 files in
kernel rpms

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Deprecating .gz format on kernel.org

[Sam Ravnborg]

On Wed, Mar 19, 2003 at 12:19:42PM -0800, H. Peter Anvin wrote:
> Hello everyone,
> 
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.

It is a convinient feature that I can download the kernel now and then
when at work (no linux) - and it makes it a bit simpler to use
an archiver that has native support for the format used.
Winzip, being the only allowed archiver at work, does not
have native bz2 support.

I have a .bz2 commandline version on the windoze machine so I will cope,
but mainly to let you know that deprecating .gz you will make it
difficult for a minority to read the kernel src.

	Sam

Re: Deprecating .gz format on kernel.org

[Tigran Aivazian]

On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> i) Does this sound reasonable to everyone?  In particular, is there any
> loss in losing the "original" compressed files?

No, there is at least one reason for the "original" .gz files. Here are 
the logical steps:

a) any Linux distribution contains their own "linux" package with the 
source base being "vanilla" Linux .tar.gz file

b) switching such to .tar.bz2 will make building the package longer
because of longer extract times

c) re-running tar to generate a .tar.gz from .tar.bz2 and store the 
.tar.gz instead will make customers suspicious --- i.e. they will have to 
ask "is this _really_ a plain Linux tree or do I need to run diff(1) to 
verify, just in case?"

See the reasoning? However, I agree that this reason is very weak. But you
were interested in any reasons, including weak ones,  I assume :)

Regards
Tigran

> 
> ii) Assuming a yes on the previous question, what time frame would it
> make sense for this changeover to happen over?
> 
> 	-hpa
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Deprecating .gz format on kernel.org

[Mr. James W. Laferriere]

	Hello Peter ,  IMO it ain't going away '.' .  My systems will not
	carry .bz2's ,  that is a Fyi .  Not that it matters for systems
	over which you have opinion &/or control .  Thanks for listening .
			JimL

On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> Hello everyone,
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
> I am envisioning this as a three-phase changeover:
> a) Get all mirrors to carry .bz2 format.  This would affect the
> following sites:
-- 
       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: Deprecating .gz format on kernel.org

[Martin J. Bligh]

> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.

Yay!

> Now, the questions that come up are:
> 
> i) Does this sound reasonable to everyone?  In particular, is there any
> loss in losing the "original" compressed files?

Can't see why ... the duplication seems rather silly.

> ii) Assuming a yes on the previous question, what time frame would it
> make sense for this changeover to happen over?

As soon as you can get the mirrors to change over?

M.

Re: Deprecating .gz format on kernel.org

[wind]

On Wed, Mar 19, 2003 at 12:19:42PM -0800, H. Peter Anvin wrote:
> Hello everyone,
> 
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
> 
> I am envisioning this as a three-phase changeover:
> 
> a) Get all mirrors to carry .bz2 format.  This would affect the
> following sites:
> 
> DUTH:format=gz
> GARBO:format=gz
> HCMC:format=gz
> IGLU:format=gz
> LINUXAID:format=gz
> LLARIAN-NET:format=gz
> MINET-FR:format=gz
> NC-ORC:format=gz
> PCSS:format=gz
> PROGRAMVAREVERKSTEDET:format=gz
> PUB-FTP-UNIVERSITY-OF-OLDENBURG:format=gz
> RN-RNO:format=gz
> TASK:format=gz
> TELEPAC:format=gz
> TENGU-EASYNET-FR:format=gz
> UNC-METALAB:format=gz
> WEBLAB:format=gz
> 
> b) Once that is done, change the robots to no longer require .gz files;
> .bz2 files uploaded would be signed but no .gz file would be generated.
> 
> -> If we get a complete loss of data here, all .gz files would be lost.
> 
> c) At some point, deprecate .gz uploads entirely and remove all the old
> .gz files.  After that point .gz files uploaded would be treated just
> like .Z, .zip or any other "unmanaged" compression format.
> 
> 
> Now, the questions that come up are:
> 
> i) Does this sound reasonable to everyone?  In particular, is there any
> loss in losing the "original" compressed files?

I don't think phase three is really needed...

It sounds reasonable, but if it's mainly due to space consuption, I
think it would be nice to code a bzip2-to-gz cgi server so that there is
no loss on functionality. This would need CPU power, so perhaps
rate-limiting the sending side of the CGI would manage to keep
the CPU low on use.

> ii) Assuming a yes on the previous question, what time frame would it
> make sense for this changeover to happen over?

Don't know.

Greets anyways, Antonio.

Deprecating .gz format on kernel.org

[H. Peter Anvin]

Hello everyone,

At some point it probably would make sense to start deprecating .gz
format files from kernel.org.

I am envisioning this as a three-phase changeover:

a) Get all mirrors to carry .bz2 format.  This would affect the
following sites:

DUTH:format=gz
GARBO:format=gz
HCMC:format=gz
IGLU:format=gz
LINUXAID:format=gz
LLARIAN-NET:format=gz
MINET-FR:format=gz
NC-ORC:format=gz
PCSS:format=gz
PROGRAMVAREVERKSTEDET:format=gz
PUB-FTP-UNIVERSITY-OF-OLDENBURG:format=gz
RN-RNO:format=gz
TASK:format=gz
TELEPAC:format=gz
TENGU-EASYNET-FR:format=gz
UNC-METALAB:format=gz
WEBLAB:format=gz

b) Once that is done, change the robots to no longer require .gz files;
.bz2 files uploaded would be signed but no .gz file would be generated.

-> If we get a complete loss of data here, all .gz files would be lost.

c) At some point, deprecate .gz uploads entirely and remove all the old
.gz files.  After that point .gz files uploaded would be treated just
like .Z, .zip or any other "unmanaged" compression format.


Now, the questions that come up are:

i) Does this sound reasonable to everyone?  In particular, is there any
loss in losing the "original" compressed files?

ii) Assuming a yes on the previous question, what time frame would it
make sense for this changeover to happen over?

	-hpa