[CFT][PATCH] cleanup of partition code

[CFT][PATCH] cleanup of partition code

[Alexander Viro]

	Folks, _please_ help to test this one.  It switches most of
the fs/partitions/* to use of pagecache, cleans it up and fixes quite
a few holes in that area.

	It should work in all cases when vanilla tree does.  Please,
try it on different partitioning schemes.

	Things to look for:

* Odd IDE disks - victims of EZ-Disk, OnCrack, etc.
* Minix, Solaris, Unixware partitions.  Changes were completely
  straightforward, but it needs testing.
* BSD disklabels.  Should work, but more testing is needed.
* non-x86 variants - Sun disklables, SGI, etc.  Need testing.

	One thing that doesn't work yet is support of Acorn partitions -
I'm switching it to pagecache right now.

	If there's no bug reports it will go into the tree, so _please_
test it now, not after it's in 2.4.11

	Patch is on ftp.math.psu.edu/pub/viro/partition-a-S11-pre1.  It's
against 2.4.11-pre1.  Again, any help with testing is very welcome.

Re: [CFT][PATCH] cleanup of partition code

[Tom Rini]

On Sun, Sep 30, 2001 at 06:31:55PM -0400, Alexander Viro wrote:

> 	Folks, _please_ help to test this one.  It switches most of
> the fs/partitions/* to use of pagecache, cleans it up and fixes quite
> a few holes in that area.
> 
> 	It should work in all cases when vanilla tree does.  Please,
> try it on different partitioning schemes.

Both my MS-DOS partition table and HFS partition table still work here
on a PPC box.

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

Re: [CFT][PATCH] cleanup of partition code

[Alexander Viro]

Update:  fixed a double-free bug in amiga_partition().  After that thing
seems to be working on Amiga disk sent by Jes (as in, "right amount
of partitions and reasonably looking boundaries").

Patch is on ftp.math.psu.edu/pub/viro/partition-b-S11-pre1.

Re: [CFT][PATCH] cleanup of partition code

[Erik Andersen]

On Sun Sep 30, 2001 at 06:31:55PM -0400, Alexander Viro wrote:
> 
> 	One thing that doesn't work yet is support of Acorn partitions -
> I'm switching it to pagecache right now.

Well, acorn is broken anyways....  Try enabling in on a device
with native 2048 byte sectors and _no_ partition table will be
found on those devices (just an error msg resulting from acorn)

 -Erik

--
Erik B. Andersen   email:  andersee@debian.org, formerly of Lineo
--This message was written using 73% post-consumer electrons--

Re: [CFT][PATCH] cleanup of partition code

[Alexander Viro]

On Sun, 30 Sep 2001, Erik Andersen wrote:

> On Sun Sep 30, 2001 at 06:31:55PM -0400, Alexander Viro wrote:
> > 
> > 	One thing that doesn't work yet is support of Acorn partitions -
> > I'm switching it to pagecache right now.
> 
> Well, acorn is broken anyways....  Try enabling in on a device
> with native 2048 byte sectors and _no_ partition table will be
> found on those devices (just an error msg resulting from acorn)

Could you send me an example of such animal?  I don't mean the disk itself -
just the contents of relevant sectors (i.e. everything except the contents
of partitions themselves).

Re: [CFT][PATCH] cleanup of partition code

[Erik Andersen]

On Mon Oct 01, 2001 at 01:27:54AM -0400, Alexander Viro wrote:
> 
> 
> On Sun, 30 Sep 2001, Erik Andersen wrote:
> 
> > On Sun Sep 30, 2001 at 06:31:55PM -0400, Alexander Viro wrote:
> > > 
> > > 	One thing that doesn't work yet is support of Acorn partitions -
> > > I'm switching it to pagecache right now.
> > 
> > Well, acorn is broken anyways....  Try enabling in on a device
> > with native 2048 byte sectors and _no_ partition table will be
> > found on those devices (just an error msg resulting from acorn)
> 
> Could you send me an example of such animal?  I don't mean the disk itself -
> just the contents of relevant sectors (i.e. everything except the contents
> of partitions themselves).

Here is what I normally see (in this case with 2.4.9-ac17):

	scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.1
		<Adaptec 2940 Ultra SCSI adapter>
		aic7880: Ultra Single Channel A, SCSI Id=7, 16/255 SCBs

	(scsi0:A:4): 10.000MB/s transfers (10.000MHz, offset 15)
	  Vendor: OLYMPUS   Model: MOS364            Rev: 1.02
	  Type:   Optical Device                     ANSI SCSI revision: 02
	(scsi0:A:5): 10.000MB/s transfers (10.000MHz, offset 15)
	  Vendor: OLYMPUS   Model: MOS364            Rev: 1.02
	  Type:   Optical Device                     ANSI SCSI revision: 02
	[----------snip-------------]
	Attached scsi removable disk sda at scsi0, channel 0, id 4, lun 0
	Attached scsi removable disk sdb at scsi0, channel 0, id 5, lun 0
	SCSI device sda: 310352 2048-byte hdwr sectors (636 MB)
	sda: Write Protect is off
	 sda: sda1
	SCSI device sdb: 310352 2048-byte hdwr sectors (636 MB)
	sdb: Write Protect is off
	 sdb: sdb1

Everything looks fairly normal...  I have 2 640Meg SCSI magneto optical drives
with a single partiton on the media in each (working as expected).  Now lets
enable some stuff: 

	+CONFIG_ACORN_PARTITION=y
	+CONFIG_ACORN_PARTITION_ICS=y
	+CONFIG_ACORN_PARTITION_ADFS=y
	+CONFIG_ACORN_PARTITION_POWERTEC=y
	+CONFIG_ACORN_PARTITION_RISCIX=y


Now I see:

	Attached scsi removable disk sda at scsi0, channel 0, id 4, lun 0
	Attached scsi removable disk sdb at scsi0, channel 0, id 5, lun 0
	SCSI device sda: 310352 2048-byte hdwr sectors (636 MB)
	sda: Write Protect is off
	 sda:<5>ll_rw_block: device 08:00: only 2048-char blocks implemented (1024)
	 unable to read boot sectors / partition sectors
	SCSI device sdb: 310352 2048-byte hdwr sectors (636 MB)
	sdb: Write Protect is off
	 sdb:<5>ll_rw_block: device 08:10: only 2048-char blocks implemented (1024)
	 unable to read boot sectors / partition sectors

Note the ll_rw_block msg from where the acorn stuff is not reading in units
of the physical sector size?  Also notice the "unable to read..." msg, which
is where acorn chokes the partition table scanning...


So now, while fdisk is still able to see that partitions exist

	[andersen@dillweed andersen]$ fdisk -l /dev/sda
	Note: sector size is 2048 (not 512)

	Disk /dev/sda: 64 heads, 32 sectors, 151 cylinders
	Units = cylinders of 2048 * 2048 bytes

	   Device Boot    Start       End    Blocks   Id  System
	/dev/sda1   *         1       151    618432   83  Linux

the acorn stuff has caused the partition scan to abort prematurely, such that
proc partitions (and Linux) know nothing about the device's partitions.  I can
give you a dd from one of these disks, but I doubt that would show the error... 

 -Erik

--
Erik B. Andersen   email:  andersee@debian.org, formerly of Lineo
--This message was written using 73% post-consumer electrons--

Re: [CFT][PATCH] cleanup of partition code

[Alexander Viro]

On Mon, 1 Oct 2001, Erik Andersen wrote:

> Here is what I normally see (in this case with 2.4.9-ac17):

> 	 sda:<5>ll_rw_block: device 08:00: only 2048-char blocks implemented (1024)
> 	 unable to read boot sectors / partition sectors
> 	SCSI device sdb: 310352 2048-byte hdwr sectors (636 MB)
> 	sdb: Write Protect is off
> 	 sdb:<5>ll_rw_block: device 08:10: only 2048-char blocks implemented (1024)
> 	 unable to read boot sectors / partition sectors
 
> Note the ll_rw_block msg from where the acorn stuff is not reading in units
> of the physical sector size?  Also notice the "unable to read..." msg, which
> is where acorn chokes the partition table scanning...

Yes.  I've finished converting this sucker and I think I understand what's
going on there.  Reliance on block size being <= 1024 seems to be a
side effect of the implementation - not something fundamental.  I'll
try to feed the stuff you've sent into it and see if it works - new
code shouldn't care about the block size.

Re: [CFT][PATCH] cleanup of partition code

[Anton Altaparmakov]

Al,

On Sun, 30 Sep 2001, Alexander Viro wrote:
> Update:  fixed a double-free bug in amiga_partition().  After that thing
> seems to be working on Amiga disk sent by Jes (as in, "right amount
> of partitions and reasonably looking boundaries").
> 
> Patch is on ftp.math.psu.edu/pub/viro/partition-b-S11-pre1.

Just a random data point: your patch works fine here both with
msdos+extended partitions and with an LDM disk. 

I haven't tried the updated patch but as I don't have an amiga it
shouldn't make a difference on my systems anyway...

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [CFT][PATCH] cleanup of partition code

[Alexander Viro]

On Mon, 1 Oct 2001, Erik Andersen wrote:

> Note the ll_rw_block msg from where the acorn stuff is not reading in units
> of the physical sector size?  Also notice the "unable to read..." msg, which
> is where acorn chokes the partition table scanning...
> 
> 
> So now, while fdisk is still able to see that partitions exist
> 
> 	[andersen@dillweed andersen]$ fdisk -l /dev/sda
> 	Note: sector size is 2048 (not 512)
> 
> 	Disk /dev/sda: 64 heads, 32 sectors, 151 cylinders
> 	Units = cylinders of 2048 * 2048 bytes
> 
> 	   Device Boot    Start       End    Blocks   Id  System
> 	/dev/sda1   *         1       151    618432   83  Linux
> 
> the acorn stuff has caused the partition scan to abort prematurely, such that
> proc partitions (and Linux) know nothing about the device's partitions.  I can
> give you a dd from one of these disks, but I doubt that would show the error... 

	OK, first of all, it's _not_ an acorn partition table at all.
It's a garden-variety DOS partition table.

	Actually, you've found a rather nasty bug in acorn.c - code in
the current tree fails if it tries to look for acorn-style partition
table on a large-sector disk.  Fails with IO error, and reports that
to high-level code in check_partitions().  Which decides to stop.
msdos_partition() would be called after acorn_partition(), so it
doesn't get called at all.

	Lovely...  OK, there are two possible fixes - one is to
add check for block size into acorn_partition() (it's checked on
almost all branches, but there's one where it's missing).  Another
is to switch to new partition code, which works with any physical
sector size.

	I'm putting the new patch on anonftp -
ftp.math.psu.edu/pub/viro/partition-c-S11-pre1

	News:
* massaged into form that should be easy to backport.
* acorn.c converted (_completely_ untested)

	Folks, please help to test that sucker. 

Re: [CFT][PATCH] cleanup of partition code

[Nick Craig-Wood]

In linux-kernel, viro@math.psu.edu wrote:
> 	Actually, you've found a rather nasty bug in acorn.c - code in
> the current tree fails if it tries to look for acorn-style partition
> table on a large-sector disk.

Just for the record - I don't think Acorn disks can have sectors
bigger than 1k.  (I never managed to get the 640Mb MO discs working, I
had to get the 512 byte sectored 512 Mb ones instead.)  I just checked
in the PRM and it agrees with my very rusty memory.

I have some acorn partitioned MO discs lying around if you want to see
the first few sectors off the disc...

-- 
Nick Craig-Wood
ncw@axis.demon.co.uk

Re: [CFT][PATCH] cleanup of partition code

[Alexander Viro]

Update:  acorn-style partitions seem to be working now, fixed an idiotic
bug in read_dev_sector() (error recovery path), did initial backport to -ac.

Patch is on ftp.math.psu.edu/pub/viro/partition-d-S11-pre1,
-ac variant is on ftp.math.psu.edu/pub/viro/partition-a-AC10-1.

Re: [CFT][PATCH] cleanup of partition code

[Alex Bligh - linux-kernel]

--On Monday, 01 October, 2001 10:06 AM +0100 Nick Craig-Wood 
<ncw@axis.demon.co.uk> wrote:

> I don't think Acorn disks can have sectors
> bigger than 1k.

>From a very long time in the past when I used
to write interfaces to ARM stuff, I am /pretty/
sure I managed to get 4k sectors to work. As
this is about 10 years ago, and involved
some editing of some grungy BASIC HD format
program, I may be either wrong, or out of
date.

--
Alex Bligh

Re: [CFT][PATCH] cleanup of partition code

[Erik Andersen]

On Mon Oct 01, 2001 at 04:03:45AM -0400, Alexander Viro wrote:
> 
> 	OK, first of all, it's _not_ an acorn partition table at all.
> It's a garden-variety DOS partition table.
> 
> 	Actually, you've found a rather nasty bug in acorn.c - code in

Looks like your patches were folded into 2.4.10-ac3 -- and I'm pleased
to say that your patch has indeed fixed the problem.  I just scratched
fixing acorn off my when-I-get-around-to-it list.  Thanks!

 -Erik

--
Erik B. Andersen   email:  andersee@debian.org, formerly of Lineo
--This message was written using 73% post-consumer electrons--

Linux kernel 2.4 and TCP terminations per second.

[Imran Badr]

Does anybody know , what is the maximum number of TCP (http)
terminations/per second a server (single/dual/.. processor)  in todays
market can do, without much CPU load. The server would be running linux
kernel 2.4 and apache web server.

Thanks,
Imran.

Re: Linux kernel 2.4 and TCP terminations per second.

[Alan Cox]

> Does anybody know , what is the maximum number of TCP (http)
> terminations/per second a server (single/dual/.. processor)  in todays
> market can do, without much CPU load. The server would be running linux
> kernel 2.4 and apache web server.

If you are running any kind of high performance connections/second load then
you dont run apache. That isnt what apache is good at

thttpd will do 2000/sec on a decent box. zeus (non free) more, and tux
(kernel http accelerator) holds some records

RE: Linux kernel 2.4 and TCP terminations per second.

[Imran Badr]

I am running openssl with apache using modssl. I will have to look at
whether could I use openssl with TUX or zeus.

Thanks.

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Alan Cox
Sent: Tuesday, November 06, 2001 3:22 PM
To: imran.badr@cavium.com
Cc: linux-kernel@vger.kernel.org
Subject: Re: Linux kernel 2.4 and TCP terminations per second.


> Does anybody know , what is the maximum number of TCP (http)
> terminations/per second a server (single/dual/.. processor)  in todays
> market can do, without much CPU load. The server would be running linux
> kernel 2.4 and apache web server.

If you are running any kind of high performance connections/second load then
you dont run apache. That isnt what apache is good at

thttpd will do 2000/sec on a decent box. zeus (non free) more, and tux
(kernel http accelerator) holds some records
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux kernel 2.4 and TCP terminations per second.

[Roland Dreier]

    Imran> I am running openssl with apache using modssl. I will have
    Imran> to look at whether could I use openssl with TUX or zeus.

If you are doing SSL termination without a hardware crypto accelerator
then the cost of the public key operations for the SSL handshake will
far outweigh the cost of TCP termination and the webserver.  With a
typical machine (say a 1 GHz P3) I would estimate you could do 200 SSL
handshakes/sec with apache/modssl (with 95% of your CPU time spent in
OpenSSL RSA code).  With a hardware crypto accelerator you could get
up to 600-1000 handshakes/sec but the crypto will still be the
bottleneck.

Roland

Re: Linux kernel 2.4 and TCP terminations per second.

[David Lang]

from a recent test I just was running with apache on a 1.2GHZ athlon 512MB
ram it looks like it will do ~1800 connections/sec.

just to put the numbers below in perspective :-)

David Lang

On Tue, 6 Nov 2001, Alan Cox wrote:

> > Does anybody know , what is the maximum number of TCP (http)
> > terminations/per second a server (single/dual/.. processor)  in todays
> > market can do, without much CPU load. The server would be running linux
> > kernel 2.4 and apache web server.
>
> If you are running any kind of high performance connections/second load then
> you dont run apache. That isnt what apache is good at
>
> thttpd will do 2000/sec on a decent box. zeus (non free) more, and tux
> (kernel http accelerator) holds some records
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

RE: Linux kernel 2.4 and TCP terminations per second.

[Imran Badr]

I am not worried about SSL handshakes/sec. I am looking forward to how much
http/sec I could extract out of a moderate linux box. I have a dell box with
PIII 800 MHz, 256 MB RAM running a very light weight server (which I wrote)
and I wanted to find out how much time does it spend in system for TCP
terminations. The system in running linux kernel version 2.4.2

The server sits in a loop and launches worker threads as it receives a tcp
socket connection. The worker thread then sits in an infinite loop and
sends/receives data from client (which is again a custom client).
I figured out that most of the CPU time was spent in the kernel mode for TCP
terminatiions and Network was not the bottleneck.



-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Roland Dreier
Sent: Tuesday, November 06, 2001 3:44 PM
To: imran.badr@cavium.com
Cc: linux-kernel@vger.kernel.org
Subject: Re: Linux kernel 2.4 and TCP terminations per second.


    Imran> I am running openssl with apache using modssl. I will have
    Imran> to look at whether could I use openssl with TUX or zeus.

If you are doing SSL termination without a hardware crypto accelerator
then the cost of the public key operations for the SSL handshake will
far outweigh the cost of TCP termination and the webserver.  With a
typical machine (say a 1 GHz P3) I would estimate you could do 200 SSL
handshakes/sec with apache/modssl (with 95% of your CPU time spent in
OpenSSL RSA code).  With a hardware crypto accelerator you could get
up to 600-1000 handshakes/sec but the crypto will still be the
bottleneck.

Roland

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Linux kernel 2.4 and TCP terminations per second.

[Alan Cox]

> from a recent test I just was running with apache on a 1.2GHZ athlon 512MB
> ram it looks like it will do ~1800 connections/sec.
> 
> just to put the numbers below in perspective :-)

I was doing 2000 a second on a P2/233 just to keep the perspective. I've
not yet hacked thttpd to try Linus new readahead syscall