archive mirror
 help / color / mirror / Atom feed
From: "Alan Shih" <>
To: "David griego" <>, <>
Cc: <>, <>
Subject: RE: Alan Shih: "TCP IP Offloading Interface"
Date: Mon, 14 Jul 2003 13:31:06 -0700	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

In a typical application of NAS ASIC, it is easier to place 2 300MHz
processor than a 1 GHz processor in cost. This line of reasoning forces me
to consider making 1 of the processor to be TOE while the other one deals
with disk/FS manipulations.


-----Original Message-----
From: David griego []
Sent: Monday, July 14, 2003 1:19 PM
Subject: Re: Alan Shih: "TCP IP Offloading Interface"

Embedded does not simply include toasters and fridges, it also includes NAS
and SAN appliances as well as telco gear.  These types of devices have
advanced memory subsystems and run processors such as PPC and ARM.  One of
the most limiting factors in these types of devices is power consumption.
This usually limits the number of cores and frequency these cores.
Offloading the processing of protocol stacks to ASICS would have a great
impact in performance.  If you are going to embed a high frequency chip in
your embedded devices I would recommend developing a heater not a fridge.

>From: Alan Cox <>
>To: David griego <>
>CC:,   Linux Kernel Mailing List
>Subject: Re: Alan Shih: "TCP IP Offloading Interface"
>Date: 14 Jul 2003 20:42:53 +0100
>On Llu, 2003-07-14 at 19:46, David griego wrote:
> > IMHO, there are several cases for some type of TCP/IP offload.  One is
> > embedded systems that are just not capable of doing 1Gbps+.  Another is
>My fridge doesn't need to do 10Gbit a second, and for most other
>embedded the constraints are ram bandwidth and nothing else. Since
>deeply embedded stuff also doesn't run with MMUs or runs 'partially
>trusted' most of the VM games and the socket api games also go away.

See PPC and ARM architecture for the use of MMUs in embedded systems

>I've done deeply embedded tcp/ip. I don't buy the argument, embedded
>gains the least of all from ToE.
> > 10GbE, even high end servers will not be able keep up with TCP
> > processing/data movement at these speeds.  Not being proactive in
>They said that about 10Mbit until Van showed them a thing or two. They
>said it about 100Mbit, they said it about gigabit.

Not the case for embedded.  I understand your viewpoint from the server
space though.
> > TCP/IP offload will force Linux into accepting some scheme that will not
> > necessarily be best.
>TCP/IP is an exercise in two things when you are running at speed
>1.	Finding the memory bandwidth - ToE doesn't help, checksums do,
>	sg does, on card target buffers do with decent chipsets.

A TOE enabled with RDDP would help eliminate the kernel to user space copy
(and in the case of SAMBA the copy back to the kernel).  This would reduce
the memory system loading by a third to a half.
>2.	Handling in order perfectly predicted data streams. ToE is
>	overkill for this. Thats about latency to memory and touching
>	as little as possible. The main CPU has a rather good connection
>	to main memory.
Yes, RDDP would be nice to have though for the reason stated for #1, so the
hardware would need to at least be TCP aware.

>ToE is also horribly vulnerable to attack because putting it on a card
>dictates relatively low CPU power and low power consumption as well as
>rather nasty pricing issues. Historically low power devices have
>repeatedly been screwed by attackers hitting software or other slow
>paths in the device to attack it.
The use of ASICs could ensure that TCP processing is as quick as wire speed

>This is before we get into the delights of multipath routing across
>different vendors cards, firewalling, traffic shaping, retrofitting new
>features, questions about what happens with an old ToE card when its
>got a hole...
Try to keep the datapath processing on the TOE, and everything else in the
OS.  Also give the API the ability to turn of the TOE if a hole exists and
use it like a regular NIC.
>The internet land speed record is held by a non ToE system, let me know
>when that changes.
Layer one network processing is often handled by ASICS, also some of the
fastest encryption engines are hardware.  I suggest we don't wait until your
proven wrong before making a decision on TOE.

MSN 8 helps eliminate e-mail viruses. Get 2 months FREE*.

  reply	other threads:[~2003-07-14 20:23 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2003-07-14 20:19 Alan Shih: "TCP IP Offloading Interface" David griego
2003-07-14 20:31 ` Alan Shih [this message]
2003-07-16 22:28   ` Error compiling, scsi 2.6.0-test1 backblue
2003-07-16 23:09     ` Rahul Karnik
2003-07-17 11:51     ` Alan Cox
2003-07-17 18:59       ` backblue
2003-07-18 16:14         ` Joshua Schmidlkofer
2003-07-18 17:00         ` Steven Cole
2003-07-24  7:51           ` Doug Ledford
2003-07-14 20:34 ` Alan Shih: "TCP IP Offloading Interface" Alan Cox
2003-07-14 21:53   ` Deepak Saxena
2003-07-17 22:31     ` Bill Davidsen
     [not found] <>
2003-07-15 11:18 ` Alan Cox
  -- strict thread matches above, loose matches on Subject: below --
2003-07-14 21:51 David griego
2003-07-14 20:29 David griego
2003-07-14 20:23 David griego
2003-07-14 19:43 David griego
2003-07-14 20:03 ` Jeff Garzik
2003-07-14 20:23   ` Alan Cox
2003-07-14 20:05 ` Alan Cox
2003-07-14 20:30   ` Shawn
2003-07-15  5:58   ` Werner Almesberger
2003-07-14 19:14 David griego
2003-07-14 19:26 ` Jeff Garzik
2003-07-15 12:42   ` Jesse Pollard
2003-07-14 19:46 ` Alan Cox
2003-07-14 18:46 David griego
2003-07-14 19:02 ` Jeff Garzik
2003-07-14 21:22   ` Deepak Saxena
2003-07-14 21:45     ` Jeff Garzik
2003-07-15  5:27     ` Werner Almesberger
2003-07-14 19:42 ` Alan Cox

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).