ata over ethernet question

ata over ethernet question

[Maciej Soltysiak]

Hi,

AOE is a bit new for me.

Would it be possible to use tha AOE driver to
attach one ATA drive in a host over ethernet to another
host ? Or is it support for specific hardware devices only?

You know, something like:
# fdisk <device_on_another_host>
# mkfs.ext2 <device_on_another_host/partition1>
# mount <device_on_another_host/partition1> /mnt/part1

--
Maciej

Re: ata over ethernet question

[David Hollis]

[-- Attachment #1: Type: text/plain, Size: 1121 bytes --]

On Wed, 2005-05-04 at 19:31 +0200, Maciej Soltysiak wrote:
> Hi,
> 
> AOE is a bit new for me.
> 
> Would it be possible to use tha AOE driver to
> attach one ATA drive in a host over ethernet to another
> host ? Or is it support for specific hardware devices only?
> 
> You know, something like:
> # fdisk <device_on_another_host>
> # mkfs.ext2 <device_on_another_host/partition1>
> # mount <device_on_another_host/partition1> /mnt/part1
> 

That seems to be the basic idea but there doesn't seem to be a provider
stack just yet, just a 'client' (though I could be wrong).  AOE is
similar in concept to iSCSI with the biggest difference being that AOE
runs over Ethernet and is thus non-routeable.  iSCSI operates over IP so
you can do all kinds of fun IP games with it.

> --
> Maciej
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
-- 
David Hollis <dhollis@davehollis.com>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re[2]: ata over ethernet question

[Maciej Soltysiak]

Hello David,

Wednesday, May 4, 2005, 9:48:36 PM, you wrote:
> That seems to be the basic idea but there doesn't seem to be a provider
> stack just yet, just a 'client' (though I could be wrong).  AOE is
> similar in concept to iSCSI with the biggest difference being that AOE
> runs over Ethernet and is thus non-routeable.  iSCSI operates over IP so
> you can do all kinds of fun IP games with it.
Thanks, this is interesting. Does the iSCSI implementation out there have
this provider stack ?

Regards,
Maciej

Re: Re[2]: ata over ethernet question

[Raf D'Halleweyn]

On Wed, 2005-05-04 at 23:17 +0200, Maciej Soltysiak wrote:
> Hello David,
> 
> Wednesday, May 4, 2005, 9:48:36 PM, you wrote:
> > That seems to be the basic idea but there doesn't seem to be a provider
> > stack just yet, just a 'client' (though I could be wrong).  AOE is
> > similar in concept to iSCSI with the biggest difference being that AOE
> > runs over Ethernet and is thus non-routeable.  iSCSI operates over IP so
> > you can do all kinds of fun IP games with it.

The aoetools project at http://sourceforge.net/projects/aoetools/
includes the package vblade:

"This is a beta release of vblade, the virtual EtherDrive (R) blade.
The vblade is a program that makes a seekable file available over an
ethernet local area network (LAN) via the ATA over Ethernet (AoE)
protocol."

PS. This month's Linux Journal has an article about AoE.

Raf.

Re: ata over ethernet question

[Alan Cox]

On Mer, 2005-05-04 at 18:31, Maciej Soltysiak wrote:
> Would it be possible to use tha AOE driver to
> attach one ATA drive in a host over ethernet to another
> host ? Or is it support for specific hardware devices only?

It talks ATA command blocks but if you write an ATA command block parser
you can write yourself a remote device stack and the protocol is very
simple. For block storage NBD is probably simpler and more efficient but
AoE might be interesting for accessing embedded microcontrollers and the
like because you don't need a TCP/IP stack.

Alan

Re: Re[2]: ata over ethernet question

[David Hollis]

[-- Attachment #1: Type: text/plain, Size: 1018 bytes --]

On Wed, 2005-05-04 at 23:17 +0200, Maciej Soltysiak wrote:
> Hello David,
> 
> Wednesday, May 4, 2005, 9:48:36 PM, you wrote:
> > That seems to be the basic idea but there doesn't seem to be a provider
> > stack just yet, just a 'client' (though I could be wrong).  AOE is
> > similar in concept to iSCSI with the biggest difference being that AOE
> > runs over Ethernet and is thus non-routeable.  iSCSI operates over IP so
> > you can do all kinds of fun IP games with it.
> Thanks, this is interesting. Does the iSCSI implementation out there have
> this provider stack ?
> 
> Regards,
> Maciej

There seem to be a few iSCSI implementations floating around for Linux,
hopefully one will be added to mainline soon.  Most of those
implementations are for the client side though there is at least one
target implementation that allows you to provide local storage to iSCSI
clients.  I don't remember the name of it or if it's still maintained or
not.

-- 
David Hollis <dhollis@davehollis.com>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Re[2]: ata over ethernet question

[Sander]

David Hollis wrote (ao):
> There seem to be a few iSCSI implementations floating around for
> Linux, hopefully one will be added to mainline soon. Most of those
> implementations are for the client side though there is at least one
> target implementation that allows you to provide local storage to
> iSCSI clients. I don't remember the name of it or if it's still
> maintained or not.

Quite active even:

http://sourceforge.net/projects/iscsitarget/

The "Quick Guide to iSCSI on Linux" is a good starting point btw.

Also check out http://www.open-iscsi.org/ (the client, aka 'initiator').

-- 
Humilis IT Services and Solutions
http://www.humilis.net

Re: Re[2]: ata over ethernet question

[Guennadi Liakhovetski]

Hi

On Sat, 7 May 2005, Sander wrote:

> David Hollis wrote (ao):
> > There seem to be a few iSCSI implementations floating around for
> > Linux, hopefully one will be added to mainline soon. Most of those
> > implementations are for the client side though there is at least one
> > target implementation that allows you to provide local storage to
> > iSCSI clients. I don't remember the name of it or if it's still
> > maintained or not.
> 
> Quite active even:
> 
> http://sourceforge.net/projects/iscsitarget/
> 
> The "Quick Guide to iSCSI on Linux" is a good starting point btw.
> 
> Also check out http://www.open-iscsi.org/ (the client, aka 'initiator').

A follow up question - I recently used nbd to access a CD-ROM. It worked 
nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
had to stop the client, the server, then replace the CD, re-start the 
server, re-start the client... I thought about extending NBD to (better) 
support removable media, but then you start thinking about all those 
features that your local block device has that don't get exported over 
NBD...

Now, my understanding (sorry, without looking at any docs - yet) is, that 
iSCSI is (or at least should be) free from these limitations. So, does it 
make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
just kept simple as it is or would it be completely superseeded by iSCSI, 
or is there still something that NBD does that iSCSI wouldn't (easily) do?

Or am I completely misunderstanding what iSCSI target does?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: ata over ethernet question

[Vladislav Bolkhovitin]

Guennadi Liakhovetski wrote:
> Hi
> 
> On Sat, 7 May 2005, Sander wrote:
> 
> 
>>David Hollis wrote (ao):
>>
>>>There seem to be a few iSCSI implementations floating around for
>>>Linux, hopefully one will be added to mainline soon. Most of those
>>>implementations are for the client side though there is at least one
>>>target implementation that allows you to provide local storage to
>>>iSCSI clients. I don't remember the name of it or if it's still
>>>maintained or not.
>>
>>Quite active even:
>>
>>http://sourceforge.net/projects/iscsitarget/
>>
>>The "Quick Guide to iSCSI on Linux" is a good starting point btw.
>>
>>Also check out http://www.open-iscsi.org/ (the client, aka 'initiator').
> 
> 
> A follow up question - I recently used nbd to access a CD-ROM. It worked 
> nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> had to stop the client, the server, then replace the CD, re-start the 
> server, re-start the client... I thought about extending NBD to (better) 
> support removable media, but then you start thinking about all those 
> features that your local block device has that don't get exported over 
> NBD...
> 
> Now, my understanding (sorry, without looking at any docs - yet) is, that 
> iSCSI is (or at least should be) free from these limitations. So, does it 
> make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> just kept simple as it is or would it be completely superseeded by iSCSI, 
> or is there still something that NBD does that iSCSI wouldn't (easily) do?
> 
> Or am I completely misunderstanding what iSCSI target does?

Actually, this is property not of iSCSI target itself, but of any SCSI 
target. So, we implemented it as part of our SCSI target mid-level 
(SCST, http://scst.sourceforge.net), therefore any target driver working 
over it will automatically benefit from this feature. Unfortunately, 
currently available only target drivers for Qlogic 2x00 cards and for 
poor UNH iSCSI target (that works not too reliable and only with very 
specific initiators). The published version supports only real SCSI 
CDROMs. CDROM FILEIO module, which allows exporting ISO images as SCSI 
CDROM devices, going to be available not later end of May.

Vlad

> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: several messages

[Guennadi Liakhovetski]

Hello and thanks for the replies

On Wed, 11 May 2005, FUJITA Tomonori wrote:
> The iSCSI protocol simply encapsulates the SCSI protocol into the
> TCP/IP protocol, and carries packets over IP networks. You can handle
...

On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> Actually, this is property not of iSCSI target itself, but of any SCSI target.
> So, we implemented it as part of our SCSI target mid-level (SCST,
> http://scst.sourceforge.net), therefore any target driver working over it will
> automatically benefit from this feature. Unfortunately, currently available
> only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
> works not too reliable and only with very specific initiators). The published
...

The above confirms basically my understanding apart from one "minor" 
confusion - I thought, that parallel to hardware solutions pure software 
implementations were possible / being developed, like a driver, that 
implements a SCSI LDD API on one side, and forwards packets to an IP 
stack, say, over an ethernet card - on the initiator side. And a counter 
part on the target side. Similarly to the USB mass-storage and storage 
gadget drivers?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: several messages

[Ming Zhang]

[-- Attachment #1: Type: text/plain, Size: 1703 bytes --]

iscsi is scsi over ip.
usb disk is scsi over usb.
so just a different transport.
u are rite. ;)

ming

On Wed, 2005-05-11 at 23:26 +0200, Guennadi Liakhovetski wrote:
> Hello and thanks for the replies
> 
> On Wed, 11 May 2005, FUJITA Tomonori wrote:
> > The iSCSI protocol simply encapsulates the SCSI protocol into the
> > TCP/IP protocol, and carries packets over IP networks. You can handle
> ...
> 
> On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> > Actually, this is property not of iSCSI target itself, but of any SCSI target.
> > So, we implemented it as part of our SCSI target mid-level (SCST,
> > http://scst.sourceforge.net), therefore any target driver working over it will
> > automatically benefit from this feature. Unfortunately, currently available
> > only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
> > works not too reliable and only with very specific initiators). The published
> ...
> 
> The above confirms basically my understanding apart from one "minor" 
> confusion - I thought, that parallel to hardware solutions pure software 
> implementations were possible / being developed, like a driver, that 
> implements a SCSI LDD API on one side, and forwards packets to an IP 
> stack, say, over an ethernet card - on the initiator side. And a counter 
> part on the target side. Similarly to the USB mass-storage and storage 
> gadget drivers?
> 
> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: several messages

[Vladislav Bolkhovitin]

Guennadi Liakhovetski wrote:
> Hello and thanks for the replies
> 
> On Wed, 11 May 2005, FUJITA Tomonori wrote:
> 
>>The iSCSI protocol simply encapsulates the SCSI protocol into the
>>TCP/IP protocol, and carries packets over IP networks. You can handle
> 
> ...
> 
> On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> 
>>Actually, this is property not of iSCSI target itself, but of any SCSI target.
>>So, we implemented it as part of our SCSI target mid-level (SCST,
>>http://scst.sourceforge.net), therefore any target driver working over it will
>>automatically benefit from this feature. Unfortunately, currently available
>>only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
>>works not too reliable and only with very specific initiators). The published
> 
> ...
> 
> The above confirms basically my understanding apart from one "minor" 
> confusion - I thought, that parallel to hardware solutions pure software 
> implementations were possible / being developed, like a driver, that 
> implements a SCSI LDD API on one side, and forwards packets to an IP 
> stack, say, over an ethernet card - on the initiator side. And a counter 
> part on the target side. Similarly to the USB mass-storage and storage 
> gadget drivers?

There is some confusion in the SCSI world between SCSI as a transport 
and SCSI as a commands set and software communication protocol, which 
works above the transport. So, you can implement SCSI transport at any 
software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
etc.) way, but if the SCSI message passing protocol is used overall 
system remains SCSI with all protocol obligations like task management. 
So, pure software SCSI solution is possible. BTW, there are pure 
hardware iSCSI implementations as well.

Vlad

> Thanks
> Guennadi
> ---
> Guennadi Liakhovetski
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: several messages

[Dmitry Yusupov]

On Wed, 2005-05-11 at 22:16 -0400, Ming Zhang wrote:
> iscsi is scsi over ip.

correction. iSCSI today has RFC at least for two transports - TCP/IP and
iSER/RDMA(in finalized progress) with RDMA over Infiniband or RNIC. And
I think people start writing initial draft for SCTP/IP transport...

>From this perspective, iSCSI certainly more advanced and matured
comparing to NBD variations. 

> usb disk is scsi over usb.
> so just a different transport.
> u are rite. ;)
> 
> ming
> 
> On Wed, 2005-05-11 at 23:26 +0200, Guennadi Liakhovetski wrote:
> > Hello and thanks for the replies
> > 
> > On Wed, 11 May 2005, FUJITA Tomonori wrote:
> > > The iSCSI protocol simply encapsulates the SCSI protocol into the
> > > TCP/IP protocol, and carries packets over IP networks. You can handle
> > ...
> > 
> > On Wed, 11 May 2005, Vladislav Bolkhovitin wrote:
> > > Actually, this is property not of iSCSI target itself, but of any SCSI target.
> > > So, we implemented it as part of our SCSI target mid-level (SCST,
> > > http://scst.sourceforge.net), therefore any target driver working over it will
> > > automatically benefit from this feature. Unfortunately, currently available
> > > only target drivers for Qlogic 2x00 cards and for poor UNH iSCSI target (that
> > > works not too reliable and only with very specific initiators). The published
> > ...
> > 
> > The above confirms basically my understanding apart from one "minor" 
> > confusion - I thought, that parallel to hardware solutions pure software 
> > implementations were possible / being developed, like a driver, that 
> > implements a SCSI LDD API on one side, and forwards packets to an IP 
> > stack, say, over an ethernet card - on the initiator side. And a counter 
> > part on the target side. Similarly to the USB mass-storage and storage 
> > gadget drivers?
> > 
> > Thanks
> > Guennadi
> > ---
> > Guennadi Liakhovetski
> > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re[2]: ata over ethernet question

[James Bottomley]

On Wed, 2005-05-11 at 00:00 +0200, Guennadi Liakhovetski wrote:
> A follow up question - I recently used nbd to access a CD-ROM. It worked 
> nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> had to stop the client, the server, then replace the CD, re-start the 
> server, re-start the client... I thought about extending NBD to (better) 
> support removable media, but then you start thinking about all those 
> features that your local block device has that don't get exported over 
> NBD...

That's correct; NBD is basically just a remote data pipe type block
device.  It doesn't understand arbitrary packet commands.

> Now, my understanding (sorry, without looking at any docs - yet) is, that 
> iSCSI is (or at least should be) free from these limitations. So, does it 
> make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> just kept simple as it is or would it be completely superseeded by iSCSI, 
> or is there still something that NBD does that iSCSI wouldn't (easily) do?

Caveat: I've done quite a bit of work on nbd, so I'm biased.  However,
for what it does, nbd is extremely small, simple and efficient, so I
think we'd want a hole in our head to replace it with something as
complex and bloated as iSCSI---remember we'd need both a target and an
initiator to do what nbd does today.

However, there is room for improvement in nbd, notably the handling of
packet commands, which looks to be eminently doable in the current
infrastructure (this would basically make nbd a replicator for the linux
block system, and would probably necessitate some client side changes to
achieve).  If you have any thoughts in this direction, you could drop an
email to the maintainer.

James

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

On Thu, 2005-05-12 at 14:52 -0400, James Bottomley wrote:
> On Wed, 2005-05-11 at 00:00 +0200, Guennadi Liakhovetski wrote:
> > A follow up question - I recently used nbd to access a CD-ROM. It worked 
> > nice, but, I had to read in 7 CDs, so, each time I had to replace a CD, I 
> > had to stop the client, the server, then replace the CD, re-start the 
> > server, re-start the client... I thought about extending NBD to (better) 
> > support removable media, but then you start thinking about all those 
> > features that your local block device has that don't get exported over 
> > NBD...
> 
> That's correct; NBD is basically just a remote data pipe type block
> device.  It doesn't understand arbitrary packet commands.
> 
> > Now, my understanding (sorry, without looking at any docs - yet) is, that 
> > iSCSI is (or at least should be) free from these limitations. So, does it 
> > make any sense at all extending NBD or just switch to iSCSI? Should NBD be 
> > just kept simple as it is or would it be completely superseeded by iSCSI, 
> > or is there still something that NBD does that iSCSI wouldn't (easily) do?
> 
> Caveat: I've done quite a bit of work on nbd, so I'm biased.  However,
> for what it does, nbd is extremely small, simple and efficient, so I
> think we'd want a hole in our head to replace it with something as
> complex and bloated as iSCSI---remember we'd need both a target and an
> initiator to do what nbd does today.

oh, please! don't compare nbd and iSCSI this way...
iSCSI is an emerging SAN technology, and the only technology to compare
is FC.

> However, there is room for improvement in nbd, notably the handling of
> packet commands, which looks to be eminently doable in the current
> infrastructure (this would basically make nbd a replicator for the linux
> block system, and would probably necessitate some client side changes to
> achieve).  If you have any thoughts in this direction, you could drop an
> email to the maintainer.
> 
> James
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re[2]: ata over ethernet question

[James Bottomley]

On Thu, 2005-05-12 at 12:05 -0700, Dmitry Yusupov wrote:
> oh, please! don't compare nbd and iSCSI this way...
> iSCSI is an emerging SAN technology, and the only technology to compare
> is FC.

Well, the question was whether iSCSI could replace nbd; It's rather
difficult to answer that question by comparing iSCSI to FC ...

But even projecting to iSCSI being totally mature, the amount of code
required to conform to the iSCSI standard is easily going to put it 10x
over the amount of code we have in nbd, principally because they're
aimed at solving different problems and nbd achieves a lot of
streamlining by being tied to the linux block subsystem instead of
trying to be a generic transport.

James

Re: SCSI/ISCSI, hardware/software

[Bryan Henderson]

>There is some confusion in the SCSI world between SCSI as a transport 
>and SCSI as a commands set and software communication protocol, which 
>works above the transport. So, you can implement SCSI transport at any 
>software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
>etc.) way, but if the SCSI message passing protocol is used overall 
>system remains SCSI with all protocol obligations like task management. 

The above doesn't really resolve the confusion, since it uses some 
ambiguous terms and constructions.  I'm not sure what it's supposed to 
say, but let me try to state in the terminology of the SCSI standards what 
SCSI is:

SCSI is a family of separate specifications.  Some are specifications of 
transports, and others are specifications of command sets (a layer above 
the transports).  A SCSI device must implement a SCSI transport spec and a 
SCSI command set spec -- and also contain a piece that actually does the 
work (e.g. a disk drive), the details of which aren't specified by SCSI.

Examples of SCSI transport specification are (I'm paraphrasing the names) 
parallel SCSI, Fibre Channel, and ISCSI.  Examples of command sets are the 
disk device command set and the tape device command set.

>So, pure software SCSI solution is possible. BTW, there are pure 
>hardware iSCSI implementations as well.

I don't think it's even meaningful to talk about whether an implementation 
is hardware or software.  The "pure hardware" implementations contain 
megabytes of software, which was written in languages like C, contains 
operating systems like Linux, and can be transmitted across a network and 
updated easily.  The "pure software" implementation involve kilograms of 
hardware in every SCSI command -- CPUs, power supplies, etc.

Not only that, but the "all hardware" ISCSI initiators people talk about, 
which are PCI cards with Ethernet jacks, are not complete initiators.  The 
computer you plug the card into, on which you run Linux and some 
application programs, is the initiator.  The card is just the 
ISCSI-specific core of it.

There's really two distinctions people mean to make when talking about 
hardware vs software:

1) Is it preassembled?  Can you lift it out of box whole, or do you have 
to acquire some special software and some more generic parts separately 
and manage their combination?

2) Does it involve a general purpose computing system, particularly one 
that you share with some other computing, or a faster special purpose 
dedicated one?

In the context of a Linux SCSI discussion, I'd just talk about how much of 
the implementation is in or above the Linux kernel, and how much is below. 
 And then we can say that ISCSI-specific function (initiator or target) 
can be implemented 1) entirely above the Linux kernel; 2) entirely inside 
the Linux kernel; 3) entirely below the Linux kernel; or 4) a combination 
of these.

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

On Thu, 2005-05-12 at 15:15 -0400, James Bottomley wrote:
> On Thu, 2005-05-12 at 12:05 -0700, Dmitry Yusupov wrote:
> > oh, please! don't compare nbd and iSCSI this way...
> > iSCSI is an emerging SAN technology, and the only technology to compare
> > is FC.
> 
> Well, the question was whether iSCSI could replace nbd; It's rather
> difficult to answer that question by comparing iSCSI to FC ...

ok.
i'm just reacting on "bloated" wording. It really depends on
implementation and design. If you were talking about amount of code in
the kernel, than take a look on open-iscsi(just one file iscsi_tcp.c)
and IET where we doing a lot of management stuff in user-space. It is
not that much code in the kernel, really, but it is doing x10 times more
useful things comparing to nbd and yet compliant with RFC.

> But even projecting to iSCSI being totally mature, the amount of code
> required to conform to the iSCSI standard is easily going to put it 10x
> over the amount of code we have in nbd, principally because they're
> aimed at solving different problems and nbd achieves a lot of
> streamlining by being tied to the linux block subsystem instead of
> trying to be a generic transport.

yeah, generic transport, recovery levels, direct data placement for HW
HBAs, etc, etc... it is all *must* features for enterprise's SAN
deployment. So, yes, there is a price as usual.

Dmitry.

> James
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SCSI/ISCSI, hardware/software

[Douglas Gilbert]

Bryan Henderson wrote:
>>There is some confusion in the SCSI world between SCSI as a transport 
>>and SCSI as a commands set and software communication protocol, which 
>>works above the transport. So, you can implement SCSI transport at any 
>>software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
>>etc.) way, but if the SCSI message passing protocol is used overall 
>>system remains SCSI with all protocol obligations like task management. 
> 
> 
> The above doesn't really resolve the confusion, since it uses some 
> ambiguous terms and constructions.  I'm not sure what it's supposed to 
> say, but let me try to state in the terminology of the SCSI standards what 
> SCSI is:
> 
> SCSI is a family of separate specifications.  Some are specifications of 
> transports, and others are specifications of command sets (a layer above 
> the transports).  A SCSI device must implement a SCSI transport spec and a 
> SCSI command set spec -- and also contain a piece that actually does the 
> work (e.g. a disk drive), the details of which aren't specified by SCSI.
> 
> Examples of SCSI transport specification are (I'm paraphrasing the names) 
> parallel SCSI, Fibre Channel, and ISCSI.  Examples of command sets are the 
> disk device command set and the tape device command set.

Bryan,
This url might help to illustrate things:

http://t10.org/scsi-3.htm

Transports are below the yellow line, SCSI command sets
are above it.

Doug Gilbert

>>So, pure software SCSI solution is possible. BTW, there are pure 
>>hardware iSCSI implementations as well.
> 
> 
> I don't think it's even meaningful to talk about whether an implementation 
> is hardware or software.  The "pure hardware" implementations contain 
> megabytes of software, which was written in languages like C, contains 
> operating systems like Linux, and can be transmitted across a network and 
> updated easily.  The "pure software" implementation involve kilograms of 
> hardware in every SCSI command -- CPUs, power supplies, etc.
> 
> Not only that, but the "all hardware" ISCSI initiators people talk about, 
> which are PCI cards with Ethernet jacks, are not complete initiators.  The 
> computer you plug the card into, on which you run Linux and some 
> application programs, is the initiator.  The card is just the 
> ISCSI-specific core of it.
> 
> There's really two distinctions people mean to make when talking about 
> hardware vs software:
> 
> 1) Is it preassembled?  Can you lift it out of box whole, or do you have 
> to acquire some special software and some more generic parts separately 
> and manage their combination?
> 
> 2) Does it involve a general purpose computing system, particularly one 
> that you share with some other computing, or a faster special purpose 
> dedicated one?
> 
> In the context of a Linux SCSI discussion, I'd just talk about how much of 
> the implementation is in or above the Linux kernel, and how much is below. 
>  And then we can say that ISCSI-specific function (initiator or target) 
> can be implemented 1) entirely above the Linux kernel; 2) entirely inside 
> the Linux kernel; 3) entirely below the Linux kernel; or 4) a combination 
> of these.

Re: several messages

[Christoph Hellwig]

On Thu, May 12, 2005 at 11:32:12AM -0700, Dmitry Yusupov wrote:
> On Wed, 2005-05-11 at 22:16 -0400, Ming Zhang wrote:
> > iscsi is scsi over ip.
> 
> correction. iSCSI today has RFC at least for two transports - TCP/IP and
> iSER/RDMA(in finalized progress) with RDMA over Infiniband or RNIC. And
> I think people start writing initial draft for SCTP/IP transport...
> 
> >From this perspective, iSCSI certainly more advanced and matured
> comparing to NBD variations. 

It's for certainly much more complicated (in marketing speak that's usually
called advanced) but far less mature.

If you want network storage to just work use nbd.

Re: Re[2]: ata over ethernet question

[Christoph Hellwig]

On Thu, May 12, 2005 at 12:05:47PM -0700, Dmitry Yusupov wrote:
> oh, please! don't compare nbd and iSCSI this way...
> iSCSI is an emerging SAN technology, and the only technology to compare
> is FC.

who cares whether A is an emergeing <insert buzzword here> technology
when B is a lot simpler and just works for the customer.

I think you're a little too marketing driven here :)

Re: Re[2]: ata over ethernet question

[Christoph Hellwig]

On Thu, May 12, 2005 at 12:44:18PM -0700, Dmitry Yusupov wrote:
> i'm just reacting on "bloated" wording. It really depends on
> implementation and design. If you were talking about amount of code in
> the kernel, than take a look on open-iscsi(just one file iscsi_tcp.c)
> and IET where we doing a lot of management stuff in user-space. It is
> not that much code in the kernel, really, but it is doing x10 times more
> useful things comparing to nbd and yet compliant with RFC.

Keeping code out of the kernel is really nice, but that doesn't meant it
isn't bloat - the bloat is just in userland.

> yeah, generic transport, recovery levels, direct data placement for HW
> HBAs, etc, etc... it is all *must* features for enterprise's SAN
> deployment. So, yes, there is a price as usual.

I'm sure your marketing department can use all these buzzwords to sell
NICs to CTOs and CEOs, but else..

Re: SCSI/ISCSI, hardware/software

[Vladislav Bolkhovitin]

Bryan Henderson wrote:
>>There is some confusion in the SCSI world between SCSI as a transport 
>>and SCSI as a commands set and software communication protocol, which 
>>works above the transport. So, you can implement SCSI transport at any 
>>software (eg iSCSI) or hardware (parallel SCSI, Fibre Channel, SATA, 
>>etc.) way, but if the SCSI message passing protocol is used overall 
>>system remains SCSI with all protocol obligations like task management. 
> 
> 
> The above doesn't really resolve the confusion, since it uses some 
> ambiguous terms and constructions.  I'm not sure what it's supposed to 
> say, but let me try to state in the terminology of the SCSI standards what 
> SCSI is:
> 
> SCSI is a family of separate specifications.  Some are specifications of 
> transports, and others are specifications of command sets (a layer above 
> the transports).  A SCSI device must implement a SCSI transport spec and a 
> SCSI command set spec -- and also contain a piece that actually does the 
> work (e.g. a disk drive), the details of which aren't specified by SCSI.
> 
> Examples of SCSI transport specification are (I'm paraphrasing the names) 
> parallel SCSI, Fibre Channel, and ISCSI.  Examples of command sets are the 
> disk device command set and the tape device command set.

This is exactly what I wanted to say. Thanks for clarification

>>So, pure software SCSI solution is possible. BTW, there are pure 
>>hardware iSCSI implementations as well.
> 
> 
> I don't think it's even meaningful to talk about whether an implementation 
> is hardware or software.  The "pure hardware" implementations contain 
> megabytes of software, which was written in languages like C, contains 
> operating systems like Linux, and can be transmitted across a network and 
> updated easily.  The "pure software" implementation involve kilograms of 
> hardware in every SCSI command -- CPUs, power supplies, etc.
> 
> Not only that, but the "all hardware" ISCSI initiators people talk about, 
> which are PCI cards with Ethernet jacks, are not complete initiators.  The 
> computer you plug the card into, on which you run Linux and some 
> application programs, is the initiator.  The card is just the 
> ISCSI-specific core of it.

Such iSCSI card from a user point of view as well as for system running 
on a computer with it is just another SCSI card, not matter which 
transport it uses and how much software it runs onboard, so for they it 
doesn't differ from FC or parallel SCSI one, which I think you would not 
call a software unit. As usually, you only need appropriate driver for 
_SCSI_ subsystem.

> There's really two distinctions people mean to make when talking about 
> hardware vs software:
> 
> 1) Is it preassembled?  Can you lift it out of box whole, or do you have 
> to acquire some special software and some more generic parts separately 
> and manage their combination?
> 
> 2) Does it involve a general purpose computing system, particularly one 
> that you share with some other computing, or a faster special purpose 
> dedicated one?
> 
> In the context of a Linux SCSI discussion, I'd just talk about how much of 
> the implementation is in or above the Linux kernel, and how much is below. 
>  And then we can say that ISCSI-specific function (initiator or target) 
> can be implemented 1) entirely above the Linux kernel; 2) entirely inside 
> the Linux kernel; 3) entirely below the Linux kernel; or 4) a combination 
> of these.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: several messages

[Dmitry Yusupov]

On Fri, 2005-05-13 at 09:12 +0100, Christoph Hellwig wrote:
> On Thu, May 12, 2005 at 11:32:12AM -0700, Dmitry Yusupov wrote:
> > On Wed, 2005-05-11 at 22:16 -0400, Ming Zhang wrote:
> > > iscsi is scsi over ip.
> > 
> > correction. iSCSI today has RFC at least for two transports - TCP/IP and
> > iSER/RDMA(in finalized progress) with RDMA over Infiniband or RNIC. And
> > I think people start writing initial draft for SCTP/IP transport...
> > 
> > >From this perspective, iSCSI certainly more advanced and matured
> > comparing to NBD variations. 
> 
> It's for certainly much more complicated (in marketing speak that's usually
> called advanced) but far less mature.
> 
> If you want network storage to just work use nbd.

You could tell this to school's computer class teacher... Serious SAN
deployment will always be based either on FC or iSCSI for the reasons I
explained before.

I do not disagree, nbd is nice and simple and for sure has its own
deployment space.

Dmitry

Re: several messages

[Christoph Hellwig]

On Fri, May 13, 2005 at 08:04:16AM -0700, Dmitry Yusupov wrote:
> You could tell this to school's computer class teacher... Serious SAN
> deployment will always be based either on FC or iSCSI for the reasons I
> explained before.

Just FYI Steeleye ships a very successful clustering product that builds
on nbd.

Re: several messages

[Dmitry Yusupov]

On Fri, 2005-05-13 at 16:07 +0100, Christoph Hellwig wrote:
> On Fri, May 13, 2005 at 08:04:16AM -0700, Dmitry Yusupov wrote:
> > You could tell this to school's computer class teacher... Serious SAN
> > deployment will always be based either on FC or iSCSI for the reasons I
> > explained before.
> 
> Just FYI Steeleye ships a very successful clustering product that builds
> on nbd.

AFAIK, it is used for Data Replication purposes only. Correct me if I'm
wrong...

Re: Re[2]: ata over ethernet question

[Dmitry Yusupov]

On Fri, 2005-05-13 at 09:16 +0100, Christoph Hellwig wrote:
> On Thu, May 12, 2005 at 12:44:18PM -0700, Dmitry Yusupov wrote:
> > i'm just reacting on "bloated" wording. It really depends on
> > implementation and design. If you were talking about amount of code in
> > the kernel, than take a look on open-iscsi(just one file iscsi_tcp.c)
> > and IET where we doing a lot of management stuff in user-space. It is
> > not that much code in the kernel, really, but it is doing x10 times more
> > useful things comparing to nbd and yet compliant with RFC.
> 
> Keeping code out of the kernel is really nice, but that doesn't meant it
> isn't bloat - the bloat is just in userland.

well, "userland" == "bloatland" anyways... Multiple discovery methods,
configuration database, bunch of security protocols, etc... all this of
course will make it "slightly" :) bigger than nbd. But again, for a good
reason and better usefulness.

Dmitry

iSCSI vs. NBD (was Re: ata over ethernet question)

[Guennadi Liakhovetski]

On Thu, 12 May 2005, James Bottomley wrote:

> However, there is room for improvement in nbd, notably the handling of
> packet commands, which looks to be eminently doable in the current
> infrastructure (this would basically make nbd a replicator for the linux
> block system, and would probably necessitate some client side changes to
> achieve).  If you have any thoughts in this direction, you could drop an
> email to the maintainer.

Thanks, James

I'll try to get some (thoughts):-) BTW, who is the maintainer of nbd? No 
one in MAINTAINERS, in nbd.c only
 * Copyright 1997-2000 Pavel Machek <pavel@ucw.cz>
 * Parts copyright 2001 Steven Whitehouse <steve@chygwyn.com>
Is it Pavel then?

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: iSCSI vs. NBD (was Re: ata over ethernet question)

[James Bottomley]

On Fri, 2005-05-13 at 20:50 +0200, Guennadi Liakhovetski wrote:
> I'll try to get some (thoughts):-) BTW, who is the maintainer of nbd? No 
> one in MAINTAINERS, in nbd.c only
>  * Copyright 1997-2000 Pavel Machek <pavel@ucw.cz>
>  * Parts copyright 2001 Steven Whitehouse <steve@chygwyn.com>
> Is it Pavel then?

Well, my copy of the MAINTAINERS file has this:

NETWORK BLOCK DEVICE
P:      Paul Clements
M:      Paul.Clements@steeleye.com
S:      Maintained

James

Re: iSCSI vs. NBD (was Re: ata over ethernet question)

[Guennadi Liakhovetski]

On Fri, 13 May 2005, James Bottomley wrote:

> Well, my copy of the MAINTAINERS file has this:
> 
> NETWORK BLOCK DEVICE
> P:      Paul Clements
> M:      Paul.Clements@steeleye.com
> S:      Maintained

Auch, `grep -i nbd MAINTAINERS`:-)))

Thanks
Guennadi
---
Guennadi Liakhovetski

Re: SCSI/ISCSI, hardware/software

[Bryan Henderson]

>Such iSCSI card from a user point of view as well as for system running 
>on a computer with it is just another SCSI card, not matter which 
>transport it uses and how much software it runs onboard, so for they it 
>doesn't differ from FC or parallel SCSI one, which I think you would not 
>call a software unit.  As usually, you only need appropriate driver for 
>_SCSI_ subsystem.

The point I'd like to make is that _I_ would not call it a software unit 
or a hardware unit, because I don't think in most contexts (including that 
of this thread), it makes any difference which technology is used in the 
implementation.  What _does_ matter is 1) this card comes preassembled.  I 
don't have to find and load independently produced software onto it, or 
worry about interoperability.  2) It's below the Linux kernel, which means 
I won't need to mess with Linux applications or kernels except to load a 
low level SCSI driver.  It also means it doesn't place any load on my main 
CPU and probably goes faster than something implemented in or above my 
Linux kernel would.

And there's the separate point that it would be a misnomer to say that 
this card is an ISCSI initiator (it's only part of one); so even if the 
card itself can be called hardware, that still doesn't mean you can say 
you have a hardware ISCSI initiator.  Same is true of a parallel SCSI host 
adapter card.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems