[RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Nicholas A. Bellinger]

Greetings all,

This is my humble request for review and inclusion of
Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
features that have now been completed: Persistent Reservation APTPL and
implict/explict ALUA are the first all optional SPC4 defined feature
implementation of PR and ALUA in the Linux OSS target theatre using an
upstream user-driven virtual filesystem directory hierarchy structure
based on configfs that can be configured in realtime in combination with
interpreted userspace code.

Both of these new features are now being used to provide Persistent Reservations
across target power loss and asymmetric multipath I/O to both Linux and non-Linux
clients.  There is also an optional ALUA transition delay logic that has been
included to futher debug and improve existing upstream client side Linux kernel code
using TCM v3.2. These patches have been diffed against v2.6.31, and tested on x86
32-bit HVM, x86_64 KVM, and powerpc.

There is cleanup work continuing to make the TCM code to use dprintk(),
and some include cleanups, and other minor fixes.  The code has also
been run through checkpatch again since the posting in April, and all
errors have been removed, with the exception of one ConfigFS macro
define triggering a checkpatch false positive.

As with the posting of the TCM v3.0 code earlier in April, I have
decided to break up the submissions into the generic target_core_mod
piece (this patch series) and then a seperate one that will be appearing
in the next days for v3.2 of the LIO-Target iSCSI fabric module.

More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:

http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
http://www.linux-iscsi.org/index.php/Persistent_Reservations
http://www.linux-iscsi.org/index.php/ALUA

The complete v3.x TCM CLI operations set can be found here:

http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations

Instructions and examples for getting setup:

http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod

Constructive comments are welcome,

--nab

==> target_core_alua.patch <==
[RFC PATCH 1/19] Target_Core_Mod Asymmetric Logical Unit Assignment (ALUA) support
==> target_core_configfs.patch <==
[RFC PATCH 2/19] Target_Core_Mod ConfigFS infrastructure
==> target_core_device.patch <==
[RFC PATCH 3/19] Target_Core_Mod device core infrastructure
==> target_core_file.patch <==
[RFC PATCH 4/19] Target_Core_Mod FILEIO <-> Linux/VFS subsystem
==> target_core_hba.patch <==
[RFC PATCH 5/19] Target_Core_Mod HBA core infrastructure
==> target_core_iblock.patch <==
[RFC PATCH 6/19] Target_Core_Mod IBLOCK <-> Linux/Block subsystem plugin
==> target_core_mib.patch <==
[RFC PATCH 7/19] Target_Core_Mod SCSI MIBs
==> target_core_plugin.patch <==
[RFC PATCH 8/19] Target_Core_Mod Subsystem and SE Object plugin infrastructure
==> target_core_pr.patch <==
[RFC PATCH 9/19] Target_Core_Mod SPC-4 Persistent Reservations
==> target_core_pscsi.patch <==
[RFC PATCH 10/19] Target_Core_Mod PSCSI <-> Linux/SCSI subsystem plugin
==> target_core_rd.patch <==
[RFC PATCH 11/19] Target_Core_Mod RAMDISK_DR and RAMDISK_MCP subsystem plugin
==> target_core_scdb.patch <==
[RFC PATCH 12/19] Target_Core_Mod READ/WRITE command descriptor block generation code
==> target_core_seobj.patch <==
[RFC PATCH 13/19] Target_Core_Mod Storage Engine Object infrastructure
==> target_core_tmr.patch <==
[RFC PATCH 14/19] Target_Core_Mod Task Management infrastructure
==> target_core_tpg.patch <==
[RFC PATCH 15/19] Target_Core_Mod portal group endpoint infrastructure
==> target_core_transport.patch <==
[RFC PATCH 16/19] Target_Core_Mod Generic Transport Core
==> target_core_ua.patch <==
[RFC PATCH 17/19] Target_Core_Mod Unit Attention emulation support
==> target_core_base_includes.patch <==
[RFC PATCH 18/19] Target_Core_Mod base and fabric_module includes
==> target_core_misc.patch <==
[RFC PATCH 19/19] Target_Core_Mod Makefile/Kconfig and div64.c

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Daniel Walker]

On Fri, 2009-09-11 at 18:55 -0700, Nicholas A. Bellinger wrote:
> There is cleanup work continuing to make the TCM code to use
> dprintk(),
> and some include cleanups, and other minor fixes.  The code has also
> been run through checkpatch again since the posting in April, and all
> errors have been removed, with the exception of one ConfigFS macro
> define triggering a checkpatch false positive.


One show stopping issue is that your adding new semaphore usage in patch
5 and 16 (and in 16 it's a LOCKED one wtf!!). Which you should not be
doing .. You need to really evaluate the warnings from checkpatch, cause
those still have meaning.. 

It's basically telling you what you should be doing in the warnings..
For instance, don't add new semaphore usage, and it tells you what you
need to be using in place of them.. Also typedefs are frowned upon, so
you could remove those..

Daniel

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Nicholas A. Bellinger]

On Fri, 2009-09-11 at 19:37 -0700, Daniel Walker wrote:
> On Fri, 2009-09-11 at 18:55 -0700, Nicholas A. Bellinger wrote:
> > There is cleanup work continuing to make the TCM code to use
> > dprintk(),
> > and some include cleanups, and other minor fixes.  The code has also
> > been run through checkpatch again since the posting in April, and all
> > errors have been removed, with the exception of one ConfigFS macro
> > define triggering a checkpatch false positive.
> 

Greetings Daniel,

> 
> One show stopping issue is that your adding new semaphore usage in patch
> 5 and 16 (and in 16 it's a LOCKED one wtf!!). Which you should not be
> doing .. You need to really evaluate the warnings from checkpatch, cause
> those still have meaning.. 
> 

Ok, I will have a look at converting the HBA semaphore to a mutex in
patch 5, and use some single use waitqueues in patch 16.  Btw, the ones
using init_MUTEX_LOCKED() in the latter source file are being used when
shutting down target_core_mod allocated kernel threads to sychronize
between the stopping processing threads and configfs process context
that waitqueues would work better for.  Thanks for pointing this out!

> It's basically telling you what you should be doing in the warnings..
> For instance, don't add new semaphore usage, and it tells you what you
> need to be using in place of them..

Ok, so I will also do the struct sempahore -> struct mutex conversion
where it makes sense to protect lists and see about the rest..

>  Also typedefs are frowned upon, so
> you could remove those..
> 

I was thinking about going through the typedefs with a sed chainsaw, but
I decided against for this TCM patch series because it makes keeping
backward ports to <= v2.6.31 slightly easier to maintain for me for the
current users of LIO kernel code.

However, I have been starting to not use typedef for the newest ALUA and
pieces of PR code, and I would be happy to have another look at typedef
style comments on this once the other larger interest items are on the
way to being resolved.

Thanks!

--nab

> Daniel
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Daniel Walker]

On Fri, 2009-09-11 at 20:08 -0700, Nicholas A. Bellinger wrote:


> 
> Ok, I will have a look at converting the HBA semaphore to a mutex in
> patch 5, and use some single use waitqueues in patch 16.  Btw, the ones
> using init_MUTEX_LOCKED() in the latter source file are being used when
> shutting down target_core_mod allocated kernel threads to sychronize
> between the stopping processing threads and configfs process context
> that waitqueues would work better for.  Thanks for pointing this out!

You could also use completions in patch 16 if the waitqueues turn out to
be not what your looking for. Completion can sometimes be a find&replace
substitute for the locked semaphores.


> I was thinking about going through the typedefs with a sed chainsaw, but
> I decided against for this TCM patch series because it makes keeping
> backward ports to <= v2.6.31 slightly easier to maintain for me for the
> current users of LIO kernel code.
> 
> However, I have been starting to not use typedef for the newest ALUA and
> pieces of PR code, and I would be happy to have another look at typedef
> style comments on this once the other larger interest items are on the
> way to being resolved.

It's up to you, but you should schedule the typedefs for removal at some
point.

Daniel

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Nicholas A. Bellinger]

On Fri, 2009-09-11 at 20:17 -0700, Daniel Walker wrote:
> On Fri, 2009-09-11 at 20:08 -0700, Nicholas A. Bellinger wrote:
> 
> 
> > 
> > Ok, I will have a look at converting the HBA semaphore to a mutex in
> > patch 5, and use some single use waitqueues in patch 16.  Btw, the ones
> > using init_MUTEX_LOCKED() in the latter source file are being used when
> > shutting down target_core_mod allocated kernel threads to sychronize
> > between the stopping processing threads and configfs process context
> > that waitqueues would work better for.  Thanks for pointing this out!
> 
> You could also use completions in patch 16 if the waitqueues turn out to
> be not what your looking for. Completion can sometimes be a find&replace
> substitute for the locked semaphores.

Sorry, include/linux/complete.h is what I was thinking here too..

> 
> 
> > I was thinking about going through the typedefs with a sed chainsaw, but
> > I decided against for this TCM patch series because it makes keeping
> > backward ports to <= v2.6.31 slightly easier to maintain for me for the
> > current users of LIO kernel code.
> > 
> > However, I have been starting to not use typedef for the newest ALUA and
> > pieces of PR code, and I would be happy to have another look at typedef
> > style comments on this once the other larger interest items are on the
> > way to being resolved.
> 
> It's up to you, but you should schedule the typedefs for removal at some
> point.
> 

Ok, I will do that.

Many thanks for your most valuable of time,

--nab

> Daniel
> 

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Boaz Harrosh]

On 09/12/2009 04:55 AM, Nicholas A. Bellinger wrote:
> Greetings all,
> 
> This is my humble request for review and inclusion of
> Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> features that have now been completed: Persistent Reservation APTPL and
> implict/explict ALUA are the first all optional SPC4 defined feature
> implementation of PR and ALUA in the Linux OSS target theatre using an
> upstream user-driven virtual filesystem directory hierarchy structure
> based on configfs that can be configured in realtime in combination with
> interpreted userspace code.
> 
> Both of these new features are now being used to provide Persistent Reservations
> across target power loss and asymmetric multipath I/O to both Linux and non-Linux
> clients.  There is also an optional ALUA transition delay logic that has been
> included to futher debug and improve existing upstream client side Linux kernel code
> using TCM v3.2. These patches have been diffed against v2.6.31, and tested on x86
> 32-bit HVM, x86_64 KVM, and powerpc.
> 
> There is cleanup work continuing to make the TCM code to use dprintk(),
> and some include cleanups, and other minor fixes.  The code has also
> been run through checkpatch again since the posting in April, and all
> errors have been removed, with the exception of one ConfigFS macro
> define triggering a checkpatch false positive.
> 
> As with the posting of the TCM v3.0 code earlier in April, I have
> decided to break up the submissions into the generic target_core_mod
> piece (this patch series) and then a seperate one that will be appearing
> in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> 
> More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> 
> http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> http://www.linux-iscsi.org/index.php/Persistent_Reservations
> http://www.linux-iscsi.org/index.php/ALUA
> 
> The complete v3.x TCM CLI operations set can be found here:
> 
> http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> 
> Instructions and examples for getting setup:
> 
> http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
> 
> Constructive comments are welcome,
> 
> --nab
> 
> ==> target_core_alua.patch <==
> [RFC PATCH 1/19] Target_Core_Mod Asymmetric Logical Unit Assignment (ALUA) support
> ==> target_core_configfs.patch <==
> [RFC PATCH 2/19] Target_Core_Mod ConfigFS infrastructure
> ==> target_core_device.patch <==
> [RFC PATCH 3/19] Target_Core_Mod device core infrastructure
> ==> target_core_file.patch <==
> [RFC PATCH 4/19] Target_Core_Mod FILEIO <-> Linux/VFS subsystem
> ==> target_core_hba.patch <==
> [RFC PATCH 5/19] Target_Core_Mod HBA core infrastructure
> ==> target_core_iblock.patch <==
> [RFC PATCH 6/19] Target_Core_Mod IBLOCK <-> Linux/Block subsystem plugin
> ==> target_core_mib.patch <==
> [RFC PATCH 7/19] Target_Core_Mod SCSI MIBs
> ==> target_core_plugin.patch <==
> [RFC PATCH 8/19] Target_Core_Mod Subsystem and SE Object plugin infrastructure
> ==> target_core_pr.patch <==
> [RFC PATCH 9/19] Target_Core_Mod SPC-4 Persistent Reservations
> ==> target_core_pscsi.patch <==
> [RFC PATCH 10/19] Target_Core_Mod PSCSI <-> Linux/SCSI subsystem plugin
> ==> target_core_rd.patch <==
> [RFC PATCH 11/19] Target_Core_Mod RAMDISK_DR and RAMDISK_MCP subsystem plugin
> ==> target_core_scdb.patch <==
> [RFC PATCH 12/19] Target_Core_Mod READ/WRITE command descriptor block generation code
> ==> target_core_seobj.patch <==
> [RFC PATCH 13/19] Target_Core_Mod Storage Engine Object infrastructure
> ==> target_core_tmr.patch <==
> [RFC PATCH 14/19] Target_Core_Mod Task Management infrastructure
> ==> target_core_tpg.patch <==
> [RFC PATCH 15/19] Target_Core_Mod portal group endpoint infrastructure
> ==> target_core_transport.patch <==
> [RFC PATCH 16/19] Target_Core_Mod Generic Transport Core
> ==> target_core_ua.patch <==
> [RFC PATCH 17/19] Target_Core_Mod Unit Attention emulation support
> ==> target_core_base_includes.patch <==
> [RFC PATCH 18/19] Target_Core_Mod base and fabric_module includes
> ==> target_core_misc.patch <==
> [RFC PATCH 19/19] Target_Core_Mod Makefile/Kconfig and div64.c
> 

I would like to see a documentation patch which adds a file under Documentation/
With some abstraction, design, usability, URLs

I promise to read it ;-)

I'm sure you have lots of stuff written laying around. Could you edit them up for
Linux?

Boaz

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Vladislav Bolkhovitin]

Nicholas A. Bellinger, on 09/12/2009 05:55 AM wrote:
> Greetings all,
> 
> This is my humble request for review and inclusion of
> Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> features that have now been completed: Persistent Reservation APTPL and
> implict/explict ALUA are the first all optional SPC4 defined feature
> implementation of PR and ALUA in the Linux OSS target theatre using an
> upstream user-driven virtual filesystem directory hierarchy structure
> based on configfs that can be configured in realtime in combination with
> interpreted userspace code.
> 
> Both of these new features are now being used to provide Persistent Reservations
> across target power loss and asymmetric multipath I/O to both Linux and non-Linux
> clients.  There is also an optional ALUA transition delay logic that has been
> included to futher debug and improve existing upstream client side Linux kernel code
> using TCM v3.2. These patches have been diffed against v2.6.31, and tested on x86
> 32-bit HVM, x86_64 KVM, and powerpc.
> 
> There is cleanup work continuing to make the TCM code to use dprintk(),
> and some include cleanups, and other minor fixes.  The code has also
> been run through checkpatch again since the posting in April, and all
> errors have been removed, with the exception of one ConfigFS macro
> define triggering a checkpatch false positive.
> 
> As with the posting of the TCM v3.0 code earlier in April, I have
> decided to break up the submissions into the generic target_core_mod
> piece (this patch series) and then a seperate one that will be appearing
> in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> 
> More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> 
> http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> http://www.linux-iscsi.org/index.php/Persistent_Reservations
> http://www.linux-iscsi.org/index.php/ALUA
> 
> The complete v3.x TCM CLI operations set can be found here:
> 
> http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> 
> Instructions and examples for getting setup:
> 
> http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod

Some questions among many I have:

1. Where and how TCM is better over the existing mainline target 
framework STGT to justify such a move as adding a new piece of code 
doing the same job?

2. On the linux-iscsi.org main page (BTW, the comparison on that page 
isn't fully correct about STGT and SCST) you claim that TCM is a generic 
target engine. But how can you prove (and test) that if there are no 
hardware target drivers for TCM?

Also patch 7 implements MIBs through procfs. Procfs is for long time 
forbidden for adding anything like that, so you can't push it in the 
mainline.

Vlad

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Nicholas A. Bellinger]

On Sun, 2009-09-13 at 13:10 +0300, Boaz Harrosh wrote:
> On 09/12/2009 04:55 AM, Nicholas A. Bellinger wrote:
> > Greetings all,
> > 
> > This is my humble request for review and inclusion of
> > Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> > v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> > features that have now been completed: Persistent Reservation APTPL and
> > implict/explict ALUA are the first all optional SPC4 defined feature
> > implementation of PR and ALUA in the Linux OSS target theatre using an
> > upstream user-driven virtual filesystem directory hierarchy structure
> > based on configfs that can be configured in realtime in combination with
> > interpreted userspace code.
> > 
> > Both of these new features are now being used to provide Persistent Reservations
> > across target power loss and asymmetric multipath I/O to both Linux and non-Linux
> > clients.  There is also an optional ALUA transition delay logic that has been
> > included to futher debug and improve existing upstream client side Linux kernel code
> > using TCM v3.2. These patches have been diffed against v2.6.31, and tested on x86
> > 32-bit HVM, x86_64 KVM, and powerpc.
> > 
> > There is cleanup work continuing to make the TCM code to use dprintk(),
> > and some include cleanups, and other minor fixes.  The code has also
> > been run through checkpatch again since the posting in April, and all
> > errors have been removed, with the exception of one ConfigFS macro
> > define triggering a checkpatch false positive.
> > 
> > As with the posting of the TCM v3.0 code earlier in April, I have
> > decided to break up the submissions into the generic target_core_mod
> > piece (this patch series) and then a seperate one that will be appearing
> > in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> > 
> > More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> > 
> > http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> > http://www.linux-iscsi.org/index.php/Persistent_Reservations
> > http://www.linux-iscsi.org/index.php/ALUA
> > 
> > The complete v3.x TCM CLI operations set can be found here:
> > 
> > http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> > 
> > Instructions and examples for getting setup:
> > 
> > http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> > http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
> > 
> > Constructive comments are welcome,
> > 
> > --nab
> > 
> > ==> target_core_alua.patch <==
> > [RFC PATCH 1/19] Target_Core_Mod Asymmetric Logical Unit Assignment (ALUA) support
> > ==> target_core_configfs.patch <==
> > [RFC PATCH 2/19] Target_Core_Mod ConfigFS infrastructure
> > ==> target_core_device.patch <==
> > [RFC PATCH 3/19] Target_Core_Mod device core infrastructure
> > ==> target_core_file.patch <==
> > [RFC PATCH 4/19] Target_Core_Mod FILEIO <-> Linux/VFS subsystem
> > ==> target_core_hba.patch <==
> > [RFC PATCH 5/19] Target_Core_Mod HBA core infrastructure
> > ==> target_core_iblock.patch <==
> > [RFC PATCH 6/19] Target_Core_Mod IBLOCK <-> Linux/Block subsystem plugin
> > ==> target_core_mib.patch <==
> > [RFC PATCH 7/19] Target_Core_Mod SCSI MIBs
> > ==> target_core_plugin.patch <==
> > [RFC PATCH 8/19] Target_Core_Mod Subsystem and SE Object plugin infrastructure
> > ==> target_core_pr.patch <==
> > [RFC PATCH 9/19] Target_Core_Mod SPC-4 Persistent Reservations
> > ==> target_core_pscsi.patch <==
> > [RFC PATCH 10/19] Target_Core_Mod PSCSI <-> Linux/SCSI subsystem plugin
> > ==> target_core_rd.patch <==
> > [RFC PATCH 11/19] Target_Core_Mod RAMDISK_DR and RAMDISK_MCP subsystem plugin
> > ==> target_core_scdb.patch <==
> > [RFC PATCH 12/19] Target_Core_Mod READ/WRITE command descriptor block generation code
> > ==> target_core_seobj.patch <==
> > [RFC PATCH 13/19] Target_Core_Mod Storage Engine Object infrastructure
> > ==> target_core_tmr.patch <==
> > [RFC PATCH 14/19] Target_Core_Mod Task Management infrastructure
> > ==> target_core_tpg.patch <==
> > [RFC PATCH 15/19] Target_Core_Mod portal group endpoint infrastructure
> > ==> target_core_transport.patch <==
> > [RFC PATCH 16/19] Target_Core_Mod Generic Transport Core
> > ==> target_core_ua.patch <==
> > [RFC PATCH 17/19] Target_Core_Mod Unit Attention emulation support
> > ==> target_core_base_includes.patch <==
> > [RFC PATCH 18/19] Target_Core_Mod base and fabric_module includes
> > ==> target_core_misc.patch <==
> > [RFC PATCH 19/19] Target_Core_Mod Makefile/Kconfig and div64.c
> > 
> 
> I would like to see a documentation patch which adds a file under Documentation/
> With some abstraction, design, usability, URLs
> 
> I promise to read it ;-)
> 
> I'm sure you have lots of stuff written laying around. Could you edit them up for
> Linux?

Understood, I will put together something for Documentation/ and include
it in the next round of patches.

Many thanks for your most valuable of time,

--nab

> 
> Boaz

RE: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Chetan.Loke]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
> owner@vger.kernel.org] On Behalf Of Nicholas A. Bellinger
> Sent: Friday, September 11, 2009 9:55 PM
> To: LKML; linux-scsi
> Cc: Andrew Morton; Greg KH; Douglas Gilbert; James Bottomley; Hannes
> Reinecke; FUJITA Tomonori; Mike Christie; Joel Becker; Martin K. Petersen;
> Christoph Hellwig; Linus Torvalds; Alan Stern; Boaz Harrosh; Florian Haas;
> Philipp Reisner
> Subject: [RFC PATCH 0/19] A kernel-level configfs enabled generic target
> engine for Linux v2.6.32
> 
> Greetings all,
> 
> This is my humble request for review and inclusion of
> Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> features that have now been completed: Persistent Reservation APTPL and
> implict/explict ALUA are the first all optional SPC4 defined feature
> implementation of PR and ALUA in the Linux OSS target theatre using an
> upstream user-driven virtual filesystem directory hierarchy structure
> based on configfs that can be configured in realtime in combination with
> interpreted userspace code.
> 
> Both of these new features are now being used to provide Persistent
> Reservations
> across target power loss and asymmetric multipath I/O to both Linux and
> non-Linux
> clients.  There is also an optional ALUA transition delay logic that has
> been
> included to futher debug and improve existing upstream client side Linux
> kernel code
> using TCM v3.2. These patches have been diffed against v2.6.31, and tested
> on x86
> 32-bit HVM, x86_64 KVM, and powerpc.
> 
> There is cleanup work continuing to make the TCM code to use dprintk(),
> and some include cleanups, and other minor fixes.  The code has also
> been run through checkpatch again since the posting in April, and all
> errors have been removed, with the exception of one ConfigFS macro
> define triggering a checkpatch false positive.
> 
> As with the posting of the TCM v3.0 code earlier in April, I have
> decided to break up the submissions into the generic target_core_mod
> piece (this patch series) and then a seperate one that will be appearing
> in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> 
> More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> 
> http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> http://www.linux-iscsi.org/index.php/Persistent_Reservations
> http://www.linux-iscsi.org/index.php/ALUA
> 
> The complete v3.x TCM CLI operations set can be found here:
> 
> http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> 
> Instructions and examples for getting setup:
> 
> http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
> 
> Constructive comments are welcome,
> 
> --nab
> 



Any pointers/URLs for the design/arch doc?


Chetan 

RE: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Chetan.Loke]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
> owner@vger.kernel.org] On Behalf Of Nicholas A. Bellinger
> Sent: Friday, September 11, 2009 9:55 PM
> To: LKML; linux-scsi
> Cc: Andrew Morton; Greg KH; Douglas Gilbert; James Bottomley; Hannes
> Reinecke; FUJITA Tomonori; Mike Christie; Joel Becker; Martin K. Petersen;
> Christoph Hellwig; Linus Torvalds; Alan Stern; Boaz Harrosh; Florian Haas;
> Philipp Reisner
> Subject: [RFC PATCH 0/19] A kernel-level configfs enabled generic target
> engine for Linux v2.6.32
> 
> Greetings all,
> 
> This is my humble request for review and inclusion of
> Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> features that have now been completed: Persistent Reservation APTPL and
> implict/explict ALUA are the first all optional SPC4 defined feature
> implementation of PR and ALUA in the Linux OSS target theatre using an
> upstream user-driven virtual filesystem directory hierarchy structure
> based on configfs that can be configured in realtime in combination with
> interpreted userspace code.
> 
> Both of these new features are now being used to provide Persistent
> Reservations
> across target power loss and asymmetric multipath I/O to both Linux and
> non-Linux
> clients.  There is also an optional ALUA transition delay logic that has
> been
> included to futher debug and improve existing upstream client side Linux
> kernel code
> using TCM v3.2. These patches have been diffed against v2.6.31, and tested
> on x86
> 32-bit HVM, x86_64 KVM, and powerpc.
> 
> There is cleanup work continuing to make the TCM code to use dprintk(),
> and some include cleanups, and other minor fixes.  The code has also
> been run through checkpatch again since the posting in April, and all
> errors have been removed, with the exception of one ConfigFS macro
> define triggering a checkpatch false positive.
> 
> As with the posting of the TCM v3.0 code earlier in April, I have
> decided to break up the submissions into the generic target_core_mod
> piece (this patch series) and then a seperate one that will be appearing
> in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> 
> More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> 
> http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> http://www.linux-iscsi.org/index.php/Persistent_Reservations
> http://www.linux-iscsi.org/index.php/ALUA
> 
> The complete v3.x TCM CLI operations set can be found here:
> 
> http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> 
> Instructions and examples for getting setup:
> 
> http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
> 
> Constructive comments are welcome,
> 
> --nab
> 



Any pointers/URLs for the design/arch doc?


Chetan 

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Nicholas A. Bellinger]

On Mon, 2009-09-14 at 22:50 +0400, Vladislav Bolkhovitin wrote:
> Nicholas A. Bellinger, on 09/12/2009 05:55 AM wrote:
> > Greetings all,
> > 
> > This is my humble request for review and inclusion of
> > Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> > v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> > features that have now been completed: Persistent Reservation APTPL and
> > implict/explict ALUA are the first all optional SPC4 defined feature
> > implementation of PR and ALUA in the Linux OSS target theatre using an
> > upstream user-driven virtual filesystem directory hierarchy structure
> > based on configfs that can be configured in realtime in combination with
> > interpreted userspace code.
> > 
> > Both of these new features are now being used to provide Persistent Reservations
> > across target power loss and asymmetric multipath I/O to both Linux and non-Linux
> > clients.  There is also an optional ALUA transition delay logic that has been
> > included to futher debug and improve existing upstream client side Linux kernel code
> > using TCM v3.2. These patches have been diffed against v2.6.31, and tested on x86
> > 32-bit HVM, x86_64 KVM, and powerpc.
> > 
> > There is cleanup work continuing to make the TCM code to use dprintk(),
> > and some include cleanups, and other minor fixes.  The code has also
> > been run through checkpatch again since the posting in April, and all
> > errors have been removed, with the exception of one ConfigFS macro
> > define triggering a checkpatch false positive.
> > 
> > As with the posting of the TCM v3.0 code earlier in April, I have
> > decided to break up the submissions into the generic target_core_mod
> > piece (this patch series) and then a seperate one that will be appearing
> > in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> > 
> > More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> > 
> > http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> > http://www.linux-iscsi.org/index.php/Persistent_Reservations
> > http://www.linux-iscsi.org/index.php/ALUA
> > 
> > The complete v3.x TCM CLI operations set can be found here:
> > 
> > http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> > 
> > Instructions and examples for getting setup:
> > 
> > http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> > http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
> 
> Some questions among many I have:
> 
> 1. Where and how TCM is better over the existing mainline target 
> framework STGT to justify such a move as adding a new piece of code 
> doing the same job?
> 

I think from reading my original email and the list of patches it is
quite obvious to the people involved what are the major pieces of
functionality available in TCM v3.2 that people actually *want* and
*care* about that is not available in the existing STGT framework.

> 2. On the linux-iscsi.org main page (BTW, the comparison on that page 
> isn't fully correct about STGT and SCST) you claim that TCM is a generic 
> target engine. But how can you prove (and test) that if there are no 
> hardware target drivers for TCM?


I am not sure why you think you need hardware target drivers to prove
that a target engine is generic..?  Why don't you look at the TCM v3.2
code (like everyone else is doing) and see for yourself..?

> 
> Also patch 7 implements MIBs through procfs. Procfs is for long time 
> forbidden for adding anything like that, so you can't push it in the 
> mainline.

There are other subsystems (Linux/Net and SCTP for example) that use
procfs for read-only MIBs statistics, so TCM actually having an
implementation of SCSI MIBs available (no other target projects have
SCSI MIBs btw) under procfs is hardly a show-stopper considering the
*entire* control path for TCM v3.2 is using upstream configfs code
anyways.

Best,

--nab


> 
> Vlad

RE: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Nicholas A. Bellinger]

On Mon, 2009-09-14 at 13:12 -0700, Chetan.Loke@Emulex.Com wrote:
> > -----Original Message-----
> > From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
> > owner@vger.kernel.org] On Behalf Of Nicholas A. Bellinger
> > Sent: Friday, September 11, 2009 9:55 PM
> > To: LKML; linux-scsi
> > Cc: Andrew Morton; Greg KH; Douglas Gilbert; James Bottomley; Hannes
> > Reinecke; FUJITA Tomonori; Mike Christie; Joel Becker; Martin K. Petersen;
> > Christoph Hellwig; Linus Torvalds; Alan Stern; Boaz Harrosh; Florian Haas;
> > Philipp Reisner
> > Subject: [RFC PATCH 0/19] A kernel-level configfs enabled generic target
> > engine for Linux v2.6.32
> > 
> > Greetings all,
> > 
> > This is my humble request for review and inclusion of
> > Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
> > v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
> > features that have now been completed: Persistent Reservation APTPL and
> > implict/explict ALUA are the first all optional SPC4 defined feature
> > implementation of PR and ALUA in the Linux OSS target theatre using an
> > upstream user-driven virtual filesystem directory hierarchy structure
> > based on configfs that can be configured in realtime in combination with
> > interpreted userspace code.
> > 
> > Both of these new features are now being used to provide Persistent
> > Reservations
> > across target power loss and asymmetric multipath I/O to both Linux and
> > non-Linux
> > clients.  There is also an optional ALUA transition delay logic that has
> > been
> > included to futher debug and improve existing upstream client side Linux
> > kernel code
> > using TCM v3.2. These patches have been diffed against v2.6.31, and tested
> > on x86
> > 32-bit HVM, x86_64 KVM, and powerpc.
> > 
> > There is cleanup work continuing to make the TCM code to use dprintk(),
> > and some include cleanups, and other minor fixes.  The code has also
> > been run through checkpatch again since the posting in April, and all
> > errors have been removed, with the exception of one ConfigFS macro
> > define triggering a checkpatch false positive.
> > 
> > As with the posting of the TCM v3.0 code earlier in April, I have
> > decided to break up the submissions into the generic target_core_mod
> > piece (this patch series) and then a seperate one that will be appearing
> > in the next days for v3.2 of the LIO-Target iSCSI fabric module.
> > 
> > More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
> > 
> > http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
> > http://www.linux-iscsi.org/index.php/Persistent_Reservations
> > http://www.linux-iscsi.org/index.php/ALUA
> > 
> > The complete v3.x TCM CLI operations set can be found here:
> > 
> > http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
> > 
> > Instructions and examples for getting setup:
> > 
> > http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
> > http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
> > 
> > Constructive comments are welcome,
> > 
> > --nab
> > 
> 
> 
> 
> Any pointers/URLs for the design/arch doc?

Greetings Chetan,

There are various bits and pieces detailing the TCM design on the
linux-iscsi.org wiki, but no definitive design document (aside from the
comments in the code) is available yet.  

I will however be putting together a first draft for this (along with
the user-level stuff) for Documentation/ soon.  Until then, I would be
happy to answer any questions you may have.

--nab 

> 
> 
> Chetan 

Re: [RFC PATCH 0/19] A kernel-level configfs enabled generic target engine for Linux v2.6.32

[Vladislav Bolkhovitin]

Nicholas A. Bellinger, on 09/15/2009 12:49 AM wrote:
> On Mon, 2009-09-14 at 22:50 +0400, Vladislav Bolkhovitin wrote:
>> Nicholas A. Bellinger, on 09/12/2009 05:55 AM wrote:
>>> Greetings all,
>>>
>>> This is my humble request for review and inclusion of
>>> Target_Core_Mod/ConfigFS v3.2 from lio-core-2.6.git code into mainline
>>> v2.6.32.  With TCM v3.2, the two major SPC-4 SCSI target mode fabric
>>> features that have now been completed: Persistent Reservation APTPL and
>>> implict/explict ALUA are the first all optional SPC4 defined feature
>>> implementation of PR and ALUA in the Linux OSS target theatre using an
>>> upstream user-driven virtual filesystem directory hierarchy structure
>>> based on configfs that can be configured in realtime in combination with
>>> interpreted userspace code.
>>>
>>> Both of these new features are now being used to provide Persistent Reservations
>>> across target power loss and asymmetric multipath I/O to both Linux and non-Linux
>>> clients.  There is also an optional ALUA transition delay logic that has been
>>> included to futher debug and improve existing upstream client side Linux kernel code
>>> using TCM v3.2. These patches have been diffed against v2.6.31, and tested on x86
>>> 32-bit HVM, x86_64 KVM, and powerpc.
>>>
>>> There is cleanup work continuing to make the TCM code to use dprintk(),
>>> and some include cleanups, and other minor fixes.  The code has also
>>> been run through checkpatch again since the posting in April, and all
>>> errors have been removed, with the exception of one ConfigFS macro
>>> define triggering a checkpatch false positive.
>>>
>>> As with the posting of the TCM v3.0 code earlier in April, I have
>>> decided to break up the submissions into the generic target_core_mod
>>> piece (this patch series) and then a seperate one that will be appearing
>>> in the next days for v3.2 of the LIO-Target iSCSI fabric module.
>>>
>>> More Information about Target_Core_Mod/ConfigFS, and PR and ALUA:
>>>
>>> http://www.linux-iscsi.org/index.php/Target_Core_Mod/ConfigFS
>>> http://www.linux-iscsi.org/index.php/Persistent_Reservations
>>> http://www.linux-iscsi.org/index.php/ALUA
>>>
>>> The complete v3.x TCM CLI operations set can be found here:
>>>
>>> http://www.linux-iscsi.org/index.php/Lio-utils#v3.1_TCM_CLI_operations
>>>
>>> Instructions and examples for getting setup:
>>>
>>> http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf
>>> http://www.linux-iscsi.org/index.php/Howto#Start_target_core_mod
>> Some questions among many I have:
>>
>> 1. Where and how TCM is better over the existing mainline target 
>> framework STGT to justify such a move as adding a new piece of code 
>> doing the same job?
> 
> I think from reading my original email and the list of patches it is
> quite obvious to the people involved what are the major pieces of
> functionality available in TCM v3.2 that people actually *want* and
> *care* about that is not available in the existing STGT framework.

Probably, I'm not sufficiently smart, but benefits of "TCM v3.2" aren't 
obvious for me at all. Could you list them point by point?

Especially interesting, why the above mentioned features can't be added 
in the STGT instead of creating something a whole new?

>> 2. On the linux-iscsi.org main page (BTW, the comparison on that page 
>> isn't fully correct about STGT and SCST) you claim that TCM is a generic 
>> target engine. But how can you prove (and test) that if there are no 
>> hardware target drivers for TCM?
> 
> I am not sure why you think you need hardware target drivers to prove
> that a target engine is generic..?  Why don't you look at the TCM v3.2
> code (like everyone else is doing) and see for yourself..?

I looked there many times. Hardware target drivers fundamentally 
different from software drivers, because they need to do some processing 
on IRQ level. You can't test the corresponding target's framework 
branches and prove they provide sufficient functionality using a driver, 
which doesn't need such functionality.

>> Also patch 7 implements MIBs through procfs. Procfs is for long time 
>> forbidden for adding anything like that, so you can't push it in the 
>> mainline.
> 
> There are other subsystems (Linux/Net and SCTP for example) that use
> procfs for read-only MIBs statistics, so TCM actually having an
> implementation of SCSI MIBs available (no other target projects have
> SCSI MIBs btw) under procfs is hardly a show-stopper considering the
> *entire* control path for TCM v3.2 is using upstream configfs code
> anyways.

Addition of new non-process related entries in /proc is fully forbidden 
and it doesn't matter if those entries used on "control path" or in 
corner cases.

Vlad