* TCM/LIO ib_srpt testing
@ 2011-05-17 10:37 Vasilis Liaskovitis
2011-05-17 21:37 ` Nicholas A. Bellinger
2011-05-18 17:05 ` Bart Van Assche
0 siblings, 2 replies; 5+ messages in thread
From: Vasilis Liaskovitis @ 2011-05-17 10:37 UTC (permalink / raw)
To: linux-scsi
Hi,
I am trying to test the TCM/LIO ib_srpt target driver from master
branch of lio-core-2.6
In the past, using branch tcm_ib_srpt-38, I have successfully
initialized the srp target and initiator with the "manual steps" in:
http://linux-iscsi.org/wiki/SCSI_RDMA_Protocol#Manual_steps
However with master, the manual steps fail at:
root@server1:~# mkdir -p /sys/kernel/config/target/srpt/mlx4_0/mlx4_0/lun/lun_0
mkdir: cannot create directory
`/sys/kernel/config/target/srpt/mlx4_0': Invalid argument
Has the recommended ib_srpt initialization procedure changed recently?
The target and initiator are the same machine in this test. This is
on Mellanox QDR (mlx4 backend driver)
When I tested the tcm_ib_srpt-38 branch of lio-core-2.6, the same
manual steps worked fine:
[ 3725.279283] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
[ 3725.279336] Initialized struct target_fabric_configfs:
ffff88061a3f5800 for srpt
[ 3725.279416] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
[ 3725.279539] TARGET_CORE[srpt]: Allocated Normal struct
se_portal_group for endpoint: 0x2c903000f5e7a, Portal Tag: 1
[ 3725.282984] Target_Core_ConfigFS: REGISTER -> group:
ffffffffa0273380 name: srpt
[ 3725.283071] Target_Core_ConfigFS: REGISTER -> Located fabric: srpt
[ 3725.284622] Target_Core_ConfigFS: REGISTER tfc_wwn_cit -> ffff88061a3f5b08
[ 3725.284677] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: srpt
[ 3725.284727] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for srpt
[ 3725.286314] iblock/srpt: Adding to default ALUA Target Port Group:
alua/default_tg_pt_gp
[ 3725.286403] srpt_TPG[1]_LUN[0] - Activated srpt Logical Unit from CORE HBA: 1
[ 3725.377681] srpt_TPG[1] - Added ACL with TCQ Depth: 1 for srpt
Initiator Node: 0x0002c903000f5e7b
[ 3725.379182] srpt_TPG[1]_LUN[0->0] - Added RW ACL for
InitiatorNode: 0x0002c903000f5e7b
And the scsi device seemed to appear correctly on the initiator side:
[ 3758.027108] scsi host1: ib_srp: new target: id_ext 0002c903000f5e7a
ioc_guid 0002c903000f5e7a pkey ffff service_id 0002c903000f5e7a dgid
fe80:0000:0000:0000:0002:c903:000f:5e7b
[ 3758.027959] Received SRP_LOGIN_REQ with i_port_id
0x0:0x2c903000f5e7b, t_port_id 0x2c903000f5e7a:0x2c903000f5e7a and
it_iu_len 260 on port 1 (guid=0xfe80000000000000:0x2c903000f5e7b)
[ 3758.031561] Session : kernel thread ib_srpt_compl (PID 2396) started
[ 3758.031631] TARGET_CORE[srpt]: Registered fabric_sess_ptr: ffff88021aaad000
[ 3758.031898] scsi1 : SRP.T10:0002C903000F5E7A
[ 3758.032317] scsi 1:0:0:0: Direct-Access LIO-ORG IBLOCK
4.0 PQ: 0 ANSI: 5
[ 3758.032751] sd 1:0:0:0: [sdc] 9762242560 512-byte logical blocks:
(4.99 TB/4.54 TiB)
[ 3758.033193] sd 1:0:0:0: [sdc] Write Protect is off
[ 3758.033245] sd 1:0:0:0: [sdc] Mode Sense: 2f 00 00 00
[ 3758.033389] sd 1:0:0:0: [sdc] Write cache: disabled, read cache:
enabled, doesn't support DPO or FUA
[ 3768.046220] sdc: unknown partition table
[ 3768.046823] sd 1:0:0:0: [sdc] Attached SCSI disk
However, the tcm_ib_srpt-38 branch results in a kernel panic during
I/O traffic (e.g. doing simple dd tests on exported disk)
I was hoping that the latest patches in master would fix this.
Does anyone have tips on initializing ib_srpt manually, with
lio-utils, or with rtsadmin?
.
Also, If there's another branch more suitable for the latest LIO
patches please lat me know.
thanks,
- Vasilis
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: TCM/LIO ib_srpt testing
2011-05-17 10:37 TCM/LIO ib_srpt testing Vasilis Liaskovitis
@ 2011-05-17 21:37 ` Nicholas A. Bellinger
2011-05-18 10:19 ` Bart Van Assche
2011-05-18 17:05 ` Bart Van Assche
1 sibling, 1 reply; 5+ messages in thread
From: Nicholas A. Bellinger @ 2011-05-17 21:37 UTC (permalink / raw)
To: Vasilis Liaskovitis; +Cc: linux-scsi, Bart Van Assche
On Tue, 2011-05-17 at 12:37 +0200, Vasilis Liaskovitis wrote:
> Hi,
>
> I am trying to test the TCM/LIO ib_srpt target driver from master
> branch of lio-core-2.6
>
Greetings Vasilis,
> In the past, using branch tcm_ib_srpt-38, I have successfully
> initialized the srp target and initiator with the "manual steps" in:
> http://linux-iscsi.org/wiki/SCSI_RDMA_Protocol#Manual_steps
>
> However with master, the manual steps fail at:
> root@server1:~# mkdir -p /sys/kernel/config/target/srpt/mlx4_0/mlx4_0/lun/lun_0
> mkdir: cannot create directory
> `/sys/kernel/config/target/srpt/mlx4_0': Invalid argument
> Has the recommended ib_srpt initialization procedure changed recently?
>
Yes. In order for ib_srpt to properly work with rtslib+rtsadmin, the
control plane has been changed from Bart's original patch to reference
HW IB Port GUID instead of the symbolic /sys/class/infiniband/ name
The example of this is available here:
http://www.linux-iscsi.org/wiki/SCSI_RDMA_Protocol/RTSadmin
> The target and initiator are the same machine in this test. This is
> on Mellanox QDR (mlx4 backend driver)
>
> When I tested the tcm_ib_srpt-38 branch of lio-core-2.6, the same
> manual steps worked fine:
>
> [ 3725.279283] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
> [ 3725.279336] Initialized struct target_fabric_configfs:
> ffff88061a3f5800 for srpt
> [ 3725.279416] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
> [ 3725.279539] TARGET_CORE[srpt]: Allocated Normal struct
> se_portal_group for endpoint: 0x2c903000f5e7a, Portal Tag: 1
> [ 3725.282984] Target_Core_ConfigFS: REGISTER -> group:
> ffffffffa0273380 name: srpt
> [ 3725.283071] Target_Core_ConfigFS: REGISTER -> Located fabric: srpt
> [ 3725.284622] Target_Core_ConfigFS: REGISTER tfc_wwn_cit -> ffff88061a3f5b08
> [ 3725.284677] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: srpt
> [ 3725.284727] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for srpt
> [ 3725.286314] iblock/srpt: Adding to default ALUA Target Port Group:
> alua/default_tg_pt_gp
> [ 3725.286403] srpt_TPG[1]_LUN[0] - Activated srpt Logical Unit from CORE HBA: 1
> [ 3725.377681] srpt_TPG[1] - Added ACL with TCQ Depth: 1 for srpt
> Initiator Node: 0x0002c903000f5e7b
> [ 3725.379182] srpt_TPG[1]_LUN[0->0] - Added RW ACL for
> InitiatorNode: 0x0002c903000f5e7b
>
> And the scsi device seemed to appear correctly on the initiator side:
>
> [ 3758.027108] scsi host1: ib_srp: new target: id_ext 0002c903000f5e7a
> ioc_guid 0002c903000f5e7a pkey ffff service_id 0002c903000f5e7a dgid
> fe80:0000:0000:0000:0002:c903:000f:5e7b
> [ 3758.027959] Received SRP_LOGIN_REQ with i_port_id
> 0x0:0x2c903000f5e7b, t_port_id 0x2c903000f5e7a:0x2c903000f5e7a and
> it_iu_len 260 on port 1 (guid=0xfe80000000000000:0x2c903000f5e7b)
> [ 3758.031561] Session : kernel thread ib_srpt_compl (PID 2396) started
> [ 3758.031631] TARGET_CORE[srpt]: Registered fabric_sess_ptr: ffff88021aaad000
> [ 3758.031898] scsi1 : SRP.T10:0002C903000F5E7A
> [ 3758.032317] scsi 1:0:0:0: Direct-Access LIO-ORG IBLOCK
> 4.0 PQ: 0 ANSI: 5
> [ 3758.032751] sd 1:0:0:0: [sdc] 9762242560 512-byte logical blocks:
> (4.99 TB/4.54 TiB)
> [ 3758.033193] sd 1:0:0:0: [sdc] Write Protect is off
> [ 3758.033245] sd 1:0:0:0: [sdc] Mode Sense: 2f 00 00 00
> [ 3758.033389] sd 1:0:0:0: [sdc] Write cache: disabled, read cache:
> enabled, doesn't support DPO or FUA
> [ 3768.046220] sdc: unknown partition table
> [ 3768.046823] sd 1:0:0:0: [sdc] Attached SCSI disk
>
> However, the tcm_ib_srpt-38 branch results in a kernel panic during
> I/O traffic (e.g. doing simple dd tests on exported disk)
> I was hoping that the latest patches in master would fix this.
>
Note that tcm_ib_srpt-38 is an out-of-date branch that is missing some
necessary target core fixes..
At this point all of the active fabric module development branches
(including the ones for ib_srpt) have been merged into branch lio-4.1
@ .39-rc7 code, and into branch lio-4.0 @ .38.3. Please use one of
these for your testing with ib_srpt to get the latest drivers/target/
bugfixes currently in James'es queue for mainline.
> Does anyone have tips on initializing ib_srpt manually, with
> lio-utils, or with rtsadmin?
Btw, rtsadmin-v2 includes a default /var/target/fabric/ib_srpt.spec very
similar to what's in the above URL, and the IB HCA Port GUIDs will
automatically appear as creatable wwn= parameters under in the top level
fabric object /ib_srpt
--nab
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: TCM/LIO ib_srpt testing
2011-05-17 21:37 ` Nicholas A. Bellinger
@ 2011-05-18 10:19 ` Bart Van Assche
2011-05-19 4:25 ` Nicholas A. Bellinger
0 siblings, 1 reply; 5+ messages in thread
From: Bart Van Assche @ 2011-05-18 10:19 UTC (permalink / raw)
To: Nicholas A. Bellinger; +Cc: Vasilis Liaskovitis, linux-scsi, Christoph Hellwig
On Tue, May 17, 2011 at 11:37 PM, Nicholas A. Bellinger
<nab@linux-iscsi.org> wrote:
> On Tue, 2011-05-17 at 12:37 +0200, Vasilis Liaskovitis wrote:
> > In the past, using branch tcm_ib_srpt-38, I have successfully
> > initialized the srp target and initiator with the "manual steps" in:
> > http://linux-iscsi.org/wiki/SCSI_RDMA_Protocol#Manual_steps
> >
> > However with master, the manual steps fail at:
> > root@server1:~# mkdir -p /sys/kernel/config/target/srpt/mlx4_0/mlx4_0/lun/lun_0
> > mkdir: cannot create directory
> > `/sys/kernel/config/target/srpt/mlx4_0': Invalid argument
> > Has the recommended ib_srpt initialization procedure changed recently?
>
> Yes. In order for ib_srpt to properly work with rtslib+rtsadmin, the
> control plane has been changed from Bart's original patch to reference
> HW IB Port GUID instead of the symbolic /sys/class/infiniband/ name
Most users expect the original behavior. If you agree I can prepare a
new patch where the target name format can be selected via a kernel
module parameter and in which Christoph Hellwig's recent comments are
addressed too.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: TCM/LIO ib_srpt testing
2011-05-17 10:37 TCM/LIO ib_srpt testing Vasilis Liaskovitis
2011-05-17 21:37 ` Nicholas A. Bellinger
@ 2011-05-18 17:05 ` Bart Van Assche
1 sibling, 0 replies; 5+ messages in thread
From: Bart Van Assche @ 2011-05-18 17:05 UTC (permalink / raw)
To: Vasilis Liaskovitis; +Cc: linux-scsi
On Tue, May 17, 2011 at 12:37 PM, Vasilis Liaskovitis
<vliaskov@gmail.com> wrote:
> I am trying to test the TCM/LIO ib_srpt target driver from master
> branch of lio-core-2.6
>
> However, the tcm_ib_srpt-38 branch results in a kernel panic during
> I/O traffic (e.g. doing simple dd tests on exported disk)
> I was hoping that the latest patches in master would fix this.
I have never seen such behavior during any of the tests I ran. I'll
see whether I can reproduce that behavior with lio-4.1.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: TCM/LIO ib_srpt testing
2011-05-18 10:19 ` Bart Van Assche
@ 2011-05-19 4:25 ` Nicholas A. Bellinger
0 siblings, 0 replies; 5+ messages in thread
From: Nicholas A. Bellinger @ 2011-05-19 4:25 UTC (permalink / raw)
To: Bart Van Assche; +Cc: Vasilis Liaskovitis, linux-scsi, Christoph Hellwig
On Wed, 2011-05-18 at 12:19 +0200, Bart Van Assche wrote:
> On Tue, May 17, 2011 at 11:37 PM, Nicholas A. Bellinger
> <nab@linux-iscsi.org> wrote:
> > On Tue, 2011-05-17 at 12:37 +0200, Vasilis Liaskovitis wrote:
> > > In the past, using branch tcm_ib_srpt-38, I have successfully
> > > initialized the srp target and initiator with the "manual steps" in:
> > > http://linux-iscsi.org/wiki/SCSI_RDMA_Protocol#Manual_steps
> > >
> > > However with master, the manual steps fail at:
> > > root@server1:~# mkdir -p /sys/kernel/config/target/srpt/mlx4_0/mlx4_0/lun/lun_0
> > > mkdir: cannot create directory
> > > `/sys/kernel/config/target/srpt/mlx4_0': Invalid argument
> > > Has the recommended ib_srpt initialization procedure changed recently?
> >
> > Yes. In order for ib_srpt to properly work with rtslib+rtsadmin, the
> > control plane has been changed from Bart's original patch to reference
> > HW IB Port GUID instead of the symbolic /sys/class/infiniband/ name
>
> Most users expect the original behavior. If you agree I can prepare a
> new patch where the target name format can be selected via a kernel
> module parameter and in which Christoph Hellwig's recent comments are
> addressed too.
>
Since we don't have any real userspace to break with
/sys/kernel/config/target/srpt/ at this point, we need to avoid multiple
configfs layouts in mainline without a really solid backwards compatible
reason for needing to do so.
I don't have a strong issue with this myself, and would even be happy to
support it in lio upstream and backports if it's really useful, but I
think for mainline we have to decide on a single configfs layout for
modern v2 userspace.
Thank you for your comments,
--nab
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2011-05-19 4:33 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-05-17 10:37 TCM/LIO ib_srpt testing Vasilis Liaskovitis
2011-05-17 21:37 ` Nicholas A. Bellinger
2011-05-18 10:19 ` Bart Van Assche
2011-05-19 4:25 ` Nicholas A. Bellinger
2011-05-18 17:05 ` Bart Van Assche
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.