* iSER initiator in 4.5 is unhappy..
@ 2016-04-04 13:46 Christoph Hellwig
[not found] ` <20160404134656.GA30269-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-04 13:46 UTC (permalink / raw)
To: sagi-NQWnxTmZq1alnMjI0IkVqw, linux-rdma-u79uwXL29TY76Z2rM5mHXA
When trying to use iSER with a 4.5 initiator it's rather unhappy and
I get a spew of messages like:
[ 236.814107] iser: iser_fast_reg_fmr: ib_fmr_pool_map_phys failed: -22
[ 236.823100] iser: iser_prepare_read_cmd: Failed to set up Data-IN RDMA
[ 236.832101] iser: iser_send_command: conn ffff88002b16e328 failed task->itt 75 err -22
If I manually comment out the FMR selection in iser_memory.c and force
it to use FRs things seem to work fine.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160404134656.GA30269-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2016-04-04 14:42 ` Max Gurtovoy
[not found] ` <57027D48.4000500-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2016-04-04 17:29 ` sagig
1 sibling, 1 reply; 20+ messages in thread
From: Max Gurtovoy @ 2016-04-04 14:42 UTC (permalink / raw)
To: Christoph Hellwig, sagi-NQWnxTmZq1alnMjI0IkVqw,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Thanks, I'll try to reprodue it.
Can you give me a scenario/setup details ?
On 4/4/2016 4:46 PM, Christoph Hellwig wrote:
> When trying to use iSER with a 4.5 initiator it's rather unhappy and
> I get a spew of messages like:
>
> [ 236.814107] iser: iser_fast_reg_fmr: ib_fmr_pool_map_phys failed: -22
> [ 236.823100] iser: iser_prepare_read_cmd: Failed to set up Data-IN RDMA
> [ 236.832101] iser: iser_send_command: conn ffff88002b16e328 failed task->itt 75 err -22
>
> If I manually comment out the FMR selection in iser_memory.c and force
> it to use FRs things seem to work fine.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <57027D48.4000500-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2016-04-04 15:10 ` Christoph Hellwig
0 siblings, 0 replies; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-04 15:10 UTC (permalink / raw)
To: Max Gurtovoy
Cc: Christoph Hellwig, sagi-NQWnxTmZq1alnMjI0IkVqw,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
LIO target running either 4.5 or 4.6-rc2 backed by an NVMe device on the
other side. The systems are connected using Connect-X 3 and 4 devices,
and based on the IP address used it should be routed to the Connect-X 3
ones on either side.
I did a mkfs.xfs on the device (works) and then started a run of the
"quick" xfstests group, which didn't even start output for the first
test - based on that it happened very early during mount.
FYI, FRs didn't look much better either a few tests into the xfstests
run:
[ 343.062657] iser: iser_prepare_read_cmd: Failed to set up Data-IN RDMA
[ 343.070861] iser: iser_send_command: conn ffff8804a4682328 failed task->itt 6 err -22
[ 343.086357] iser: iser_fast_reg_mr: failed to map sg (68/69)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160404134656.GA30269-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-04 14:42 ` Max Gurtovoy
@ 2016-04-04 17:29 ` sagig
[not found] ` <5702A46F.5070109-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
1 sibling, 1 reply; 20+ messages in thread
From: sagig @ 2016-04-04 17:29 UTC (permalink / raw)
To: Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
> When trying to use iSER with a 4.5 initiator it's rather unhappy and
> I get a spew of messages like:
>
> [ 236.814107] iser: iser_fast_reg_fmr: ib_fmr_pool_map_phys failed: -22
> [ 236.823100] iser: iser_prepare_read_cmd: Failed to set up Data-IN RDMA
> [ 236.832101] iser: iser_send_command: conn ffff88002b16e328 failed task->itt 75 err -22
>
> If I manually comment out the FMR selection in iser_memory.c and force
> it to use FRs things seem to work fine.
Thanks for reporting,
This must be the bio splitting code again...
Christoph, any chance you can run the sg_gaps I/O generator
I sent a while back (Don't yet have access to rdma devices)?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <5702A46F.5070109-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
@ 2016-04-04 17:38 ` sagig
[not found] ` <5702A6AC.9020008-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-04 22:46 ` Christoph Hellwig
1 sibling, 1 reply; 20+ messages in thread
From: sagig @ 2016-04-04 17:38 UTC (permalink / raw)
To: Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
Odd, I see that Ming's patches made it to 4.5...
Can you verify that this patches exist in your tree:
e827091cb1bc block: merge: get the 1st and last bvec via helpers
25e71a99f10e block: get the 1st and last bvec via helpers
e0af29171aa8 block: check virt boundary in bio_will_gap()
7bcd79ac50d9 block: bio: introduce helpers to get the 1st and last bvec
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <5702A46F.5070109-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-04 17:38 ` sagig
@ 2016-04-04 22:46 ` Christoph Hellwig
1 sibling, 0 replies; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-04 22:46 UTC (permalink / raw)
To: sagig; +Cc: Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Mon, Apr 04, 2016 at 08:29:19PM +0300, sagig wrote:
> Christoph, any chance you can run the sg_gaps I/O generator
> I sent a while back (Don't yet have access to rdma devices)?
A run as ./sg-gaps -d /dev/sdb -s 16 -n 8 -v doesn't show any errors,
anything else I should try?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <5702A6AC.9020008-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
@ 2016-04-04 22:58 ` Christoph Hellwig
[not found] ` <20160404225831.GA6725-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-04 22:58 UTC (permalink / raw)
To: sagig; +Cc: Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Mon, Apr 04, 2016 at 08:38:52PM +0300, sagig wrote:
> Odd, I see that Ming's patches made it to 4.5...
>
> Can you verify that this patches exist in your tree:
> e827091cb1bc block: merge: get the 1st and last bvec via helpers
> 25e71a99f10e block: get the 1st and last bvec via helpers
> e0af29171aa8 block: check virt boundary in bio_will_gap()
> 7bcd79ac50d9 block: bio: introduce helpers to get the 1st and last bvec
I've done a clean rebuild because I didn't trust the previous one
lingering around, and the syptoms are slightly different now, in that I
still get the same spew of messages, but it seems like xfstests is
very slowly progressing regardless.
Also when testing srp I see similar messages with the same setup:
[ 130.896116] scsi host7: ib_srp: Failed to map data (-12)
[ 130.896225] scsi host7: ib_srp: Failed to map data (-12)
[ 130.896232] scsi host7: ib_srp: Failed to map data (-12)
[ 130.896235] scsi host7: ib_srp: Failed to map data (-12)
[ 130.896835] scsi host7: ib_srp: Failed to map data (-12)
[ 130.993339] scsi host7: ib_srp: Failed to map data (-12)
[ 130.993345] scsi host7: ib_srp: Failed to map data (-12)
[ 130.993349] scsi host7: ib_srp: Failed to map data (-12)
[ 130.993841] scsi host7: ib_srp: Failed to map data (-12)
[ 130.993844] scsi host7: ib_srp: Failed to map data (-12)
[ 131.035723] scsi host7: ib_srp: Failed to map data (-12)
[ 131.058006] scsi host7: ib_srp: Failed to map data (-12)
[ 131.058059] scsi host7: ib_srp: Failed to map data (-12)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160404225831.GA6725-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2016-04-05 8:55 ` sagig
[not found] ` <57037D8E.4030506-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: sagig @ 2016-04-05 8:55 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
>> Odd, I see that Ming's patches made it to 4.5...
>>
>> Can you verify that this patches exist in your tree:
>> e827091cb1bc block: merge: get the 1st and last bvec via helpers
>> 25e71a99f10e block: get the 1st and last bvec via helpers
>> e0af29171aa8 block: check virt boundary in bio_will_gap()
>> 7bcd79ac50d9 block: bio: introduce helpers to get the 1st and last bvec
>
> I've done a clean rebuild because I didn't trust the previous one
> lingering around, and the syptoms are slightly different now, in that I
> still get the same spew of messages, but it seems like xfstests is
> very slowly progressing regardless.
>
> Also when testing srp I see similar messages with the same setup:
>
> [ 130.896116] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.896225] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.896232] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.896235] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.896835] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.993339] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.993345] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.993349] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.993841] scsi host7: ib_srp: Failed to map data (-12)
> [ 130.993844] scsi host7: ib_srp: Failed to map data (-12)
> [ 131.035723] scsi host7: ib_srp: Failed to map data (-12)
> [ 131.058006] scsi host7: ib_srp: Failed to map data (-12)
> [ 131.058059] scsi host7: ib_srp: Failed to map data (-12)
>
This implies that maybe something broke in ib_sg_to_pages:
Can you run iser with this patch applied (FR or FMR):
--
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c
b/drivers/infiniband/ulp/iser/iser_memory.c
index 44cc85f206f3..67499d8f6a3a 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -143,7 +143,7 @@ static void iser_data_buf_dump(struct iser_data_buf
*data,
int i;
for_each_sg(data->sg, sg, data->dma_nents, i)
- iser_dbg("sg[%d] dma_addr:0x%lX page:0x%p "
+ iser_err("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
i, (unsigned long)ib_sg_dma_address(ibdev, sg),
sg_page(sg), sg->offset,
@@ -450,6 +450,7 @@ static int iser_fast_reg_mr(struct iscsi_iser_task
*iser_task,
if (unlikely(n != mem->size)) {
iser_err("failed to map sg (%d/%d)\n",
n, mem->size);
+ iser_data_buf_dump(mem,
iser_task->iser_conn->ib_conn.device->ib_device);
return n < 0 ? n : -EINVAL;
}
--
This sg dump will tell us what sort of scatterlist we can't
handle.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <57037D8E.4030506-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
@ 2016-04-05 9:43 ` Christoph Hellwig
[not found] ` <20160405094339.GA1622-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-05 9:43 UTC (permalink / raw)
To: sagig; +Cc: Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Tue, Apr 05, 2016 at 11:55:42AM +0300, sagig wrote:
> This implies that maybe something broke in ib_sg_to_pages:
>
> Can you run iser with this patch applied (FR or FMR):
For FMR this won't trigger, but here is the FR run:
[ 63.016616] iser: iser_fast_reg_mr: failed to map sg (101/102)
[ 63.023183] iser: iser_data_buf_dump: sg[0] dma_addr:0x90E15D200
page:0xffffea0024385740 off:0x200 sz:0xe00 dma_len:0xe00
[ 63.035466] iser: iser_data_buf_dump: sg[1] dma_addr:0x90611D000
page:0xffffea0024184740 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.047749] iser: iser_data_buf_dump: sg[2] dma_addr:0x90EDB3000
page:0xffffea00243b6cc0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.060050] iser: iser_data_buf_dump: sg[3] dma_addr:0x9062CB000
page:0xffffea002418b2c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.072328] iser: iser_data_buf_dump: sg[4] dma_addr:0x9061EA000
page:0xffffea0024187a80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.084614] iser: iser_data_buf_dump: sg[5] dma_addr:0x90E7D2000
page:0xffffea002439f480 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.096907] iser: iser_data_buf_dump: sg[6] dma_addr:0x90E5EF000
page:0xffffea0024397bc0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.109195] iser: iser_data_buf_dump: sg[7] dma_addr:0x90B755000
page:0xffffea00242dd540 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.121487] iser: iser_data_buf_dump: sg[8] dma_addr:0x9016F2000
page:0xffffea002405bc80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.133780] iser: iser_data_buf_dump: sg[9] dma_addr:0x906194000
page:0xffffea0024186500 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.146074] iser: iser_data_buf_dump: sg[10] dma_addr:0x9017A2000
page:0xffffea002405e880 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.158456] iser: iser_data_buf_dump: sg[11] dma_addr:0x92D4BC000
page:0xffffea0024b52f00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.170841] iser: iser_data_buf_dump: sg[12] dma_addr:0x92D4BF000
page:0xffffea0024b52fc0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.183233] iser: iser_data_buf_dump: sg[13] dma_addr:0x93FA69000
page:0xffffea0024fe9a40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.195608] iser: iser_data_buf_dump: sg[14] dma_addr:0x90E011000
page:0xffffea0024380440 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.208082] iser: iser_data_buf_dump: sg[15] dma_addr:0x90EACD000
page:0xffffea00243ab340 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.220499] iser: iser_data_buf_dump: sg[16] dma_addr:0x93FA6A000
page:0xffffea0024fe9a80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.232916] iser: iser_data_buf_dump: sg[17] dma_addr:0x93FD23000
page:0xffffea0024ff48c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.245334] iser: iser_data_buf_dump: sg[18] dma_addr:0x903D57000
page:0xffffea00240f55c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.257748] iser: iser_data_buf_dump: sg[19] dma_addr:0x93FF38000
page:0xffffea0024ffce00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.270167] iser: iser_data_buf_dump: sg[20] dma_addr:0x90F035000
page:0xffffea00243c0d40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.282584] iser: iser_data_buf_dump: sg[21] dma_addr:0x901719000
page:0xffffea002405c640 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.295000] iser: iser_data_buf_dump: sg[22] dma_addr:0x90EBA3000
page:0xffffea00243ae8c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.307422] iser: iser_data_buf_dump: sg[23] dma_addr:0x90E20B000
page:0xffffea00243882c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.319837] iser: iser_data_buf_dump: sg[24] dma_addr:0x901722000
page:0xffffea002405c880 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.332255] iser: iser_data_buf_dump: sg[25] dma_addr:0x93FA52000
page:0xffffea0024fe9480 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.344679] iser: iser_data_buf_dump: sg[26] dma_addr:0x90E023000
page:0xffffea00243808c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.357095] iser: iser_data_buf_dump: sg[27] dma_addr:0x901679000
page:0xffffea0024059e40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.369518] iser: iser_data_buf_dump: sg[28] dma_addr:0x90DC4A000
page:0xffffea0024371280 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.381937] iser: iser_data_buf_dump: sg[29] dma_addr:0x90160D000
page:0xffffea0024058340 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.394353] iser: iser_data_buf_dump: sg[30] dma_addr:0x90E9F3000
page:0xffffea00243a7cc0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.406769] iser: iser_data_buf_dump: sg[31] dma_addr:0x90E6CA000
page:0xffffea002439b280 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.419175] iser: iser_data_buf_dump: sg[32] dma_addr:0x90E6DA000
page:0xffffea002439b680 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.431596] iser: iser_data_buf_dump: sg[33] dma_addr:0x901787000
page:0xffffea002405e1c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.444013] iser: iser_data_buf_dump: sg[34] dma_addr:0x90E6A7000
page:0xffffea002439a9c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.456430] iser: iser_data_buf_dump: sg[35] dma_addr:0x90178C000
page:0xffffea002405e300 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.468853] iser: iser_data_buf_dump: sg[36] dma_addr:0x90E27A000
page:0xffffea0024389e80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.481278] iser: iser_data_buf_dump: sg[37] dma_addr:0x90E6CC000
page:0xffffea002439b300 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.493702] iser: iser_data_buf_dump: sg[38] dma_addr:0x90E6CF000
page:0xffffea002439b3c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.506116] iser: iser_data_buf_dump: sg[39] dma_addr:0x901790000
page:0xffffea002405e400 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.518531] iser: iser_data_buf_dump: sg[40] dma_addr:0x90E6D0000
page:0xffffea002439b400 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.530955] iser: iser_data_buf_dump: sg[41] dma_addr:0x90E6D6000
page:0xffffea002439b580 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.543370] iser: iser_data_buf_dump: sg[42] dma_addr:0x92D266000
page:0xffffea0024b49980 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.555778] iser: iser_data_buf_dump: sg[43] dma_addr:0x90177B000
page:0xffffea002405dec0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.568886] iser: iser_data_buf_dump: sg[44] dma_addr:0x92D265000
page:0xffffea0024b49940 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.581950] iser: iser_data_buf_dump: sg[45] dma_addr:0x90E240000
page:0xffffea0024389000 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.595005] iser: iser_data_buf_dump: sg[46] dma_addr:0x901635000
page:0xffffea0024058d40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.608022] iser: iser_data_buf_dump: sg[47] dma_addr:0x9062B8000
page:0xffffea002418ae00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.620986] iser: iser_data_buf_dump: sg[48] dma_addr:0x90EACF000
page:0xffffea00243ab3c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.633939] iser: iser_data_buf_dump: sg[49] dma_addr:0x901631000
page:0xffffea0024058c40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.646887] iser: iser_data_buf_dump: sg[50] dma_addr:0x90E8DE000
page:0xffffea00243a3780 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.659842] iser: iser_data_buf_dump: sg[51] dma_addr:0x9034F4000
page:0xffffea00240d3d00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.672794] iser: iser_data_buf_dump: sg[52] dma_addr:0x903CF7000
page:0xffffea00240f3dc0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.685744] iser: iser_data_buf_dump: sg[53] dma_addr:0x90E511000
page:0xffffea0024394440 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.698371] iser: iser_data_buf_dump: sg[54] dma_addr:0x90F328000
page:0xffffea00243cca00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.711008] iser: iser_data_buf_dump: sg[55] dma_addr:0x90E3BC000
page:0xffffea002438ef00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.723643] iser: iser_data_buf_dump: sg[56] dma_addr:0x90F32E000
page:0xffffea00243ccb80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.736277] iser: iser_data_buf_dump: sg[57] dma_addr:0x90F323000
page:0xffffea00243cc8c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.748893] iser: iser_data_buf_dump: sg[58] dma_addr:0x9064CD000
page:0xffffea0024193340 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.761509] iser: iser_data_buf_dump: sg[59] dma_addr:0x90169F000
page:0xffffea002405a7c0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.774127] iser: iser_data_buf_dump: sg[60] dma_addr:0x9062B4000
page:0xffffea002418ad00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.786744] iser: iser_data_buf_dump: sg[61] dma_addr:0x90310A000
page:0xffffea00240c4280 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.799371] iser: iser_data_buf_dump: sg[62] dma_addr:0x93FE24000
page:0xffffea0024ff8900 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.811990] iser: iser_data_buf_dump: sg[63] dma_addr:0x9063BA000
page:0xffffea002418ee80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.824608] iser: iser_data_buf_dump: sg[64] dma_addr:0x903CFC000
page:0xffffea00240f3f00 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.837228] iser: iser_data_buf_dump: sg[65] dma_addr:0x903106000
page:0xffffea00240c4180 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.849835] iser: iser_data_buf_dump: sg[66] dma_addr:0x90B93F000
page:0xffffea00242e4fc0 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.862455] iser: iser_data_buf_dump: sg[67] dma_addr:0x90E9C9000
page:0xffffea00243a7240 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.875073] iser: iser_data_buf_dump: sg[68] dma_addr:0x90176D000
page:0xffffea002405db40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.887689] iser: iser_data_buf_dump: sg[69] dma_addr:0x90177E000
page:0xffffea002405df80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.900308] iser: iser_data_buf_dump: sg[70] dma_addr:0x90E501000
page:0xffffea0024394040 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.912926] iser: iser_data_buf_dump: sg[71] dma_addr:0x9016FE000
page:0xffffea002405bf80 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.925549] iser: iser_data_buf_dump: sg[72] dma_addr:0x9014F5000
page:0xffffea0024053d40 off:0x0 sz:0x1000 dma_len:0x1000
[ 63.938165] iser: iser_data_buf_dump: sg[73] dma_addr:0x90EB96000
page:0xffffea00243ae580 off:0x0 sz:0x2000 dma_len:0x2000
[ 63.950779] iser: iser_data_buf_dump: sg[74] dma_addr:0x90E22E000
page:0xffffea0024388b80 off:0x0 sz:0x2000 dma_len:0x2000
[ 63.963400] iser: iser_data_buf_dump: sg[75] dma_addr:0x901636000
page:0xffffea0024058d80 off:0x0 sz:0x2000 dma_len:0x2000
[ 63.976015] iser: iser_data_buf_dump: sg[76] dma_addr:0x90169C000
page:0xffffea002405a700 off:0x0 sz:0x2000 dma_len:0x2000
[ 63.988629] iser: iser_data_buf_dump: sg[77] dma_addr:0x90163A000
page:0xffffea0024058e80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.001238] iser: iser_data_buf_dump: sg[78] dma_addr:0x90177C000
page:0xffffea002405df00 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.013852] iser: iser_data_buf_dump: sg[79] dma_addr:0x903FBE000
page:0xffffea00240fef80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.026462] iser: iser_data_buf_dump: sg[80] dma_addr:0x9061B6000
page:0xffffea0024186d80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.039077] iser: iser_data_buf_dump: sg[81] dma_addr:0x90E7FA000
page:0xffffea002439fe80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.051697] iser: iser_data_buf_dump: sg[82] dma_addr:0x90E9D4000
page:0xffffea00243a7500 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.064310] iser: iser_data_buf_dump: sg[83] dma_addr:0x90E502000
page:0xffffea0024394080 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.076933] iser: iser_data_buf_dump: sg[84] dma_addr:0x901686000
page:0xffffea002405a180 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.089548] iser: iser_data_buf_dump: sg[85] dma_addr:0x903458000
page:0xffffea00240d1600 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.102163] iser: iser_data_buf_dump: sg[86] dma_addr:0x90E43A000
page:0xffffea0024390e80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.114779] iser: iser_data_buf_dump: sg[87] dma_addr:0x90F190000
page:0xffffea00243c6400 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.127395] iser: iser_data_buf_dump: sg[88] dma_addr:0x9016F6000
page:0xffffea002405bd80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.140002] iser: iser_data_buf_dump: sg[89] dma_addr:0x92D3AC000
page:0xffffea0024b4eb00 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.152621] iser: iser_data_buf_dump: sg[90] dma_addr:0x903F86000
page:0xffffea00240fe180 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.165239] iser: iser_data_buf_dump: sg[91] dma_addr:0x901598000
page:0xffffea0024056600 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.177857] iser: iser_data_buf_dump: sg[92] dma_addr:0x90F7D4000
page:0xffffea00243df500 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.190476] iser: iser_data_buf_dump: sg[93] dma_addr:0x90E31E000
page:0xffffea002438c780 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.203147] iser: iser_data_buf_dump: sg[94] dma_addr:0x90E31A000
page:0xffffea002438c680 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.215764] iser: iser_data_buf_dump: sg[95] dma_addr:0x90E440000
page:0xffffea0024391000 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.228380] iser: iser_data_buf_dump: sg[96] dma_addr:0x90E43E000
page:0xffffea0024390f80 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.240995] iser: iser_data_buf_dump: sg[97] dma_addr:0x901586000
page:0xffffea0024056180 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.253611] iser: iser_data_buf_dump: sg[98] dma_addr:0x90F390000
page:0xffffea00243ce400 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.266219] iser: iser_data_buf_dump: sg[99] dma_addr:0x90660E000
page:0xffffea0024198380 off:0x0 sz:0x2000 dma_len:0x2000
[ 64.278836] iser: iser_data_buf_dump: sg[100] dma_addr:0x9017E6000
page:0xffffea002405f980 off:0x0 sz:0x1000 dma_len:0x1000
[ 64.291577] iser: iser_data_buf_dump: sg[101] dma_addr:0x93FBF1000
page:0xffffea0024fefc40 off:0x0 sz:0x200 dma_len:0x200
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160405094339.GA1622-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2016-04-06 17:34 ` sagig
[not found] ` <570548AB.4090802-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: sagig @ 2016-04-06 17:34 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
>> This implies that maybe something broke in ib_sg_to_pages:
>>
>> Can you run iser with this patch applied (FR or FMR):
>
> For FMR this won't trigger, but here is the FR run:
>
> [ 63.016616] iser: iser_fast_reg_mr: failed to map sg (101/102)
> [ 63.023183] iser: iser_data_buf_dump: sg[0] dma_addr:0x90E15D200
> page:0xffffea0024385740 off:0x200 sz:0xe00 dma_len:0xe00
> [ 63.035466] iser: iser_data_buf_dump: sg[1] dma_addr:0x90611D000
> page:0xffffea0024184740 off:0x0 sz:0x1000 dma_len:0x1000
> [ 63.047749] iser: iser_data_buf_dump: sg[2] dma_addr:0x90EDB3000
> page:0xffffea00243b6cc0 off:0x0 sz:0x1000 dma_len:0x1000
> [ 63.060050] iser: iser_data_buf_dump: sg[3] dma_addr:0x9062CB000
> page:0xffffea002418b2c0 off:0x0 sz:0x1000 dma_len:0x1000
...
> [ 64.215764] iser: iser_data_buf_dump: sg[95] dma_addr:0x90E440000
> page:0xffffea0024391000 off:0x0 sz:0x2000 dma_len:0x2000
> [ 64.228380] iser: iser_data_buf_dump: sg[96] dma_addr:0x90E43E000
> page:0xffffea0024390f80 off:0x0 sz:0x2000 dma_len:0x2000
> [ 64.240995] iser: iser_data_buf_dump: sg[97] dma_addr:0x901586000
> page:0xffffea0024056180 off:0x0 sz:0x2000 dma_len:0x2000
> [ 64.253611] iser: iser_data_buf_dump: sg[98] dma_addr:0x90F390000
> page:0xffffea00243ce400 off:0x0 sz:0x2000 dma_len:0x2000
> [ 64.266219] iser: iser_data_buf_dump: sg[99] dma_addr:0x90660E000
> page:0xffffea0024198380 off:0x0 sz:0x2000 dma_len:0x2000
> [ 64.278836] iser: iser_data_buf_dump: sg[100] dma_addr:0x9017E6000
> page:0xffffea002405f980 off:0x0 sz:0x1000 dma_len:0x1000
> [ 64.291577] iser: iser_data_buf_dump: sg[101] dma_addr:0x93FBF1000
> page:0xffffea0024fefc40 off:0x0 sz:0x200 dma_len:0x200
>
So this scatterlist seems perfectly fine (no gaps). The only thing I
can think of is that we don't have enough space reserved, but I counted
127 pages in the scatterlist when iser reserves 128 (by default).
Just in case I mis-counted, can you (or Max) try with the below
patch:
--
diff --git a/drivers/infiniband/ulp/iser/iser_initiator.c
b/drivers/infiniband/ulp/iser/iser_initiator.c
index ed54b388e7ad..f2ac891c432a 100644
--- a/drivers/infiniband/ulp/iser/iser_initiator.c
+++ b/drivers/infiniband/ulp/iser/iser_initiator.c
@@ -250,7 +250,7 @@ int iser_alloc_rx_descriptors(struct iser_conn
*iser_conn,
iser_conn->min_posted_rx = iser_conn->qp_max_recv_dtos >> 2;
if (device->reg_ops->alloc_reg_res(ib_conn, session->scsi_cmds_max,
- iser_conn->scsi_sg_tablesize))
+ iser_conn->scsi_sg_tablesize
+ 1))
goto create_rdma_reg_res_failed;
if (iser_alloc_login_buf(iser_conn))
diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c
b/drivers/infiniband/ulp/iser/iser_verbs.c
index 40c0f4978e2f..937b97f34be1 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -721,7 +721,7 @@ iser_calc_scsi_params(struct iser_conn *iser_conn,
sg_tablesize = DIV_ROUND_UP(max_sectors * 512, SIZE_4K);
sup_sg_tablesize = min_t(unsigned, ISCSI_ISER_MAX_SG_TABLESIZE,
-
device->ib_device->attrs.max_fast_reg_page_list_len);
+
device->ib_device->attrs.max_fast_reg_page_list_len - 1);
if (sg_tablesize > sup_sg_tablesize) {
sg_tablesize = sup_sg_tablesize;
--
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <570548AB.4090802-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
@ 2016-04-06 23:16 ` Christoph Hellwig
[not found] ` <20160406231658.GB11588-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-06 23:16 UTC (permalink / raw)
To: sagig; +Cc: Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Wed, Apr 06, 2016 at 08:34:35PM +0300, sagig wrote:
> So this scatterlist seems perfectly fine (no gaps). The only thing I
> can think of is that we don't have enough space reserved, but I counted
> 127 pages in the scatterlist when iser reserves 128 (by default).
>
> Just in case I mis-counted, can you (or Max) try with the below
> patch:
This works fine for me, but I wonder if the miscounting is in the
block layer?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160406231658.GB11588-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2016-04-07 7:14 ` Sagi Grimberg
[not found] ` <570608C9.1070805-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Sagi Grimberg @ 2016-04-07 7:14 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
>> So this scatterlist seems perfectly fine (no gaps). The only thing I
>> can think of is that we don't have enough space reserved, but I counted
>> 127 pages in the scatterlist when iser reserves 128 (by default).
>>
>> Just in case I mis-counted, can you (or Max) try with the below
>> patch:
>
> This works fine for me, but I wonder if the miscounting is in the
> block layer?
We communicate the "max_pages" via sg_tablesize and max_hw_sectors.
iSER communicates (or at least supposed to) sg_tablesize=128 and
max_sectors=1024 to scsi (which sets the block queues attributes), that
should be sufficient for handling all the incoming IO. Unless we didn't
account for the first-page offset and the reminder coming at the last
segment (which results in an extra page). The patch should fix the
problem, however in this particular sg list I don't see the violation
(counted 127 pages)...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <570608C9.1070805-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
@ 2016-04-07 9:09 ` Max Gurtovoy
[not found] ` <570623B9.8040108-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Max Gurtovoy @ 2016-04-07 9:09 UTC (permalink / raw)
To: Sagi Grimberg, Christoph Hellwig; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
On 4/7/2016 10:14 AM, Sagi Grimberg wrote:
>
>>> So this scatterlist seems perfectly fine (no gaps). The only thing I
>>> can think of is that we don't have enough space reserved, but I counted
>>> 127 pages in the scatterlist when iser reserves 128 (by default).
>>>
>>> Just in case I mis-counted, can you (or Max) try with the below
>>> patch:
>>
>> This works fine for me, but I wonder if the miscounting is in the
>> block layer?
>
> We communicate the "max_pages" via sg_tablesize and max_hw_sectors.
>
> iSER communicates (or at least supposed to) sg_tablesize=128 and
> max_sectors=1024 to scsi (which sets the block queues attributes), that
> should be sufficient for handling all the incoming IO. Unless we didn't
> account for the first-page offset and the reminder coming at the last
> segment (which results in an extra page). The patch should fix the
> problem, however in this particular sg list I don't see the violation
> (counted 127 pages)...
I counted 129 pages.
I will run some tests in our labs too.
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <570623B9.8040108-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2016-04-07 15:16 ` Sagi Grimberg
[not found] ` <570679B6.10409-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
0 siblings, 1 reply; 20+ messages in thread
From: Sagi Grimberg @ 2016-04-07 15:16 UTC (permalink / raw)
To: Max Gurtovoy, Christoph Hellwig; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
>>
>> We communicate the "max_pages" via sg_tablesize and max_hw_sectors.
>>
>> iSER communicates (or at least supposed to) sg_tablesize=128 and
>> max_sectors=1024 to scsi (which sets the block queues attributes), that
>> should be sufficient for handling all the incoming IO. Unless we didn't
>> account for the first-page offset and the reminder coming at the last
>> segment (which results in an extra page). The patch should fix the
>> problem, however in this particular sg list I don't see the violation
>> (counted 127 pages)...
>
> I counted 129 pages.
> I will run some tests in our labs too.
If you did then that indicates the block layer violates
its obligation to respect max_segments/max_hw_sectors?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <570679B6.10409-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
@ 2016-04-07 15:20 ` Christoph Hellwig
2016-04-07 22:13 ` Christoph Hellwig
1 sibling, 0 replies; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-07 15:20 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Max Gurtovoy, Christoph Hellwig, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Thu, Apr 07, 2016 at 06:16:06PM +0300, Sagi Grimberg wrote:
> >>segment (which results in an extra page). The patch should fix the
> >>problem, however in this particular sg list I don't see the violation
> >>(counted 127 pages)...
> >
> >I counted 129 pages.
> >I will run some tests in our labs too.
>
> If you did then that indicates the block layer violates
> its obligation to respect max_segments/max_hw_sectors?
I spent some time looking over the code and remembered an issue I
fixed for NVMf (which might explain why we're not seeing it there).
max_segments just means contiguous segments, not nessecarily 4k
pages. For that we either need a small enough max_sectors_hw or set
a max_segment_size of 4k, otherwise we might get 128 block segments,
but due to lacking alignment they'll expand to 129 pages.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <570679B6.10409-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-07 15:20 ` Christoph Hellwig
@ 2016-04-07 22:13 ` Christoph Hellwig
[not found] ` <20160407221359.GA16686-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
1 sibling, 1 reply; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-07 22:13 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Max Gurtovoy, bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
The patch below fixes the issue for me. But for SRP I'm still seeing
lots of these failures even with the equivalent patch applied:
[ 426.027905] scsi host8: ib_srp: Failed to map data (-12)
[ 426.053203] scsi host8: ib_srp: Failed to map data (-12)
[ 426.053265] scsi host8: ib_srp: Failed to map data (-12)
[ 426.053270] scsi host8: ib_srp: Failed to map data (-12)
[ 426.053273] scsi host8: ib_srp: Failed to map data (-12)
[ 426.053276] scsi host8: ib_srp: Failed to map data (-12)
[ 426.053279] scsi host8: ib_srp: Failed to map data (-12)
It seems the actual reproducer is xfs_repair as run by xfstests.
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 60b169a..3ac4755 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -3205,6 +3205,7 @@ static ssize_t srp_create_target(struct device *dev,
target_host->max_id = 1;
target_host->max_lun = -1LL;
target_host->max_cmd_len = sizeof ((struct srp_cmd *) (void *) 0L)->cdb;
+ target_host->max_segment_size = PAGE_SIZE;
target = host_to_target(target_host);
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 8c6e318..ae331e0 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2120,7 +2120,8 @@ static void __scsi_init_queue(struct Scsi_Host *shost, struct request_queue *q)
blk_queue_segment_boundary(q, shost->dma_boundary);
dma_set_seg_boundary(dev, shost->dma_boundary);
- blk_queue_max_segment_size(q, dma_get_max_seg_size(dev));
+ blk_queue_max_segment_size(q,
+ min(shost->max_segment_size, dma_get_max_seg_size(dev)));
if (!shost->use_clustering)
q->limits.cluster = 0;
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index fcfa3d7..f11d3fe 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -621,6 +621,7 @@ struct Scsi_Host {
short unsigned int sg_tablesize;
short unsigned int sg_prot_tablesize;
unsigned int max_sectors;
+ unsigned int max_segment_size;
unsigned long dma_boundary;
/*
* In scsi-mq mode, the number of hardware queues supported by the LLD.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160407221359.GA16686-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2016-04-08 1:42 ` Bart Van Assche
2016-04-10 13:30 ` Max Gurtovoy
2016-04-12 21:33 ` Bart Van Assche
2 siblings, 0 replies; 20+ messages in thread
From: Bart Van Assche @ 2016-04-08 1:42 UTC (permalink / raw)
To: Christoph Hellwig, Sagi Grimberg
Cc: Max Gurtovoy, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On 04/07/16 15:14, Christoph Hellwig wrote:
> The patch below fixes the issue for me. But for SRP I'm still seeing
> lots of these failures even with the equivalent patch applied:
>
> [ 426.027905] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053203] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053265] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053270] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053273] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053276] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053279] scsi host8: ib_srp: Failed to map data (-12)
>
> It seems the actual reproducer is xfs_repair as run by xfstests.
Hello Christoph,
Thanks for the report. I will try to reproduce this behavior and will
see whether I can find the root cause.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160407221359.GA16686-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-08 1:42 ` Bart Van Assche
@ 2016-04-10 13:30 ` Max Gurtovoy
[not found] ` <570A5563.5060104-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2016-04-12 21:33 ` Bart Van Assche
2 siblings, 1 reply; 20+ messages in thread
From: Max Gurtovoy @ 2016-04-10 13:30 UTC (permalink / raw)
To: Christoph Hellwig, Sagi Grimberg
Cc: bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ, linux-rdma-u79uwXL29TY76Z2rM5mHXA
On 4/8/2016 1:13 AM, Christoph Hellwig wrote:
> The patch below fixes the issue for me. But for SRP I'm still seeing
> lots of these failures even with the equivalent patch applied:
>
> [ 426.027905] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053203] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053265] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053270] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053273] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053276] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053279] scsi host8: ib_srp: Failed to map data (-12)
>
> It seems the actual reproducer is xfs_repair as run by xfstests.
>
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index 60b169a..3ac4755 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -3205,6 +3205,7 @@ static ssize_t srp_create_target(struct device *dev,
> target_host->max_id = 1;
> target_host->max_lun = -1LL;
> target_host->max_cmd_len = sizeof ((struct srp_cmd *) (void *) 0L)->cdb;
> + target_host->max_segment_size = PAGE_SIZE;
>
> target = host_to_target(target_host);
Hi Christoph,
did you add similar assignment in iser code or only used the scsi patches ?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <570A5563.5060104-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
@ 2016-04-10 14:52 ` Christoph Hellwig
0 siblings, 0 replies; 20+ messages in thread
From: Christoph Hellwig @ 2016-04-10 14:52 UTC (permalink / raw)
To: Max Gurtovoy
Cc: Christoph Hellwig, Sagi Grimberg,
bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Sun, Apr 10, 2016 at 04:30:11PM +0300, Max Gurtovoy wrote:
> Hi Christoph,
> did you add similar assignment in iser code or only used the scsi patches ?
Yes, to fix it I had to do the assignment in iser. Not sure why
I didn't manage to send it out. Let me prepare a proper patch series.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: iSER initiator in 4.5 is unhappy..
[not found] ` <20160407221359.GA16686-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-08 1:42 ` Bart Van Assche
2016-04-10 13:30 ` Max Gurtovoy
@ 2016-04-12 21:33 ` Bart Van Assche
2 siblings, 0 replies; 20+ messages in thread
From: Bart Van Assche @ 2016-04-12 21:33 UTC (permalink / raw)
To: Christoph Hellwig, Sagi Grimberg
Cc: Max Gurtovoy, bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
On 04/07/2016 03:13 PM, Christoph Hellwig wrote:
> The patch below fixes the issue for me. But for SRP I'm still seeing
> lots of these failures even with the equivalent patch applied:
>
> [ 426.027905] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053203] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053265] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053270] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053273] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053276] scsi host8: ib_srp: Failed to map data (-12)
> [ 426.053279] scsi host8: ib_srp: Failed to map data (-12)
>
> It seems the actual reproducer is xfs_repair as run by xfstests.
Hello Christoph,
So far I have not yet been able to reproduce the ib_srp mapping failure
with kernel v4.6-rc3. How was ib_srp configured in your test (cat
/etc/modprobe.d/ib_srp.conf)?
Thanks,
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2016-04-12 21:33 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-04 13:46 iSER initiator in 4.5 is unhappy Christoph Hellwig
[not found] ` <20160404134656.GA30269-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-04 14:42 ` Max Gurtovoy
[not found] ` <57027D48.4000500-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2016-04-04 15:10 ` Christoph Hellwig
2016-04-04 17:29 ` sagig
[not found] ` <5702A46F.5070109-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-04 17:38 ` sagig
[not found] ` <5702A6AC.9020008-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-04 22:58 ` Christoph Hellwig
[not found] ` <20160404225831.GA6725-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-05 8:55 ` sagig
[not found] ` <57037D8E.4030506-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-05 9:43 ` Christoph Hellwig
[not found] ` <20160405094339.GA1622-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-06 17:34 ` sagig
[not found] ` <570548AB.4090802-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-06 23:16 ` Christoph Hellwig
[not found] ` <20160406231658.GB11588-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-07 7:14 ` Sagi Grimberg
[not found] ` <570608C9.1070805-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-07 9:09 ` Max Gurtovoy
[not found] ` <570623B9.8040108-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2016-04-07 15:16 ` Sagi Grimberg
[not found] ` <570679B6.10409-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2016-04-07 15:20 ` Christoph Hellwig
2016-04-07 22:13 ` Christoph Hellwig
[not found] ` <20160407221359.GA16686-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2016-04-08 1:42 ` Bart Van Assche
2016-04-10 13:30 ` Max Gurtovoy
[not found] ` <570A5563.5060104-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2016-04-10 14:52 ` Christoph Hellwig
2016-04-12 21:33 ` Bart Van Assche
2016-04-04 22:46 ` Christoph Hellwig
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.