* Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
@ 2023-11-25 18:28 Lee Duncan
2023-11-26 12:01 ` Hannes Reinecke
2023-12-06 22:50 ` Lee Duncan
0 siblings, 2 replies; 8+ messages in thread
From: Lee Duncan @ 2023-11-25 18:28 UTC (permalink / raw)
To: target-devel; +Cc: Martin K. Petersen, linux-scsi, Chris Leech, Mike Christie
Hi Martin/list:
I am trying to track down an issue I am seeing when trying to boot using
iSCSI root using the fastlinq qedi converged network adapter initiator
and an LIO target.
In this configuration, but using a non-LIO (hardware) iSCSI target,
everything "just works". But when I switch to using an LIO software
target, I get this error when booting:
> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set Immediate Bit in iSCSI Initiator Scsi Command PDU.
> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
OpCode: 0x52
It looks like the fastlinq adapter is sending a SCSI write (0x12) with
the Immediate bit set (0x40).
The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
> " Scsi Command PDU.\n");
> return iscsit_add_reject_cmd(cmd,
> ISCSI_REASON_BOOKMARK_INVALID, buf);
> }
>
But I looked at rfc3720, and it looks like SCSI PDUs can have the
Immediate Bit set:
> 10.3. SCSI Command
>
> The format of the SCSI Command PDU is:
>
> Byte/ 0 | 1 | 2 | 3 |
> / | | | |
> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
> +---------------+---------------+---------------+---------------+
> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
> +---------------+---------------+---------------+---------------+
> 4|TotalAHSLength | DataSegmentLength |
> +---------------+---------------+---------------+---------------+
> 8| Logical Unit Number (LUN) |
> + +
> 12| |
> +---------------+---------------+---------------+---------------+
> ...
Where "I" is the immediate bit.
Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if
I'm mistaken here, but it looks like LIO does not allow the Immediate
bit (and hence Immediate data), even though the SPEC does allow it. But
it also looks like, during negotiation phase, LIO negotiates
ImmediateData like any other parameter, allowing "YES".
In our testing, if we set ImmediateData=No in the target, then our
problem goes away, i.e. we can now boot from an LIO target. This is
because Immediate Data is negotiated off, of course.
Is this a bug, or is this adapter doing something wrong? I would
appreciate other viewpoints.
Thank you.
--
Lee Duncan
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-25 18:28 Possible Bug? LIO does not support SCSI commands with Immediate Bit Set? Lee Duncan
@ 2023-11-26 12:01 ` Hannes Reinecke
2023-11-27 0:39 ` Mike Christie
2023-12-06 22:50 ` Lee Duncan
1 sibling, 1 reply; 8+ messages in thread
From: Hannes Reinecke @ 2023-11-26 12:01 UTC (permalink / raw)
To: Lee Duncan, target-devel
Cc: Martin K. Petersen, linux-scsi, Chris Leech, Mike Christie
On 11/25/23 19:28, Lee Duncan wrote:
> Hi Martin/list:
>
> I am trying to track down an issue I am seeing when trying to boot using
> iSCSI root using the fastlinq qedi converged network adapter initiator
> and an LIO target.
>
> In this configuration, but using a non-LIO (hardware) iSCSI target,
> everything "just works". But when I switch to using an LIO software
> target, I get this error when booting:
>
>> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set
>> Immediate Bit in iSCSI Initiator Scsi Command PDU.
>> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
> OpCode: 0x52
>
> It looks like the fastlinq adapter is sending a SCSI write (0x12) with
> the Immediate bit set (0x40).
>
> The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
>
>> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>> " Scsi Command PDU.\n");
>> return iscsit_add_reject_cmd(cmd,
>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>> }
>>
>
> But I looked at rfc3720, and it looks like SCSI PDUs can have the
> Immediate Bit set:
>
>> 10.3. SCSI Command
>> The format of the SCSI Command PDU is:
>> Byte/ 0 | 1 | 2 | 3 |
>> / | | | |
>> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
>> +---------------+---------------+---------------+---------------+
>> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
>> +---------------+---------------+---------------+---------------+
>> 4|TotalAHSLength | DataSegmentLength |
>> +---------------+---------------+---------------+---------------+
>> 8| Logical Unit Number (LUN) |
>> + +
>> 12| |
>> +---------------+---------------+---------------+---------------+
>> ...
>
> Where "I" is the immediate bit.
>
> Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if
> I'm mistaken here, but it looks like LIO does not allow the Immediate
> bit (and hence Immediate data), even though the SPEC does allow it. But
> it also looks like, during negotiation phase, LIO negotiates
> ImmediateData like any other parameter, allowing "YES".
>
> In our testing, if we set ImmediateData=No in the target, then our
> problem goes away, i.e. we can now boot from an LIO target. This is
> because Immediate Data is negotiated off, of course.
>
> Is this a bug, or is this adapter doing something wrong? I would
> appreciate other viewpoints.
>
> Thank you.
Sounds like a bug.
Can you check if this helps?
diff --git a/drivers/target/iscsi/iscsi_target.c
b/drivers/target/iscsi/iscsi_target.c
index b516c2893420..ad68706ad9f7 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -1060,7 +1060,8 @@ int iscsit_setup_scsi_cmd(struct iscsit_conn
*conn, struct iscsit_cmd *cmd,
ISCSI_REASON_BOOKMARK_INVALID, buf);
}
- if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
+ if ((hdr->opcode & ISCSI_OP_IMMEDIATE) &&
+ !conn->sess->sess_ops->ImmediateData) {
pr_err("Illegally set Immediate Bit in iSCSI Initiator"
" Scsi Command PDU.\n");
return iscsit_add_reject_cmd(cmd,
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), GF: Ivo Totev, Andrew McDonald,
Werner Knoblich
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-26 12:01 ` Hannes Reinecke
@ 2023-11-27 0:39 ` Mike Christie
2023-11-27 19:01 ` Laurence Oberman
2023-11-27 20:23 ` Lee Duncan
0 siblings, 2 replies; 8+ messages in thread
From: Mike Christie @ 2023-11-27 0:39 UTC (permalink / raw)
To: Hannes Reinecke, Lee Duncan, target-devel
Cc: Martin K. Petersen, linux-scsi, Chris Leech
On 11/26/23 6:01 AM, Hannes Reinecke wrote:
> On 11/25/23 19:28, Lee Duncan wrote:
>> Hi Martin/list:
>>
>> I am trying to track down an issue I am seeing when trying to boot using iSCSI root using the fastlinq qedi converged network adapter initiator and an LIO target.
>>
>> In this configuration, but using a non-LIO (hardware) iSCSI target, everything "just works". But when I switch to using an LIO software target, I get this error when booting:
>>
>>> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set Immediate Bit in iSCSI Initiator Scsi Command PDU.
>>> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
>> OpCode: 0x52
>>
>> It looks like the fastlinq adapter is sending a SCSI write (0x12) with the Immediate bit set (0x40).
>>
>> The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
>>
>>> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>>> " Scsi Command PDU.\n");
>>> return iscsit_add_reject_cmd(cmd,
>>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>>> }
>>>
>>
>> But I looked at rfc3720, and it looks like SCSI PDUs can have the Immediate Bit set:
>>
>>> 10.3. SCSI Command
>>> The format of the SCSI Command PDU is:
>>> Byte/ 0 | 1 | 2 | 3 |
>>> / | | | |
>>> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
>>> +---------------+---------------+---------------+---------------+
>>> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
>>> +---------------+---------------+---------------+---------------+
>>> 4|TotalAHSLength | DataSegmentLength |
>>> +---------------+---------------+---------------+---------------+
>>> 8| Logical Unit Number (LUN) |
>>> + +
>>> 12| |
>>> +---------------+---------------+---------------+---------------+
>>> ...
>>
>> Where "I" is the immediate bit.
>>
>> Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if I'm mistaken here, but it looks like LIO does not allow the Immediate bit (and hence Immediate data), even though the SPEC does allow it. But it also looks like, during negotiation phase, LIO negotiates ImmediateData like any other parameter, allowing "YES".
>>
>> In our testing, if we set ImmediateData=No in the target, then our problem goes away, i.e. we can now boot from an LIO target. This is because Immediate Data is negotiated off, of course.
>>
>> Is this a bug, or is this adapter doing something wrong? I would appreciate other viewpoints.
>>
>> Thank you.
>
> Sounds like a bug.
> Can you check if this helps?
>
> diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
> index b516c2893420..ad68706ad9f7 100644
> --- a/drivers/target/iscsi/iscsi_target.c
> +++ b/drivers/target/iscsi/iscsi_target.c
> @@ -1060,7 +1060,8 @@ int iscsit_setup_scsi_cmd(struct iscsit_conn *conn, struct iscsit_cmd *cmd,
>
> ISCSI_REASON_BOOKMARK_INVALID, buf);
> }
>
> - if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
> + if ((hdr->opcode & ISCSI_OP_IMMEDIATE) &&
> + !conn->sess->sess_ops->ImmediateData) {
> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
> " Scsi Command PDU.\n");
> return iscsit_add_reject_cmd(cmd,
>
These are different things.
Immediate data means you can have data after the header in the PDU.
The immediate bit on the header has CmdSN rules which allows you to
send commands without having to increment the CmdSN. It's used for
things like TMFs because at that time, the window might be closed due
to SCSI commands having filled it.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-27 0:39 ` Mike Christie
@ 2023-11-27 19:01 ` Laurence Oberman
2023-11-27 20:23 ` Lee Duncan
1 sibling, 0 replies; 8+ messages in thread
From: Laurence Oberman @ 2023-11-27 19:01 UTC (permalink / raw)
To: Mike Christie, Hannes Reinecke, Lee Duncan, target-devel
Cc: Martin K. Petersen, linux-scsi, Chris Leech
On Sun, 2023-11-26 at 18:39 -0600, Mike Christie wrote:
> On 11/26/23 6:01 AM, Hannes Reinecke wrote:
> > On 11/25/23 19:28, Lee Duncan wrote:
> > > Hi Martin/list:
> > >
> > > I am trying to track down an issue I am seeing when trying to
> > > boot using iSCSI root using the fastlinq qedi converged network
> > > adapter initiator and an LIO target.
> > >
> > > In this configuration, but using a non-LIO (hardware) iSCSI
> > > target, everything "just works". But when I switch to using an
> > > LIO software target, I get this error when booting:
> > >
> > > > 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set
> > > > Immediate Bit in iSCSI Initiator Scsi Command PDU.
> > > > ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got
> > > > unknown iSCSI
> > > OpCode: 0x52
> > >
> > > It looks like the fastlinq adapter is sending a SCSI write (0x12)
> > > with the Immediate bit set (0x40).
> > >
> > > The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like
> > > this:
> > >
> > > > if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
> > > > pr_err("Illegally set Immediate Bit in iSCSI Initiator"
> > > > " Scsi Command PDU.\n");
> > > > return iscsit_add_reject_cmd(cmd,
> > > > ISCSI_REASON_BOOKMARK_INVALID, buf);
> > > > }
> > > >
> > >
> > > But I looked at rfc3720, and it looks like SCSI PDUs can have the
> > > Immediate Bit set:
> > >
> > > > 10.3. SCSI Command
> > > > The format of the SCSI Command PDU is:
> > > > Byte/ 0 | 1 | 2 |
> > > > 3 |
> > > > / | |
> > > > | |
> > > > |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4
> > > > 5 6 7|
> > > > +---------------+---------------+---------------+---------
> > > > ------+
> > > > 0|.|I| 0x01 |F|R|W|. .|ATTR |
> > > > Reserved |
> > > > +---------------+---------------+---------------+---------
> > > > ------+
> > > > 4|TotalAHSLength |
> > > > DataSegmentLength |
> > > > +---------------+---------------+---------------+---------
> > > > ------+
> > > > 8| Logical Unit Number
> > > > (LUN) |
> > > >
> > > > +
> > > > +
> > > >
> > > > 12|
> > > > |
> > > > +---------------+---------------+---------------+---------
> > > > ------+
> > > > ...
> > >
> > > Where "I" is the immediate bit.
> > >
> > > Frankly, I'm never sure I read the SPEC correctly, so I'm not
> > > sure if I'm mistaken here, but it looks like LIO does not allow
> > > the Immediate bit (and hence Immediate data), even though the
> > > SPEC does allow it. But it also looks like, during negotiation
> > > phase, LIO negotiates ImmediateData like any other parameter,
> > > allowing "YES".
> > >
> > > In our testing, if we set ImmediateData=No in the target, then
> > > our problem goes away, i.e. we can now boot from an LIO target.
> > > This is because Immediate Data is negotiated off, of course.
> > >
> > > Is this a bug, or is this adapter doing something wrong? I would
> > > appreciate other viewpoints.
> > >
> > > Thank you.
> >
> > Sounds like a bug.
> > Can you check if this helps?
> >
> > diff --git a/drivers/target/iscsi/iscsi_target.c
> > b/drivers/target/iscsi/iscsi_target.c
> > index b516c2893420..ad68706ad9f7 100644
> > --- a/drivers/target/iscsi/iscsi_target.c
> > +++ b/drivers/target/iscsi/iscsi_target.c
> > @@ -1060,7 +1060,8 @@ int iscsit_setup_scsi_cmd(struct iscsit_conn
> > *conn, struct iscsit_cmd *cmd,
> >
> > ISCSI_REASON_BOOKMARK_INVALID, buf);
> > }
> >
> > - if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
> > + if ((hdr->opcode & ISCSI_OP_IMMEDIATE) &&
> > + !conn->sess->sess_ops->ImmediateData) {
> > pr_err("Illegally set Immediate Bit in iSCSI
> > Initiator"
> > " Scsi Command PDU.\n");
> > return iscsit_add_reject_cmd(cmd,
> >
>
>
> These are different things.
>
> Immediate data means you can have data after the header in the PDU.
>
> The immediate bit on the header has CmdSN rules which allows you to
> send commands without having to increment the CmdSN. It's used for
> things like TMFs because at that time, the window might be closed due
> to SCSI commands having filled it.
>
>
I have iscsi booted for years with LIO as the target but never with
qedi as the base ISCSI driver. I will see if I see the same behavior as
Lee with the qedi involved.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-27 0:39 ` Mike Christie
2023-11-27 19:01 ` Laurence Oberman
@ 2023-11-27 20:23 ` Lee Duncan
2023-11-27 20:44 ` Mike Christie
1 sibling, 1 reply; 8+ messages in thread
From: Lee Duncan @ 2023-11-27 20:23 UTC (permalink / raw)
To: Mike Christie, Hannes Reinecke, target-devel
Cc: Martin K. Petersen, linux-scsi, Chris Leech
On 11/26/23 16:39, Mike Christie wrote:
> On 11/26/23 6:01 AM, Hannes Reinecke wrote:
>> On 11/25/23 19:28, Lee Duncan wrote:
>>> Hi Martin/list:
>>>
>>> I am trying to track down an issue I am seeing when trying to boot using iSCSI root using the fastlinq qedi converged network adapter initiator and an LIO target.
>>>
>>> In this configuration, but using a non-LIO (hardware) iSCSI target, everything "just works". But when I switch to using an LIO software target, I get this error when booting:
>>>
>>>> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set Immediate Bit in iSCSI Initiator Scsi Command PDU.
>>>> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
>>> OpCode: 0x52
>>>
>>> It looks like the fastlinq adapter is sending a SCSI write (0x12) with the Immediate bit set (0x40).
>>>
>>> The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
>>>
>>>> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>>>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>>>> " Scsi Command PDU.\n");
>>>> return iscsit_add_reject_cmd(cmd,
>>>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>>>> }
>>>>
>>>
>>> But I looked at rfc3720, and it looks like SCSI PDUs can have the Immediate Bit set:
>>>
>>>> 10.3. SCSI Command
>>>> The format of the SCSI Command PDU is:
>>>> Byte/ 0 | 1 | 2 | 3 |
>>>> / | | | |
>>>> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
>>>> +---------------+---------------+---------------+---------------+
>>>> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
>>>> +---------------+---------------+---------------+---------------+
>>>> 4|TotalAHSLength | DataSegmentLength |
>>>> +---------------+---------------+---------------+---------------+
>>>> 8| Logical Unit Number (LUN) |
>>>> + +
>>>> 12| |
>>>> +---------------+---------------+---------------+---------------+
>>>> ...
>>>
>>> Where "I" is the immediate bit.
>>>
>>> Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if I'm mistaken here, but it looks like LIO does not allow the Immediate bit (and hence Immediate data), even though the SPEC does allow it. But it also looks like, during negotiation phase, LIO negotiates ImmediateData like any other parameter, allowing "YES".
>>>
>>> In our testing, if we set ImmediateData=No in the target, then our problem goes away, i.e. we can now boot from an LIO target. This is because Immediate Data is negotiated off, of course.
>>>
>>> Is this a bug, or is this adapter doing something wrong? I would appreciate other viewpoints.
>>>
>>> Thank you.
>>
>> Sounds like a bug.
>> Can you check if this helps?
>>
>> diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
>> index b516c2893420..ad68706ad9f7 100644
>> --- a/drivers/target/iscsi/iscsi_target.c
>> +++ b/drivers/target/iscsi/iscsi_target.c
>> @@ -1060,7 +1060,8 @@ int iscsit_setup_scsi_cmd(struct iscsit_conn *conn, struct iscsit_cmd *cmd,
>>
>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>> }
>>
>> - if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>> + if ((hdr->opcode & ISCSI_OP_IMMEDIATE) &&
>> + !conn->sess->sess_ops->ImmediateData) {
>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>> " Scsi Command PDU.\n");
>> return iscsit_add_reject_cmd(cmd,
>>
>
>
> These are different things.
>
> Immediate data means you can have data after the header in the PDU.
>
> The immediate bit on the header has CmdSN rules which allows you to
> send commands without having to increment the CmdSN. It's used for
> things like TMFs because at that time, the window might be closed due
> to SCSI commands having filled it.
>
Mike: it sounds like that implies that setting the immediate bit in the
header is actually not allowed for regular SCSI PDUs (like Read, Write),
correct?
--
Lee
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-27 20:23 ` Lee Duncan
@ 2023-11-27 20:44 ` Mike Christie
2023-11-27 22:08 ` Lee Duncan
0 siblings, 1 reply; 8+ messages in thread
From: Mike Christie @ 2023-11-27 20:44 UTC (permalink / raw)
To: Lee Duncan, Hannes Reinecke, target-devel
Cc: Martin K. Petersen, linux-scsi, Chris Leech
On 11/27/23 2:23 PM, Lee Duncan wrote:
> On 11/26/23 16:39, Mike Christie wrote:
>> On 11/26/23 6:01 AM, Hannes Reinecke wrote:
>>> On 11/25/23 19:28, Lee Duncan wrote:
>>>> Hi Martin/list:
>>>>
>>>> I am trying to track down an issue I am seeing when trying to boot using iSCSI root using the fastlinq qedi converged network adapter initiator and an LIO target.
>>>>
>>>> In this configuration, but using a non-LIO (hardware) iSCSI target, everything "just works". But when I switch to using an LIO software target, I get this error when booting:
>>>>
>>>>> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set Immediate Bit in iSCSI Initiator Scsi Command PDU.
>>>>> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
>>>> OpCode: 0x52
>>>>
>>>> It looks like the fastlinq adapter is sending a SCSI write (0x12) with the Immediate bit set (0x40).
>>>>
>>>> The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
>>>>
>>>>> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>>>>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>>>>> " Scsi Command PDU.\n");
>>>>> return iscsit_add_reject_cmd(cmd,
>>>>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>>>>> }
>>>>>
>>>>
>>>> But I looked at rfc3720, and it looks like SCSI PDUs can have the Immediate Bit set:
>>>>
>>>>> 10.3. SCSI Command
>>>>> The format of the SCSI Command PDU is:
>>>>> Byte/ 0 | 1 | 2 | 3 |
>>>>> / | | | |
>>>>> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
>>>>> +---------------+---------------+---------------+---------------+
>>>>> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
>>>>> +---------------+---------------+---------------+---------------+
>>>>> 4|TotalAHSLength | DataSegmentLength |
>>>>> +---------------+---------------+---------------+---------------+
>>>>> 8| Logical Unit Number (LUN) |
>>>>> + +
>>>>> 12| |
>>>>> +---------------+---------------+---------------+---------------+
>>>>> ...
>>>>
>>>> Where "I" is the immediate bit.
>>>>
>>>> Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if I'm mistaken here, but it looks like LIO does not allow the Immediate bit (and hence Immediate data), even though the SPEC does allow it. But it also looks like, during negotiation phase, LIO negotiates ImmediateData like any other parameter, allowing "YES".
>>>>
>>>> In our testing, if we set ImmediateData=No in the target, then our problem goes away, i.e. we can now boot from an LIO target. This is because Immediate Data is negotiated off, of course.
>>>>
>>>> Is this a bug, or is this adapter doing something wrong? I would appreciate other viewpoints.
>>>>
>>>> Thank you.
>>>
>>> Sounds like a bug.
>>> Can you check if this helps?
>>>
>>> diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
>>> index b516c2893420..ad68706ad9f7 100644
>>> --- a/drivers/target/iscsi/iscsi_target.c
>>> +++ b/drivers/target/iscsi/iscsi_target.c
>>> @@ -1060,7 +1060,8 @@ int iscsit_setup_scsi_cmd(struct iscsit_conn *conn, struct iscsit_cmd *cmd,
>>>
>>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>>> }
>>>
>>> - if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>>> + if ((hdr->opcode & ISCSI_OP_IMMEDIATE) &&
>>> + !conn->sess->sess_ops->ImmediateData) {
>>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>>> " Scsi Command PDU.\n");
>>> return iscsit_add_reject_cmd(cmd,
>>>
>>
>>
>> These are different things.
>>
>> Immediate data means you can have data after the header in the PDU.
>>
>> The immediate bit on the header has CmdSN rules which allows you to
>> send commands without having to increment the CmdSN. It's used for
>> things like TMFs because at that time, the window might be closed due
>> to SCSI commands having filled it.
>>
>
> Mike: it sounds like that implies that setting the immediate bit in the header is actually not allowed for regular SCSI PDUs (like Read, Write), correct?
I didn't mean to say that. I was just giving you a common use, so you
could check it out in the linux iscsi initiator code and spec and better
understand it if you were going to add support for it in the target code.
Scan for "immediate delivery" in the spec for more details but to
answer you question about support it says:
https://datatracker.ietf.org/doc/html/rfc7143
An iSCSI target MUST be able to handle at least one immediate task
management command and one immediate non-task-management iSCSI
command per connection at any time.
For software iscsi in linux, we only set use the bit for TMFs and Nops.
We don't use it for SCSI command PDUs, but from the text above it seems
like its supposed to be supported.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-27 20:44 ` Mike Christie
@ 2023-11-27 22:08 ` Lee Duncan
0 siblings, 0 replies; 8+ messages in thread
From: Lee Duncan @ 2023-11-27 22:08 UTC (permalink / raw)
To: Mike Christie, Hannes Reinecke, target-devel
Cc: Martin K. Petersen, linux-scsi, Chris Leech
On 11/27/23 12:44, Mike Christie wrote:
> On 11/27/23 2:23 PM, Lee Duncan wrote:
>> On 11/26/23 16:39, Mike Christie wrote:
>>> On 11/26/23 6:01 AM, Hannes Reinecke wrote:
>>>> On 11/25/23 19:28, Lee Duncan wrote:
>>>>> Hi Martin/list:
>>>>>
>>>>> I am trying to track down an issue I am seeing when trying to boot using iSCSI root using the fastlinq qedi converged network adapter initiator and an LIO target.
>>>>>
>>>>> In this configuration, but using a non-LIO (hardware) iSCSI target, everything "just works". But when I switch to using an LIO software target, I get this error when booting:
>>>>>
>>>>>> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set Immediate Bit in iSCSI Initiator Scsi Command PDU.
>>>>>> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
>>>>> OpCode: 0x52
>>>>>
>>>>> It looks like the fastlinq adapter is sending a SCSI write (0x12) with the Immediate bit set (0x40).
>>>>>
>>>>> The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
>>>>>
>>>>>> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>>>>>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>>>>>> " Scsi Command PDU.\n");
>>>>>> return iscsit_add_reject_cmd(cmd,
>>>>>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>>>>>> }
>>>>>>
>>>>>
>>>>> But I looked at rfc3720, and it looks like SCSI PDUs can have the Immediate Bit set:
>>>>>
>>>>>> 10.3. SCSI Command
>>>>>> The format of the SCSI Command PDU is:
>>>>>> Byte/ 0 | 1 | 2 | 3 |
>>>>>> / | | | |
>>>>>> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
>>>>>> +---------------+---------------+---------------+---------------+
>>>>>> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
>>>>>> +---------------+---------------+---------------+---------------+
>>>>>> 4|TotalAHSLength | DataSegmentLength |
>>>>>> +---------------+---------------+---------------+---------------+
>>>>>> 8| Logical Unit Number (LUN) |
>>>>>> + +
>>>>>> 12| |
>>>>>> +---------------+---------------+---------------+---------------+
>>>>>> ...
>>>>>
>>>>> Where "I" is the immediate bit.
>>>>>
>>>>> Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if I'm mistaken here, but it looks like LIO does not allow the Immediate bit (and hence Immediate data), even though the SPEC does allow it. But it also looks like, during negotiation phase, LIO negotiates ImmediateData like any other parameter, allowing "YES".
>>>>>
>>>>> In our testing, if we set ImmediateData=No in the target, then our problem goes away, i.e. we can now boot from an LIO target. This is because Immediate Data is negotiated off, of course.
>>>>>
>>>>> Is this a bug, or is this adapter doing something wrong? I would appreciate other viewpoints.
>>>>>
>>>>> Thank you.
>>>>
>>>> Sounds like a bug.
>>>> Can you check if this helps?
>>>>
>>>> diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
>>>> index b516c2893420..ad68706ad9f7 100644
>>>> --- a/drivers/target/iscsi/iscsi_target.c
>>>> +++ b/drivers/target/iscsi/iscsi_target.c
>>>> @@ -1060,7 +1060,8 @@ int iscsit_setup_scsi_cmd(struct iscsit_conn *conn, struct iscsit_cmd *cmd,
>>>>
>>>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>>>> }
>>>>
>>>> - if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>>>> + if ((hdr->opcode & ISCSI_OP_IMMEDIATE) &&
>>>> + !conn->sess->sess_ops->ImmediateData) {
>>>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>>>> " Scsi Command PDU.\n");
>>>> return iscsit_add_reject_cmd(cmd,
>>>>
>>>
>>>
>>> These are different things.
>>>
>>> Immediate data means you can have data after the header in the PDU.
>>>
>>> The immediate bit on the header has CmdSN rules which allows you to
>>> send commands without having to increment the CmdSN. It's used for
>>> things like TMFs because at that time, the window might be closed due
>>> to SCSI commands having filled it.
>>>
>>
>> Mike: it sounds like that implies that setting the immediate bit in the header is actually not allowed for regular SCSI PDUs (like Read, Write), correct?
>
> I didn't mean to say that. I was just giving you a common use, so you
> could check it out in the linux iscsi initiator code and spec and better
> understand it if you were going to add support for it in the target code.
>
> Scan for "immediate delivery" in the spec for more details but to
> answer you question about support it says:
>
> https://datatracker.ietf.org/doc/html/rfc7143
>
> An iSCSI target MUST be able to handle at least one immediate task
> management command and one immediate non-task-management iSCSI
> command per connection at any time.
>
> For software iscsi in linux, we only set use the bit for TMFs and Nops.
> We don't use it for SCSI command PDUs, but from the text above it seems
> like its supposed to be supported.
I have scanned rfc3720 quite a bit (not enough yet, though), but I
hadn't looked at the other RFCs yet. Thanks for the reference. And even
in rfc3720 it looked like it was supposed to be allowed.
Thanks Mike.
--
Lee
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Possible Bug? LIO does not support SCSI commands with Immediate Bit Set?
2023-11-25 18:28 Possible Bug? LIO does not support SCSI commands with Immediate Bit Set? Lee Duncan
2023-11-26 12:01 ` Hannes Reinecke
@ 2023-12-06 22:50 ` Lee Duncan
1 sibling, 0 replies; 8+ messages in thread
From: Lee Duncan @ 2023-12-06 22:50 UTC (permalink / raw)
To: target-devel; +Cc: Martin K. Petersen, linux-scsi, Chris Leech, Mike Christie
On 11/25/23 10:28, Lee Duncan wrote:
> Hi Martin/list:
>
> I am trying to track down an issue I am seeing when trying to boot using
> iSCSI root using the fastlinq qedi converged network adapter initiator
> and an LIO target.
>
> In this configuration, but using a non-LIO (hardware) iSCSI target,
> everything "just works". But when I switch to using an LIO software
> target, I get this error when booting:
>
>> 2023-09-20T14:10:32.835358-0500 worker5 kernel: Illegally set
>> Immediate Bit in iSCSI Initiator Scsi Command PDU.
>> ... > 2023-09-20T14:10:32.835422-0500 worker5 kernel: Got unknown iSCSI
> OpCode: 0x52
>
> It looks like the fastlinq adapter is sending a SCSI write (0x12) with
> the Immediate bit set (0x40).
>
> The code in iscsi_target.c:iscsit_setup_scsi_cmd(): looks like this:
>
>> if (hdr->opcode & ISCSI_OP_IMMEDIATE) {
>> pr_err("Illegally set Immediate Bit in iSCSI Initiator"
>> " Scsi Command PDU.\n");
>> return iscsit_add_reject_cmd(cmd,
>> ISCSI_REASON_BOOKMARK_INVALID, buf);
>> }
>>
>
> But I looked at rfc3720, and it looks like SCSI PDUs can have the
> Immediate Bit set:
>
>> 10.3. SCSI Command
>> The format of the SCSI Command PDU is:
>> Byte/ 0 | 1 | 2 | 3 |
>> / | | | |
>> |0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|0 1 2 3 4 5 6 7|
>> +---------------+---------------+---------------+---------------+
>> 0|.|I| 0x01 |F|R|W|. .|ATTR | Reserved |
>> +---------------+---------------+---------------+---------------+
>> 4|TotalAHSLength | DataSegmentLength |
>> +---------------+---------------+---------------+---------------+
>> 8| Logical Unit Number (LUN) |
>> + +
>> 12| |
>> +---------------+---------------+---------------+---------------+
>> ...
>
> Where "I" is the immediate bit.
>
> Frankly, I'm never sure I read the SPEC correctly, so I'm not sure if
> I'm mistaken here, but it looks like LIO does not allow the Immediate
> bit (and hence Immediate data), even though the SPEC does allow it. But
> it also looks like, during negotiation phase, LIO negotiates
> ImmediateData like any other parameter, allowing "YES".
>
> In our testing, if we set ImmediateData=No in the target, then our
> problem goes away, i.e. we can now boot from an LIO target. This is
> because Immediate Data is negotiated off, of course.
>
> Is this a bug, or is this adapter doing something wrong? I would
> appreciate other viewpoints.
>
> Thank you.
Note: I have sent a set of two patches for this, but evidently my mailer
is considered SPAM now? I'm working on getting it figured out.
--
Lee Duncan
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-12-06 22:50 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-25 18:28 Possible Bug? LIO does not support SCSI commands with Immediate Bit Set? Lee Duncan
2023-11-26 12:01 ` Hannes Reinecke
2023-11-27 0:39 ` Mike Christie
2023-11-27 19:01 ` Laurence Oberman
2023-11-27 20:23 ` Lee Duncan
2023-11-27 20:44 ` Mike Christie
2023-11-27 22:08 ` Lee Duncan
2023-12-06 22:50 ` Lee Duncan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).