Multi-Actuator SAS HDD First Look

Multi-Actuator SAS HDD First Look

[Tim Walker]

Seagate announced their split actuator SAS drive, which will probably
require some kernel changes for full support. It's targeted at cloud
provider JBODs and RAID.

Here are some of the drive's architectural points. Since the two LUNs
share many common components (e.g. spindle) Seagate allocated some
SCSI operations to be LUN specific and some to affect the entire
device, that is, both LUNs.

1. Two LUNs, 0 & 1, each with independent lba space, and each
connected to an independent read channel, actuator, and set of heads.
2. Each actuator addresses 1/2 of the media - no media is shared
across the actuators. They seek independently.
3. One World Wide Name (WWN) is assigned to the port for device
address. Each Logical Unit has a separate World Wide Name for
identification in VPD page.
4. 128 deep command queue, shared across both LUNs
5. Each LUN can pull commands from the queue independently, so they
can implement their own sorting and optimization.
6. Ordered tag attribute causes the command to be ordered across both
Logical Units
7. Head of Queue attribute causes the command to be ordered with
respect to a single Logical Unit
8. Mode pages are device-based (shared across both Logical Units)
9. Log pages are device-based.
10. Inquiry VPD pages (with minor exceptions) are device based.
11. Device health features (SMART, etc) are device based

Seagate wants the multi-actuator design to integrate into the stack as
painlessly as possible.The interface design is still in the early
stages, so I am gathering requirements and recommendations, and also
providing any information necessary to help scope integrating a
multi-LUN device into the MQ stack. So, I am soliciting any pertinent
feedback including:

1. Painful incompatibilities between the Seagate proposal and current
MQ architecture
2. Linux changes needed
3. Drive interface changes needed
4. Anything else I may have overlooked

Please feel free to send any questions or comments.

Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Hannes Reinecke]

On Fri, 23 Mar 2018 08:57:12 -0600
Tim Walker <tim.t.walker@seagate.com> wrote:

> Seagate announced their split actuator SAS drive, which will probably
> require some kernel changes for full support. It's targeted at cloud
> provider JBODs and RAID.
> 
> Here are some of the drive's architectural points. Since the two LUNs
> share many common components (e.g. spindle) Seagate allocated some
> SCSI operations to be LUN specific and some to affect the entire
> device, that is, both LUNs.
> 
> 1. Two LUNs, 0 & 1, each with independent lba space, and each
> connected to an independent read channel, actuator, and set of heads.
> 2. Each actuator addresses 1/2 of the media - no media is shared
> across the actuators. They seek independently.
> 3. One World Wide Name (WWN) is assigned to the port for device
> address. Each Logical Unit has a separate World Wide Name for
> identification in VPD page.
> 4. 128 deep command queue, shared across both LUNs
> 5. Each LUN can pull commands from the queue independently, so they
> can implement their own sorting and optimization.
> 6. Ordered tag attribute causes the command to be ordered across both
> Logical Units
> 7. Head of Queue attribute causes the command to be ordered with
> respect to a single Logical Unit
> 8. Mode pages are device-based (shared across both Logical Units)
> 9. Log pages are device-based.
> 10. Inquiry VPD pages (with minor exceptions) are device based.
> 11. Device health features (SMART, etc) are device based
> 
> Seagate wants the multi-actuator design to integrate into the stack as
> painlessly as possible.The interface design is still in the early
> stages, so I am gathering requirements and recommendations, and also
> providing any information necessary to help scope integrating a
> multi-LUN device into the MQ stack. So, I am soliciting any pertinent
> feedback including:
> 
> 1. Painful incompatibilities between the Seagate proposal and current
> MQ architecture
> 2. Linux changes needed
> 3. Drive interface changes needed
> 4. Anything else I may have overlooked
> 
So far it looks okay; just make sure to have VPD page 0x83
entries properly associated.
To all intents and purposes these devices seem to look like 'normal'
devices with two LUNs; nothing special with that.
Real question would be in which areas those devices differentiate from
the two indepdendent LUN scenario.

There might be issues with per-device informations like SMART etc;
ideally they are available from _both_ LUNs.
Otherwise they'll show up as blank from one LUN, causing consternation
with any management software.

Cheers,

Hannes

Re: Multi-Actuator SAS HDD First Look

[Douglas Gilbert]

On 2018-03-26 11:08 AM, Hannes Reinecke wrote:
> On Fri, 23 Mar 2018 08:57:12 -0600
> Tim Walker <tim.t.walker@seagate.com> wrote:
> 
>> Seagate announced their split actuator SAS drive, which will probably
>> require some kernel changes for full support. It's targeted at cloud
>> provider JBODs and RAID.
>>
>> Here are some of the drive's architectural points. Since the two LUNs
>> share many common components (e.g. spindle) Seagate allocated some
>> SCSI operations to be LUN specific and some to affect the entire
>> device, that is, both LUNs.
>>
>> 1. Two LUNs, 0 & 1, each with independent lba space, and each
>> connected to an independent read channel, actuator, and set of heads.
>> 2. Each actuator addresses 1/2 of the media - no media is shared
>> across the actuators. They seek independently.
>> 3. One World Wide Name (WWN) is assigned to the port for device
>> address. Each Logical Unit has a separate World Wide Name for
>> identification in VPD page.
>> 4. 128 deep command queue, shared across both LUNs
>> 5. Each LUN can pull commands from the queue independently, so they
>> can implement their own sorting and optimization.
>> 6. Ordered tag attribute causes the command to be ordered across both
>> Logical Units
>> 7. Head of Queue attribute causes the command to be ordered with
>> respect to a single Logical Unit
>> 8. Mode pages are device-based (shared across both Logical Units)
>> 9. Log pages are device-based.
>> 10. Inquiry VPD pages (with minor exceptions) are device based.
>> 11. Device health features (SMART, etc) are device based
>>
>> Seagate wants the multi-actuator design to integrate into the stack as
>> painlessly as possible.The interface design is still in the early
>> stages, so I am gathering requirements and recommendations, and also
>> providing any information necessary to help scope integrating a
>> multi-LUN device into the MQ stack. So, I am soliciting any pertinent
>> feedback including:
>>
>> 1. Painful incompatibilities between the Seagate proposal and current
>> MQ architecture
>> 2. Linux changes needed
>> 3. Drive interface changes needed
>> 4. Anything else I may have overlooked
>>
> So far it looks okay; just make sure to have VPD page 0x83
> entries properly associated.
> To all intents and purposes these devices seem to look like 'normal'
> devices with two LUNs; nothing special with that.
> Real question would be in which areas those devices differentiate from
> the two indepdendent LUN scenario.
> 
> There might be issues with per-device informations like SMART etc;
> ideally they are available from _both_ LUNs.
> Otherwise they'll show up as blank from one LUN, causing consternation
> with any management software.

Further to this point, some types of damage, such as to a head
or (one side of) a platter would degrade one LU, possibly making
it unusable for storage, while the other side (and the other LU)
would be fine.

I'm curious how you plan to implement the START STOP UNIT command.
If one side of the platter is in "start" state and the other side
in "stop" state, will the heads on the stopped side be parked (if
they can be parked)? And if both sides (LUs) are stopped I would
hope you really would spin down the disk, then if either is started
the disk would be spun up.

Getting T10 to add a bit to the Block Device Characteristics VPD page
might be helpful. It could be a "shares a spindle" bit with the other
LUs identified in the SCSI Ports VPD page. Such an indication would
help an enclosure find out if a Multi-Actuator disk was really spun down
and ready to be removed or replaced. I think SES and smartmontools may
need tweaks to handle this new device model sensibly.

Doug Gilbert

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

Hi Doug-

Currently, the dual actuator firmware safely spins the drive down if
either LUN receives the START STOP UNIT command.  In other words, if
LUN1 receives the command, it will flush any dirty data from LUN1l and
LUN0, then spin down, taking both LUN1 & LUN0 off line. Alternatively,
we've had input that either:
a) Both LUNs must receive the START STOP UNIT command before the drive
will spin down, OR
b) Move the storage to LUN1 & LUN2, keeping LUN0 (with no storage) for
device specific commands such as START STOP UNIT that do not directly
access the media.

Thanks for the question.

Best regards,
-Tim

On Thu, Mar 29, 2018 at 12:03 PM, Douglas Gilbert <dgilbert@interlog.com> wrote:
> On 2018-03-26 11:08 AM, Hannes Reinecke wrote:
>>
>> On Fri, 23 Mar 2018 08:57:12 -0600
>> Tim Walker <tim.t.walker@seagate.com> wrote:
>>
>>> Seagate announced their split actuator SAS drive, which will probably
>>> require some kernel changes for full support. It's targeted at cloud
>>> provider JBODs and RAID.
>>>
>>> Here are some of the drive's architectural points. Since the two LUNs
>>> share many common components (e.g. spindle) Seagate allocated some
>>> SCSI operations to be LUN specific and some to affect the entire
>>> device, that is, both LUNs.
>>>
>>> 1. Two LUNs, 0 & 1, each with independent lba space, and each
>>> connected to an independent read channel, actuator, and set of heads.
>>> 2. Each actuator addresses 1/2 of the media - no media is shared
>>> across the actuators. They seek independently.
>>> 3. One World Wide Name (WWN) is assigned to the port for device
>>> address. Each Logical Unit has a separate World Wide Name for
>>> identification in VPD page.
>>> 4. 128 deep command queue, shared across both LUNs
>>> 5. Each LUN can pull commands from the queue independently, so they
>>> can implement their own sorting and optimization.
>>> 6. Ordered tag attribute causes the command to be ordered across both
>>> Logical Units
>>> 7. Head of Queue attribute causes the command to be ordered with
>>> respect to a single Logical Unit
>>> 8. Mode pages are device-based (shared across both Logical Units)
>>> 9. Log pages are device-based.
>>> 10. Inquiry VPD pages (with minor exceptions) are device based.
>>> 11. Device health features (SMART, etc) are device based
>>>
>>> Seagate wants the multi-actuator design to integrate into the stack as
>>> painlessly as possible.The interface design is still in the early
>>> stages, so I am gathering requirements and recommendations, and also
>>> providing any information necessary to help scope integrating a
>>> multi-LUN device into the MQ stack. So, I am soliciting any pertinent
>>> feedback including:
>>>
>>> 1. Painful incompatibilities between the Seagate proposal and current
>>> MQ architecture
>>> 2. Linux changes needed
>>> 3. Drive interface changes needed
>>> 4. Anything else I may have overlooked
>>>
>> So far it looks okay; just make sure to have VPD page 0x83
>> entries properly associated.
>> To all intents and purposes these devices seem to look like 'normal'
>> devices with two LUNs; nothing special with that.
>> Real question would be in which areas those devices differentiate from
>> the two indepdendent LUN scenario.
>>
>> There might be issues with per-device informations like SMART etc;
>> ideally they are available from _both_ LUNs.
>> Otherwise they'll show up as blank from one LUN, causing consternation
>> with any management software.
>
>
> Further to this point, some types of damage, such as to a head
> or (one side of) a platter would degrade one LU, possibly making
> it unusable for storage, while the other side (and the other LU)
> would be fine.
>
> I'm curious how you plan to implement the START STOP UNIT command.
> If one side of the platter is in "start" state and the other side
> in "stop" state, will the heads on the stopped side be parked (if
> they can be parked)? And if both sides (LUs) are stopped I would
> hope you really would spin down the disk, then if either is started
> the disk would be spun up.
>
> Getting T10 to add a bit to the Block Device Characteristics VPD page
> might be helpful. It could be a "shares a spindle" bit with the other
> LUs identified in the SCSI Ports VPD page. Such an indication would
> help an enclosure find out if a Multi-Actuator disk was really spun down
> and ready to be removed or replaced. I think SES and smartmontools may
> need tweaks to handle this new device model sensibly.
>
> Doug Gilbert
>
>



-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

Hello-

Concerning how we are currently allocating commands to LUNs or the
device as a whole, here is a list based on the current two LUN model.
This model has LUN0 & LUN1, both reporting 1/2 the total storage. Our
definition of "device based" is that it ignores the LUN field and
executes the command on the entire device. In other words, sending a
device based command to LUN1 will also act on LUN0. "LUN-based"
commands affect only the LUN they're addressed to. I'm soliciting
feedback and suggestions, as well as subject matter experts to point
out pain points and incompatibilities. Thank you for your input.

These commands ignore the LUN field and affect all LUNs on the device:
0x00: TEST UNIT READY. Applies to entire device. The drive will return
a GOOD status only if both LUNs can service medium access commands.
0x01: REZERO. Applies to entire device. The command will force the
seek to LBA 0 on both LUNs. The thermal compensation and other actions
are also taken at both LUNs (actuators)."
0x04: FORMAT UNIT. Applies to entire device. The format parameters are
applied to both LUN's. The format operation is done in parallel on the
two LUN's. Format with defect list is not supported for the Dual LUN
drive."
0x12: INQUIRY. Applies to entire device. The same information is
returned for the Inquiry command regardless of LUN setting. Each LUN
has different identifier.
0x1B: START STOP UNIT. Applies to entire device. The command will
apply to both actuators - it will cause both actuators to be either
spin down or spin-up depending on the command options. If the command
fails on either actuator check condition is returned.
0x35: SYNCRONIZE CACHE. Applies to entire device. This will be a
device command and only support the option to flush the entire cache.
The drive does not support the flush of a particular LBA range only.
0x37: READ DEFECT DATA (10). Applies to entire device. Device based
defect list is returned - this will include the defects from both the
LUNs. The heads are sequentially numbered across both LUNs.
0x3B: WRITE BUFFER (10) Download. Applies to entire device. This is a
device based command - as part of the download the code on both the
LUN's will be updated.
0x3B: WRITE BUFFER (10) other than download. Applies to entire device.
Other than download Device based command - there is only one common
buffer for the two LUNs.
0x3C: READ BUFFER (10). Applies to entire device. Device based command
- there is only one common buffer for the two LUNs.
0x48 0x01: SANITIZE overwrite. Applies to entire device. Treated as a
device level command - sanitize operation performed on both LUNs when
command received.
0x48 0x03: SANITIZE security erase. Applies to entire device. Treated
as a device level command - sanitize operation performed on both LUNs
when command received.
0x48 0x1F: SANITIZE exit failure mode. Applies to entire device.
0x91: SYNCRONIZE CACHE (16). Applies to entire device. Same as Sync Cache.
0x4C: LOG SELECT (10). Applies to entire device. One global set of log
pages for both LUNs. Any LBA information is stored as an internal LBA
value, i.e. LUN1 LBAs start at LUN0 last_LBA + 1.
0x4D: LOG SENSE (10). Applies to entire device. One global set of log
pages for both LUNs. Any LBA information is stored as an internal LBA
value, i.e. LUN1 LBAs start at LUN0 last_LBA + 1.
0x55: MODE SELECT (10). Applies to entire device. Same as Mode select.
0x5A: MODE SENSE (10). Applies to entire device. Same as Mode sense.
0x9E 0x17: GET PHYSICAL ELEMENT STATUS. Applies to entire device.
0x9E 0x18: REMOVE ELEMENT AND TRUNCATE. Applies to entire device.
0xA0: REPORT LUNS. Applies to entire device. Returns information on
the two/multiple LUNs supported by the drive.
0xA2: SECURITY PROTOCOL IN. Applies to entire device.
0xA3 0x0C: REPORT SUPPORTED OP CODES. Applies to entire device.
0xA3 0x0D: REPORT SUPPORTED TMFS. Applies to entire device.
0xA3 0x0F: REPORT TIMESTAMP. Applies to entire device.
0xA4 0x0C: REMOVE I_T NEXUS. Applies to entire device.
0xB7: READ DEFECT DATA (12). Applies to entire device.
0xF7: READ UDS DATA. Applies to entire device.

These commands honor the LUN field and affect the addressed LUN only:
0x5E: PERSISTENT RESERVE IN. LUN Specific.
0x5F: PERSISTENT RESERVE OUT. LUN Specific.
0x9F 0x11: WRITE LONG (16). LUN Specific. Only support WR_UNCOR option
to make the sectors un-correctable.
0xA3 0x05: REPORT DEVICE ID. LUN Specific.
0xA4 0x06: SET DEVICE ID. LUN Specific.
0xB5: SECURITY PROTOCOL OUT. LUN Specific.
0x03: REQUEST SENSE.  LUN Specific. The command returns the sense data
for the respective LUN.
0x07: REASSIGN BLOCKS. LUN Specific. The reassign command will be LUN
specific. It reassigns the defective blocks in the defect list to the
reassign area on the respective LUN.
0x25: READ CAPACITY (10). LUN Specific. The capacity for the LUN
specified in the CDB is returned - the capacity can be different for
the two LUN's in the drive.
0x3E: READ LONG (10). LUN Specific
0x3F 0x11: WRITE LONG (10). LUN Specific
0x94 0x01: CLOSE ZONE. LUN Specific. ZBC.
0x94 0x02: FINISH ZONE. LUN Specific. ZBC.
0x94 0x03: OPEN ZONE. LUN Specific. ZBC.
0x94 0x04: RESET WRITE POINTER. LUN Specific. ZBC.
0x95 0x00: REPORT_ZONES. LUN Specific. The ZBC zone information is
returned per LUN.
0x9B: READ BUFFER (16). LUN Specific. Same as Read Buffer.
0x9E 0x10: READ CAPACITY (16). LUN Specific.

These commands can affect the device OR the LUN:
0x15: MODE SELECT (6). Device/ LUN. Single set of mode page parameters
are supported for the two LUN's. Only the 'Number of Blocks' in the
Block Descriptor may be different for the two LUNs. The option to set
the capacity for a LUN is not supported. If sector size is changed it
will impact both LUNs."
0x1A: MODE SENSE (6). Device/ LUN. Capacity on each LUN can be
different and so the 'Number of Blocks' in the Block Descriptor may be
different for the two LUNs.
0x1C: RECEIVE DIAGNOSTIC. Device/LUN. The data for the LUN specified
in the command is returned.
0x1D: SEND DIAGNOSTIC. Device/ LUN. The device will perform the
diagnostic operations (self-test) on both the LUNs. The 'translate
address' operation is performed on the LUN specified.

Best regards,
-Tim
(303) 775-3770

On Fri, Mar 30, 2018 at 7:07 AM, Tim Walker <tim.t.walker@seagate.com> wrote:
> Hi Doug-
>
> Currently, the dual actuator firmware safely spins the drive down if
> either LUN receives the START STOP UNIT command.  In other words, if
> LUN1 receives the command, it will flush any dirty data from LUN1l and
> LUN0, then spin down, taking both LUN1 & LUN0 off line. Alternatively,
> we've had input that either:
> a) Both LUNs must receive the START STOP UNIT command before the drive
> will spin down, OR
> b) Move the storage to LUN1 & LUN2, keeping LUN0 (with no storage) for
> device specific commands such as START STOP UNIT that do not directly
> access the media.
>
> Thanks for the question.
>
> Best regards,
> -Tim
>
> On Thu, Mar 29, 2018 at 12:03 PM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>> On 2018-03-26 11:08 AM, Hannes Reinecke wrote:
>>>
>>> On Fri, 23 Mar 2018 08:57:12 -0600
>>> Tim Walker <tim.t.walker@seagate.com> wrote:
>>>
>>>> Seagate announced their split actuator SAS drive, which will probably
>>>> require some kernel changes for full support. It's targeted at cloud
>>>> provider JBODs and RAID.
>>>>
>>>> Here are some of the drive's architectural points. Since the two LUNs
>>>> share many common components (e.g. spindle) Seagate allocated some
>>>> SCSI operations to be LUN specific and some to affect the entire
>>>> device, that is, both LUNs.
>>>>
>>>> 1. Two LUNs, 0 & 1, each with independent lba space, and each
>>>> connected to an independent read channel, actuator, and set of heads.
>>>> 2. Each actuator addresses 1/2 of the media - no media is shared
>>>> across the actuators. They seek independently.
>>>> 3. One World Wide Name (WWN) is assigned to the port for device
>>>> address. Each Logical Unit has a separate World Wide Name for
>>>> identification in VPD page.
>>>> 4. 128 deep command queue, shared across both LUNs
>>>> 5. Each LUN can pull commands from the queue independently, so they
>>>> can implement their own sorting and optimization.
>>>> 6. Ordered tag attribute causes the command to be ordered across both
>>>> Logical Units
>>>> 7. Head of Queue attribute causes the command to be ordered with
>>>> respect to a single Logical Unit
>>>> 8. Mode pages are device-based (shared across both Logical Units)
>>>> 9. Log pages are device-based.
>>>> 10. Inquiry VPD pages (with minor exceptions) are device based.
>>>> 11. Device health features (SMART, etc) are device based
>>>>
>>>> Seagate wants the multi-actuator design to integrate into the stack as
>>>> painlessly as possible.The interface design is still in the early
>>>> stages, so I am gathering requirements and recommendations, and also
>>>> providing any information necessary to help scope integrating a
>>>> multi-LUN device into the MQ stack. So, I am soliciting any pertinent
>>>> feedback including:
>>>>
>>>> 1. Painful incompatibilities between the Seagate proposal and current
>>>> MQ architecture
>>>> 2. Linux changes needed
>>>> 3. Drive interface changes needed
>>>> 4. Anything else I may have overlooked
>>>>
>>> So far it looks okay; just make sure to have VPD page 0x83
>>> entries properly associated.
>>> To all intents and purposes these devices seem to look like 'normal'
>>> devices with two LUNs; nothing special with that.
>>> Real question would be in which areas those devices differentiate from
>>> the two indepdendent LUN scenario.
>>>
>>> There might be issues with per-device informations like SMART etc;
>>> ideally they are available from _both_ LUNs.
>>> Otherwise they'll show up as blank from one LUN, causing consternation
>>> with any management software.
>>
>>
>> Further to this point, some types of damage, such as to a head
>> or (one side of) a platter would degrade one LU, possibly making
>> it unusable for storage, while the other side (and the other LU)
>> would be fine.
>>
>> I'm curious how you plan to implement the START STOP UNIT command.
>> If one side of the platter is in "start" state and the other side
>> in "stop" state, will the heads on the stopped side be parked (if
>> they can be parked)? And if both sides (LUs) are stopped I would
>> hope you really would spin down the disk, then if either is started
>> the disk would be spun up.
>>
>> Getting T10 to add a bit to the Block Device Characteristics VPD page
>> might be helpful. It could be a "shares a spindle" bit with the other
>> LUs identified in the SCSI Ports VPD page. Such an indication would
>> help an enclosure find out if a Multi-Actuator disk was really spun down
>> and ready to be removed or replaced. I think SES and smartmontools may
>> need tweaks to handle this new device model sensibly.
>>
>> Doug Gilbert
>>
>>
>
>
>
> --
> Tim Walker
> Product Design Systems Engineering, Seagate Technology
> (303) 775-3770



-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Bart Van Assche]

T24gRnJpLCAyMDE4LTAzLTMwIGF0IDEyOjA3IC0wNjAwLCBUaW0gV2Fsa2VyIHdyb3RlOg0KPiBD
b25jZXJuaW5nIGhvdyB3ZSBhcmUgY3VycmVudGx5IGFsbG9jYXRpbmcgY29tbWFuZHMgdG8gTFVO
cyBvciB0aGUNCj4gZGV2aWNlIGFzIGEgd2hvbGUsIGhlcmUgaXMgYSBsaXN0IGJhc2VkIG9uIHRo
ZSBjdXJyZW50IHR3byBMVU4gbW9kZWwuDQo+IFRoaXMgbW9kZWwgaGFzIExVTjAgJiBMVU4xLCBi
b3RoIHJlcG9ydGluZyAxLzIgdGhlIHRvdGFsIHN0b3JhZ2UuIE91cg0KPiBkZWZpbml0aW9uIG9m
ICJkZXZpY2UgYmFzZWQiIGlzIHRoYXQgaXQgaWdub3JlcyB0aGUgTFVOIGZpZWxkIGFuZA0KPiBl
eGVjdXRlcyB0aGUgY29tbWFuZCBvbiB0aGUgZW50aXJlIGRldmljZS4gWyAuLi4gXQ0KDQpIZWxs
byBUaW0sDQoNCkNhbiB5b3UgY2xhcmlmeSB3aGF0ICJMVU4gZmllbGQiIG1lYW5zIGluIHRoaXMg
Y29udGV4dD8gQXJlIHlvdSByZWZlcnJpbmcNCnRvIGEgZmllbGQgaW4gdGhlIFNBUyB0cmFuc3Bv
cnQgaGVhZGVyIG9yIHBlcmhhcHMgdG8gdGhlIG9ic29sZXRlIExVTg0KZmllbGQgaW4gdGhlIFND
U0kgQ0RCPyBBZGRpdGlvbmFsbHksIGFzIGZhciBhcyBJIGtub3cgZXZlcnkgU0NTSSBjb21tYW5k
DQpzaG91bGQgYmUgZXhlY3V0ZWQgYnkgZXhhY3RseSBvbmUgU0NTSSBMVU4uIEknbSBub3QgYXdh
cmUgb2YgYW55IHN1cHBvcnQNCmluIHRoZSBTQ1NJIHNwZWNzIGZvciBicm9hZGNhc3RpbmcgYSBz
aW5nbGUgU0NTSSBjb21tYW5kIHRvIG11bHRpcGxlIExVTnMuDQoNClRoYW5rcywNCg0KQmFydC4N
Cg0KDQoNCg0K

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

Hi Bart-

Yes, the header LUN field. Sorry!

We hadn't intended to broadcast - we expect to see a LUN specified.
For a device specific command both LUNs will be affected regardless of
which LUN is specified in the transport field. e.g. if we command LUN1
to stop (START STOP UNIT) then LUN0 will also be stopped.

Sorry if my terminology wasn't entirely clear - some jargon and
informal usage slipping through.

Best regards,
-Tim

-Tim

On Fri, Mar 30, 2018 at 12:17 PM, Bart Van Assche
<Bart.VanAssche@wdc.com> wrote:
> On Fri, 2018-03-30 at 12:07 -0600, Tim Walker wrote:
>> Concerning how we are currently allocating commands to LUNs or the
>> device as a whole, here is a list based on the current two LUN model.
>> This model has LUN0 & LUN1, both reporting 1/2 the total storage. Our
>> definition of "device based" is that it ignores the LUN field and
>> executes the command on the entire device. [ ... ]
>
> Hello Tim,
>
> Can you clarify what "LUN field" means in this context? Are you referring
> to a field in the SAS transport header or perhaps to the obsolete LUN
> field in the SCSI CDB? Additionally, as far as I know every SCSI command
> should be executed by exactly one SCSI LUN. I'm not aware of any support
> in the SCSI specs for broadcasting a single SCSI command to multiple LUNs.
>
> Thanks,
>
> Bart.
>
>
>
>



-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Bart Van Assche]

T24gRnJpLCAyMDE4LTAzLTMwIGF0IDEyOjIxIC0wNjAwLCBUaW0gV2Fsa2VyIHdyb3RlOg0KPiBZ
ZXMsIHRoZSBoZWFkZXIgTFVOIGZpZWxkLiBTb3JyeSENCj4gDQo+IFdlIGhhZG4ndCBpbnRlbmRl
ZCB0byBicm9hZGNhc3QgLSB3ZSBleHBlY3QgdG8gc2VlIGEgTFVOIHNwZWNpZmllZC4NCj4gRm9y
IGEgZGV2aWNlIHNwZWNpZmljIGNvbW1hbmQgYm90aCBMVU5zIHdpbGwgYmUgYWZmZWN0ZWQgcmVn
YXJkbGVzcyBvZg0KPiB3aGljaCBMVU4gaXMgc3BlY2lmaWVkIGluIHRoZSB0cmFuc3BvcnQgZmll
bGQuIGUuZy4gaWYgd2UgY29tbWFuZCBMVU4xDQo+IHRvIHN0b3AgKFNUQVJUIFNUT1AgVU5JVCkg
dGhlbiBMVU4wIHdpbGwgYWxzbyBiZSBzdG9wcGVkLg0KPiANCj4gU29ycnkgaWYgbXkgdGVybWlu
b2xvZ3kgd2Fzbid0IGVudGlyZWx5IGNsZWFyIC0gc29tZSBqYXJnb24gYW5kDQo+IGluZm9ybWFs
IHVzYWdlIHNsaXBwaW5nIHRocm91Z2guDQoNCkhlbGxvIFRpbSwNCg0KRG8geW91IHBlcmhhcHMg
cGxhbiB0byBhdHRlbmQgTFNGL01NIG5leHQgbW9udGg/IEl0IHByb2JhYmx5IHdpbGwgYmUgZWFz
aWVyDQp0byBkaXNjdXNzIHRoaXMgZnVydGhlciBpbiBwZXJzb24gaW5zdGVhZCBvZiBjb250aW51
aW5nIHRoaXMgZS1tYWlsIHRocmVhZC4NClNlZSBhbHNvIGh0dHBzOi8vZXZlbnRzLmxpbnV4Zm91
bmRhdGlvbi5vcmcvZXZlbnRzL2xzZm1tLTIwMTgvLg0KDQpUaGFua3MsDQoNCkJhcnQuDQoNCg0K
DQoNCg==

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

Yes I will be there to discuss the multi-LUN approach. I wanted to get
these interface details out so we could have some background and
perhaps folks would come with ideas. I don't have much more to put
out, but I will certainly answer questions - either on this list or
directly.

Best regards,
-Tim

On Fri, Mar 30, 2018 at 12:31 PM, Bart Van Assche
<Bart.VanAssche@wdc.com> wrote:
> On Fri, 2018-03-30 at 12:21 -0600, Tim Walker wrote:
>
>> Yes, the header LUN field. Sorry!
>
>>
>
>> We hadn't intended to broadcast - we expect to see a LUN specified.
>
>> For a device specific command both LUNs will be affected regardless of
>
>> which LUN is specified in the transport field. e.g. if we command LUN1
>
>> to stop (START STOP UNIT) then LUN0 will also be stopped.
>
>>
>
>> Sorry if my terminology wasn't entirely clear - some jargon and
>
>> informal usage slipping through.
>
>
>
> Hello Tim,
>
>
>
> Do you perhaps plan to attend LSF/MM next month? It probably will be easi=
er
>
> to discuss this further in person instead of continuing this e-mail threa=
d.
>
> See also https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__events.li=
nuxfoundation.org_events_lsfmm-2D2018_&d=3DDwIGaQ&c=3DIGDlg0lD0b-nebmJJ0Kp8=
A&r=3DNW1X0yRHNNEluZ8sOGXBxCbQJZPWcIkPT0Uy3ynVsFU&m=3DMebnGjWzpdtum7IWMvt4l=
PVFr7aZr4pLz9A4pVL9uFM&s=3DSPtZIJ6pKfO5rlo4MokPxd09lri5kKheGP56t-zooiY&e=3D=
.
>
>
>
> Thanks,
>
>
>
> Bart.
>
>
>
>
>
>
>
>
>



--=20
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Bart Van Assche]

T24gRnJpLCAyMDE4LTAzLTMwIGF0IDEyOjM2IC0wNjAwLCBUaW0gV2Fsa2VyIHdyb3RlOg0KPiBZ
ZXMgSSB3aWxsIGJlIHRoZXJlIHRvIGRpc2N1c3MgdGhlIG11bHRpLUxVTiBhcHByb2FjaC4gSSB3
YW50ZWQgdG8gZ2V0DQo+IHRoZXNlIGludGVyZmFjZSBkZXRhaWxzIG91dCBzbyB3ZSBjb3VsZCBo
YXZlIHNvbWUgYmFja2dyb3VuZCBhbmQNCj4gcGVyaGFwcyBmb2xrcyB3b3VsZCBjb21lIHdpdGgg
aWRlYXMuIEkgZG9uJ3QgaGF2ZSBtdWNoIG1vcmUgdG8gcHV0DQo+IG91dCwgYnV0IEkgd2lsbCBj
ZXJ0YWlubHkgYW5zd2VyIHF1ZXN0aW9ucyAtIGVpdGhlciBvbiB0aGlzIGxpc3Qgb3INCj4gZGly
ZWN0bHkuDQoNCkhlbGxvIFRpbSwNCg0KQXMgZmFyIGFzIEkga25vdyB0aGUgTGludXggU0NTSSBz
dGFjayBkb2VzIG5vdCB5ZXQgc3VwcG9ydCBhbnkgU0NTSSBkZXZpY2VzDQpmb3Igd2hpY2ggYSBz
aW5nbGUgU0NTSSBjb21tYW5kIGFmZmVjdHMgbXVsdGlwbGUgKHR3byBpbiB0aGlzIGNhc2UpIExV
TnMuDQpBZGRpbmcgc3VjaCBzdXBwb3J0IG1heSB0YWtlIHNpZ25pZmljYW50IHdvcmsuIFRoZXJl
IHdpbGwgYWxzbyBiZSB0aGUNCmRpc2FkdmFudGFnZSB0aGF0IG1vc3QgU0NTSSBjb3JlIGNvbnRy
aWJ1dG9ycyBkbyBub3QgaGF2ZSBhY2Nlc3MgdG8gYSBtdWx0aS0NCmFjdHVhdG9yIGRldmljZSBh
bmQgaGVuY2UgdGhhdCB0aGVpciBjaGFuZ2VzIG1heSBicmVhayBzdXBwb3J0IGZvciBtdWx0aS0N
CmFjdHVhdG9yIGRldmljZXMuDQoNClBTOiB0aGUgcHJlZmVycmVkIHN0eWxlIGZvciBhbGwgTGlu
dXgga2VybmVsIG1haWxpbmcgbGlzdHMgSSBrbm93IG9mIGlzIHRvIHJlcGx5DQpiZWxvdyBhIHBy
ZXZpb3VzIG1lc3NhZ2UuIFNlZSBhbHNvIGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL1Bv
c3Rpbmdfc3R5bGUuDQoNClRoYW5rcywNCg0KQmFydC4NCg0KDQoNCg==

Re: Multi-Actuator SAS HDD First Look

[Hannes Reinecke]

On 03/30/2018 03:07 PM, Tim Walker wrote:
> Hi Doug-
> 
> Currently, the dual actuator firmware safely spins the drive down if
> either LUN receives the START STOP UNIT command.  In other words, if
> LUN1 receives the command, it will flush any dirty data from LUN1l and
> LUN0, then spin down, taking both LUN1 & LUN0 off line. Alternatively,
> we've had input that either:
> a) Both LUNs must receive the START STOP UNIT command before the drive
> will spin down, OR
> b) Move the storage to LUN1 & LUN2, keeping LUN0 (with no storage) for
> device specific commands such as START STOP UNIT that do not directly
> access the media.
> 
That will get interesting when suspending the device; physical
interactions between independent LUNs are always tricky. And I don't
even want to imagine the implications when assigning each LUN to
individual VM guests; shutting down one VM will also interfere with the
other VM ... shudder.

Actually I would propose to have a 'management' LUN at LUN0, who could
handle all the device-wide commands (eg things like START STOP UNIT,
firmware update, or even SMART commands), and ignoring them for the
remaining LUNs.

I guess that would make the whole setup easier to handle.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: Multi-Actuator SAS HDD First Look

[Hannes Reinecke]

On 03/30/2018 08:07 PM, Tim Walker wrote:
> Hello-
> 
> Concerning how we are currently allocating commands to LUNs or the
> device as a whole, here is a list based on the current two LUN model.
> This model has LUN0 & LUN1, both reporting 1/2 the total storage. Our
> definition of "device based" is that it ignores the LUN field and
> executes the command on the entire device. In other words, sending a
> device based command to LUN1 will also act on LUN0. "LUN-based"
> commands affect only the LUN they're addressed to. I'm soliciting
> feedback and suggestions, as well as subject matter experts to point
> out pain points and incompatibilities. Thank you for your input.
> 
Uh. You will have fun pushing that past T-10 ...

> These commands ignore the LUN field and affect all LUNs on the device:
> 0x00: TEST UNIT READY. Applies to entire device. The drive will return
> a GOOD status only if both LUNs can service medium access commands.
Please, don't.
TEST UNIT READY is the _only_ command at SPC level allowing us to check
the state of the LUN. Moving that up the device (ie target) level is
leaving us with no idea about the status of the actual LUN.

> 0x01: REZERO. Applies to entire device. The command will force the
> seek to LBA 0 on both LUNs. The thermal compensation and other actions
> are also taken at both LUNs (actuators)."
Why? Is there any necessity that a REZERO of one LUN has a dependency on
the other?

> 0x04: FORMAT UNIT. Applies to entire device. The format parameters are
> applied to both LUN's. The format operation is done in parallel on the
> two LUN's. Format with defect list is not supported for the Dual LUN
> drive."
Again, why? Just for performance reasons?
That surely can be done by issuing a FORMAT UNIT command asynchonously
to both devices ...

> 0x12: INQUIRY. Applies to entire device. The same information is
> returned for the Inquiry command regardless of LUN setting. Each LUN
> has different identifier.
Strongly against it.
INQUIRY is _THE_ prime identifer of the LUN, Both LUNs might return the
same INQUIRY data, at the very least page 0x93 of the VPD data _NEEDS_
to be different both both LUNs.
Please keep it at LUN level.

> 0x1B: START STOP UNIT. Applies to entire device. The command will
> apply to both actuators - it will cause both actuators to be either
> spin down or spin-up depending on the command options. If the command
> fails on either actuator check condition is returned.
Already discussed. Probably no other way around it.

> 0x35: SYNCRONIZE CACHE. Applies to entire device. This will be a
> device command and only support the option to flush the entire cache.
> The drive does not support the flush of a particular LBA range only.
Fine with that; most HBAs have the same limitation.

> 0x37: READ DEFECT DATA (10). Applies to entire device. Device based
> defect list is returned - this will include the defects from both the
> LUNs. The heads are sequentially numbered across both LUNs.
Hmm. Again, why?

> 0x3B: WRITE BUFFER (10) Download. Applies to entire device. This is a
> device based command - as part of the download the code on both the
> LUN's will be updated.
About to be expected. Okay.

> 0x3B: WRITE BUFFER (10) other than download. Applies to entire device.
> Other than download Device based command - there is only one common
> buffer for the two LUNs.
See above. Okay.

> 0x3C: READ BUFFER (10). Applies to entire device. Device based command
> - there is only one common buffer for the two LUNs.
Might be worthwhile adding a new option to READ/WRITE BUFFER (eg using
one of the bits before the 'MODE' field), specifying that this command
applies to all LUNs on this device.

> 0x48 0x01: SANITIZE overwrite. Applies to entire device. Treated as a
> device level command - sanitize operation performed on both LUNs when
> command received.> 0x48 0x03: SANITIZE security erase. Applies to entire device. Treated
> as a device level command - sanitize operation performed on both LUNs
> when command received.
> 0x48 0x1F: SANITIZE exit failure mode. Applies to entire device.
Emphatically no.
That would allow one application on one LUN erasing the contents of the
other LUN, which might be in use by a completely different application.

> 0x91: SYNCRONIZE CACHE (16). Applies to entire device. Same as Sync Cache.
> 0x4C: LOG SELECT (10). Applies to entire device. One global set of log
> pages for both LUNs. Any LBA information is stored as an internal LBA
> value, i.e. LUN1 LBAs start at LUN0 last_LBA + 1.
> 0x4D: LOG SENSE (10). Applies to entire device. One global set of log
> pages for both LUNs. Any LBA information is stored as an internal LBA
> value, i.e. LUN1 LBAs start at LUN0 last_LBA + 1.
> 0x55: MODE SELECT (10). Applies to entire device. Same as Mode select.
> 0x5A: MODE SENSE (10). Applies to entire device. Same as Mode sense.
> 0x9E 0x17: GET PHYSICAL ELEMENT STATUS. Applies to entire device.
> 0x9E 0x18: REMOVE ELEMENT AND TRUNCATE. Applies to entire device.
> 0xA0: REPORT LUNS. Applies to entire device. Returns information on
> the two/multiple LUNs supported by the drive.
> 0xA2: SECURITY PROTOCOL IN. Applies to entire device.
> 0xA3 0x0C: REPORT SUPPORTED OP CODES. Applies to entire device.
> 0xA3 0x0D: REPORT SUPPORTED TMFS. Applies to entire device.
> 0xA3 0x0F: REPORT TIMESTAMP. Applies to entire device.
> 0xA4 0x0C: REMOVE I_T NEXUS. Applies to entire device.
> 0xB7: READ DEFECT DATA (12). Applies to entire device.
> 0xF7: READ UDS DATA. Applies to entire device.
> 
> These commands honor the LUN field and affect the addressed LUN only:
> 0x5E: PERSISTENT RESERVE IN. LUN Specific.
> 0x5F: PERSISTENT RESERVE OUT. LUN Specific.
> 0x9F 0x11: WRITE LONG (16). LUN Specific. Only support WR_UNCOR option
> to make the sectors un-correctable.
> 0xA3 0x05: REPORT DEVICE ID. LUN Specific.
> 0xA4 0x06: SET DEVICE ID. LUN Specific.
> 0xB5: SECURITY PROTOCOL OUT. LUN Specific.
> 0x03: REQUEST SENSE.  LUN Specific. The command returns the sense data
> for the respective LUN.
> 0x07: REASSIGN BLOCKS. LUN Specific. The reassign command will be LUN
> specific. It reassigns the defective blocks in the defect list to the
> reassign area on the respective LUN.
> 0x25: READ CAPACITY (10). LUN Specific. The capacity for the LUN
> specified in the CDB is returned - the capacity can be different for
> the two LUN's in the drive.
> 0x3E: READ LONG (10). LUN Specific
> 0x3F 0x11: WRITE LONG (10). LUN Specific
> 0x94 0x01: CLOSE ZONE. LUN Specific. ZBC.
> 0x94 0x02: FINISH ZONE. LUN Specific. ZBC.
> 0x94 0x03: OPEN ZONE. LUN Specific. ZBC.
> 0x94 0x04: RESET WRITE POINTER. LUN Specific. ZBC.
> 0x95 0x00: REPORT_ZONES. LUN Specific. The ZBC zone information is
> returned per LUN.
> 0x9B: READ BUFFER (16). LUN Specific. Same as Read Buffer.
> 0x9E 0x10: READ CAPACITY (16). LUN Specific.
> 
> These commands can affect the device OR the LUN:
> 0x15: MODE SELECT (6). Device/ LUN. Single set of mode page parameters
> are supported for the two LUN's. Only the 'Number of Blocks' in the
> Block Descriptor may be different for the two LUNs. The option to set
> the capacity for a LUN is not supported. If sector size is changed it
> will impact both LUNs."
> 0x1A: MODE SENSE (6). Device/ LUN. Capacity on each LUN can be
> different and so the 'Number of Blocks' in the Block Descriptor may be
> different for the two LUNs.
> 0x1C: RECEIVE DIAGNOSTIC. Device/LUN. The data for the LUN specified
> in the command is returned.
> 0x1D: SEND DIAGNOSTIC. Device/ LUN. The device will perform the
> diagnostic operations (self-test) on both the LUNs. The 'translate
> address' operation is performed on the LUN specified.
> 
In general, I think it would be far better treating this entire device
just like a normal target having two LUNs.
I would actually advocating having a device with _three_ LUNs, where one
LUN would serve as an endpoint for all device-centric commands, and the
other LUNs simply won't implement the device-centric commands.
Then you wouldn't need this dance with commands which might be executed
at device or LUN context, and the whole thing might be better aligned
with T10.

I guess we should talk at LSF about this ...

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: Multi-Actuator SAS HDD First Look

[Douglas Gilbert]

On 2018-03-30 04:01 PM, Bart Van Assche wrote:
> On Fri, 2018-03-30 at 12:36 -0600, Tim Walker wrote:
>> Yes I will be there to discuss the multi-LUN approach. I wanted to get
>> these interface details out so we could have some background and
>> perhaps folks would come with ideas. I don't have much more to put
>> out, but I will certainly answer questions - either on this list or
>> directly.
> 
> Hello Tim,
> 
> As far as I know the Linux SCSI stack does not yet support any SCSI devices
> for which a single SCSI command affects multiple (two in this case) LUNs.
> Adding such support may take significant work. There will also be the
> disadvantage that most SCSI core contributors do not have access to a multi-
> actuator device and hence that their changes may break support for multi-
> actuator devices.

Hmmm, INQUIRY (3PC bit) and REPORT LUNS seem to be counter examples to
Bart's assertion. Plus there are a more that tell you about things outside
the addressed LU, for example the SCSI Ports VPD page tells you about other
SCSI ports hence other LUs) on the current device.


 From Tim's command list:

Device level
------------
0x0, 0x1: okay
0x4 (Format unit): yikes, that could be a nasty surprise, accessing a file
   system on the other LU and getting an error "Not ready, format in progress"!!
0x12: standard INQUIRY okay, VPD pages not so much LU id different; relative
   port id, different; target port id different (at the least)
0x1b (SSU): storage LUs need to know this model, otherwise the logic on
   each LU could get into a duel: "I want it spun up; no, I want it spun
   down ..."
0x35, 0x37, 0x3b, 0x3c: okay
0x48 (sanitize): similar to Format unit above
0x91,0x4c,0x4d: okay
MODE SENSE/SELECT(6,10): depends on page, block descriptor needs to be
   partially device level (since LB size may be changed by FU which is
   device level)
rest of device level: okay or (I) don't know
0xf7: READ UDS DATA, that's interesting, but proprietary I guess

Perhaps you could add a rider on FU and SAN: they get rejected unless the
other storage LU is in (logical) spun down state.


LU specific
-----------
all okay, I hoping READ(6,10,12,16,32) and their WRITE cousins will be
   there also :-) Plus the TMF: LU reset

Device or LU
------------
all okay


I'm intrigued by your 3 LU model. My wish list for that:

LUN 0 would be processor device type (0x3) so it wouldn't confuse the
OS (Linux) that it held storage (READ CAPACITY is mandatory for PDT 0x0
and cannot represent a 0 block LU) and you could pick and choose which
SCSI commands to implement on it. LUN 0 TUR could report what the spindle
was really doing, SSU could do precisely what it is told (and SSU on LUNs
1 and 2 would be an "and" for spin down and an "or" for spin up). I've
got several boxes full of SAS cables and only one cable that I can think
of that lets me get to the secondary SAS port. So on LUN 0 you could have
a proprietary (unless T10 takes it on board) mode page to enable the user
to say which ports that LUNs 1 and 2 where accessible on. Obviously LUN 0
would need to be accessible on both ports. [A non-accessible LUN would
still respond to INQUIRY but with a first byte of 07f: PDT=0x1f (unknown)
and PQ=3 which implies it is there but inaccessible via the current port.]

A processor PDT opens up the possibility of putting a copy manager on
LUN 0. Think offloaded (from main machine's perspective) backups and
restores where LUN 1 or 2 is the source or destination.

Enough dreaming.

Doug Gilbert

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Sat, Mar 31, 2018 at 10:52 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
> On 2018-03-30 04:01 PM, Bart Van Assche wrote:
>>
>> On Fri, 2018-03-30 at 12:36 -0600, Tim Walker wrote:
>>>
>>> Yes I will be there to discuss the multi-LUN approach. I wanted to get
>>> these interface details out so we could have some background and
>>> perhaps folks would come with ideas. I don't have much more to put
>>> out, but I will certainly answer questions - either on this list or
>>> directly.
>>
>>
>> Hello Tim,
>>
>> As far as I know the Linux SCSI stack does not yet support any SCSI
>> devices
>> for which a single SCSI command affects multiple (two in this case) LUNs.
>> Adding such support may take significant work. There will also be the
>> disadvantage that most SCSI core contributors do not have access to a
>> multi-
>> actuator device and hence that their changes may break support for multi-
>> actuator devices.
>
>
> Hmmm, INQUIRY (3PC bit) and REPORT LUNS seem to be counter examples to
> Bart's assertion. Plus there are a more that tell you about things outside
> the addressed LU, for example the SCSI Ports VPD page tells you about other
> SCSI ports hence other LUs) on the current device.
>
>
> From Tim's command list:
>
> Device level
> ------------
> 0x0, 0x1: okay
> 0x4 (Format unit): yikes, that could be a nasty surprise, accessing a file
>   system on the other LU and getting an error "Not ready, format in
> progress"!!
> 0x12: standard INQUIRY okay, VPD pages not so much LU id different; relative
>   port id, different; target port id different (at the least)
> 0x1b (SSU): storage LUs need to know this model, otherwise the logic on
>   each LU could get into a duel: "I want it spun up; no, I want it spun
>   down ..."
> 0x35, 0x37, 0x3b, 0x3c: okay
> 0x48 (sanitize): similar to Format unit above
> 0x91,0x4c,0x4d: okay
> MODE SENSE/SELECT(6,10): depends on page, block descriptor needs to be
>   partially device level (since LB size may be changed by FU which is
>   device level)
> rest of device level: okay or (I) don't know
> 0xf7: READ UDS DATA, that's interesting, but proprietary I guess
>
> Perhaps you could add a rider on FU and SAN: they get rejected unless the
> other storage LU is in (logical) spun down state.
>
>
> LU specific
> -----------
> all okay, I hoping READ(6,10,12,16,32) and their WRITE cousins will be
>   there also :-) Plus the TMF: LU reset
>
> Device or LU
> ------------
> all okay
>
>
> I'm intrigued by your 3 LU model. My wish list for that:
>
> LUN 0 would be processor device type (0x3) so it wouldn't confuse the
> OS (Linux) that it held storage (READ CAPACITY is mandatory for PDT 0x0
> and cannot represent a 0 block LU) and you could pick and choose which
> SCSI commands to implement on it. LUN 0 TUR could report what the spindle
> was really doing, SSU could do precisely what it is told (and SSU on LUNs
> 1 and 2 would be an "and" for spin down and an "or" for spin up). I've
> got several boxes full of SAS cables and only one cable that I can think
> of that lets me get to the secondary SAS port. So on LUN 0 you could have
> a proprietary (unless T10 takes it on board) mode page to enable the user
> to say which ports that LUNs 1 and 2 where accessible on. Obviously LUN 0
> would need to be accessible on both ports. [A non-accessible LUN would
> still respond to INQUIRY but with a first byte of 07f: PDT=0x1f (unknown)
> and PQ=3 which implies it is there but inaccessible via the current port.]
>
> A processor PDT opens up the possibility of putting a copy manager on
> LUN 0. Think offloaded (from main machine's perspective) backups and
> restores where LUN 1 or 2 is the source or destination.
>
> Enough dreaming.
>
> Doug Gilbert



Thanks for all the input from everybody. I'll collate it for a meeting
with our interface architect.

Best regards,
-TIm

-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Douglas Gilbert]

On 2018-04-02 11:34 AM, Tim Walker wrote:
> On Sat, Mar 31, 2018 at 10:52 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>> On 2018-03-30 04:01 PM, Bart Van Assche wrote:
>>>
>>> On Fri, 2018-03-30 at 12:36 -0600, Tim Walker wrote:
>>>>
>>>> Yes I will be there to discuss the multi-LUN approach. I wanted to get
>>>> these interface details out so we could have some background and
>>>> perhaps folks would come with ideas. I don't have much more to put
>>>> out, but I will certainly answer questions - either on this list or
>>>> directly.
>>>
>>>
>>> Hello Tim,
>>>
>>> As far as I know the Linux SCSI stack does not yet support any SCSI
>>> devices
>>> for which a single SCSI command affects multiple (two in this case) LUNs.
>>> Adding such support may take significant work. There will also be the
>>> disadvantage that most SCSI core contributors do not have access to a
>>> multi-
>>> actuator device and hence that their changes may break support for multi-
>>> actuator devices.
>>
>>
>> Hmmm, INQUIRY (3PC bit) and REPORT LUNS seem to be counter examples to
>> Bart's assertion. Plus there are a more that tell you about things outside
>> the addressed LU, for example the SCSI Ports VPD page tells you about other
>> SCSI ports hence other LUs) on the current device.
>>
>>
>>  From Tim's command list:
>>
>> Device level
>> ------------
>> 0x0, 0x1: okay
>> 0x4 (Format unit): yikes, that could be a nasty surprise, accessing a file
>>    system on the other LU and getting an error "Not ready, format in
>> progress"!!
>> 0x12: standard INQUIRY okay, VPD pages not so much LU id different; relative
>>    port id, different; target port id different (at the least)
>> 0x1b (SSU): storage LUs need to know this model, otherwise the logic on
>>    each LU could get into a duel: "I want it spun up; no, I want it spun
>>    down ..."
>> 0x35, 0x37, 0x3b, 0x3c: okay
>> 0x48 (sanitize): similar to Format unit above
>> 0x91,0x4c,0x4d: okay
>> MODE SENSE/SELECT(6,10): depends on page, block descriptor needs to be
>>    partially device level (since LB size may be changed by FU which is
>>    device level)
>> rest of device level: okay or (I) don't know
>> 0xf7: READ UDS DATA, that's interesting, but proprietary I guess
>>
>> Perhaps you could add a rider on FU and SAN: they get rejected unless the
>> other storage LU is in (logical) spun down state.
>>
>>
>> LU specific
>> -----------
>> all okay, I hoping READ(6,10,12,16,32) and their WRITE cousins will be
>>    there also :-) Plus the TMF: LU reset
>>
>> Device or LU
>> ------------
>> all okay
>>
>>
>> I'm intrigued by your 3 LU model. My wish list for that:
>>
>> LUN 0 would be processor device type (0x3) so it wouldn't confuse the
>> OS (Linux) that it held storage (READ CAPACITY is mandatory for PDT 0x0
>> and cannot represent a 0 block LU) and you could pick and choose which
>> SCSI commands to implement on it. LUN 0 TUR could report what the spindle
>> was really doing, SSU could do precisely what it is told (and SSU on LUNs
>> 1 and 2 would be an "and" for spin down and an "or" for spin up). I've
>> got several boxes full of SAS cables and only one cable that I can think
>> of that lets me get to the secondary SAS port. So on LUN 0 you could have
>> a proprietary (unless T10 takes it on board) mode page to enable the user
>> to say which ports that LUNs 1 and 2 where accessible on. Obviously LUN 0
>> would need to be accessible on both ports. [A non-accessible LUN would
>> still respond to INQUIRY but with a first byte of 07f: PDT=0x1f (unknown)
>> and PQ=3 which implies it is there but inaccessible via the current port.]
>>
>> A processor PDT opens up the possibility of putting a copy manager on
>> LUN 0. Think offloaded (from main machine's perspective) backups and
>> restores where LUN 1 or 2 is the source or destination.
>>
>> Enough dreaming.
>>
>> Doug Gilbert
> 
> 
> 
> Thanks for all the input from everybody. I'll collate it for a meeting
> with our interface architect.

Just to amplify the case against a FU or SAN being allowed at almost any
time from either storage LU irrespective of what the other was doing.
Imagine one initiator has some important data on one LU and has an
EXCLUSIVE ACCESS persistent reservation on it. Then another initiator
(e.g. on a different machine) sends a FU to the other LU which is
honoured, wiping the whole device. One unhappy customer ....

Doug Gilbert

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Mon, Apr 2, 2018 at 10:29 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
> On 2018-04-02 11:34 AM, Tim Walker wrote:
>>
>> On Sat, Mar 31, 2018 at 10:52 AM, Douglas Gilbert <dgilbert@interlog.com>
>> wrote:
>>>
>>> On 2018-03-30 04:01 PM, Bart Van Assche wrote:
>>>>
>>>>
>>>> On Fri, 2018-03-30 at 12:36 -0600, Tim Walker wrote:
>>>>>
>>>>>
>>>>> Yes I will be there to discuss the multi-LUN approach. I wanted to get
>>>>> these interface details out so we could have some background and
>>>>> perhaps folks would come with ideas. I don't have much more to put
>>>>> out, but I will certainly answer questions - either on this list or
>>>>> directly.
>>>>
>>>>
>>>>
>>>> Hello Tim,
>>>>
>>>> As far as I know the Linux SCSI stack does not yet support any SCSI
>>>> devices
>>>> for which a single SCSI command affects multiple (two in this case)
>>>> LUNs.
>>>> Adding such support may take significant work. There will also be the
>>>> disadvantage that most SCSI core contributors do not have access to a
>>>> multi-
>>>> actuator device and hence that their changes may break support for
>>>> multi-
>>>> actuator devices.
>>>
>>>
>>>
>>> Hmmm, INQUIRY (3PC bit) and REPORT LUNS seem to be counter examples to
>>> Bart's assertion. Plus there are a more that tell you about things
>>> outside
>>> the addressed LU, for example the SCSI Ports VPD page tells you about
>>> other
>>> SCSI ports hence other LUs) on the current device.
>>>
>>>
>>>  From Tim's command list:
>>>
>>> Device level
>>> ------------
>>> 0x0, 0x1: okay
>>> 0x4 (Format unit): yikes, that could be a nasty surprise, accessing a
>>> file
>>>    system on the other LU and getting an error "Not ready, format in
>>> progress"!!
>>> 0x12: standard INQUIRY okay, VPD pages not so much LU id different;
>>> relative
>>>    port id, different; target port id different (at the least)
>>> 0x1b (SSU): storage LUs need to know this model, otherwise the logic on
>>>    each LU could get into a duel: "I want it spun up; no, I want it spun
>>>    down ..."
>>> 0x35, 0x37, 0x3b, 0x3c: okay
>>> 0x48 (sanitize): similar to Format unit above
>>> 0x91,0x4c,0x4d: okay
>>> MODE SENSE/SELECT(6,10): depends on page, block descriptor needs to be
>>>    partially device level (since LB size may be changed by FU which is
>>>    device level)
>>> rest of device level: okay or (I) don't know
>>> 0xf7: READ UDS DATA, that's interesting, but proprietary I guess
>>>
>>> Perhaps you could add a rider on FU and SAN: they get rejected unless the
>>> other storage LU is in (logical) spun down state.
>>>
>>>
>>> LU specific
>>> -----------
>>> all okay, I hoping READ(6,10,12,16,32) and their WRITE cousins will be
>>>    there also :-) Plus the TMF: LU reset
>>>
>>> Device or LU
>>> ------------
>>> all okay
>>>
>>>
>>> I'm intrigued by your 3 LU model. My wish list for that:
>>>
>>> LUN 0 would be processor device type (0x3) so it wouldn't confuse the
>>> OS (Linux) that it held storage (READ CAPACITY is mandatory for PDT 0x0
>>> and cannot represent a 0 block LU) and you could pick and choose which
>>> SCSI commands to implement on it. LUN 0 TUR could report what the spindle
>>> was really doing, SSU could do precisely what it is told (and SSU on LUNs
>>> 1 and 2 would be an "and" for spin down and an "or" for spin up). I've
>>> got several boxes full of SAS cables and only one cable that I can think
>>> of that lets me get to the secondary SAS port. So on LUN 0 you could have
>>> a proprietary (unless T10 takes it on board) mode page to enable the user
>>> to say which ports that LUNs 1 and 2 where accessible on. Obviously LUN 0
>>> would need to be accessible on both ports. [A non-accessible LUN would
>>> still respond to INQUIRY but with a first byte of 07f: PDT=0x1f (unknown)
>>> and PQ=3 which implies it is there but inaccessible via the current
>>> port.]
>>>
>>> A processor PDT opens up the possibility of putting a copy manager on
>>> LUN 0. Think offloaded (from main machine's perspective) backups and
>>> restores where LUN 1 or 2 is the source or destination.
>>>
>>> Enough dreaming.
>>>
>>> Doug Gilbert
>>
>>
>>
>>
>> Thanks for all the input from everybody. I'll collate it for a meeting
>> with our interface architect.
>
>
> Just to amplify the case against a FU or SAN being allowed at almost any
> time from either storage LU irrespective of what the other was doing.
> Imagine one initiator has some important data on one LU and has an
> EXCLUSIVE ACCESS persistent reservation on it. Then another initiator
> (e.g. on a different machine) sends a FU to the other LU which is
> honoured, wiping the whole device. One unhappy customer ....
>
> Doug Gilbert

Thanks, Doug. That is very clear, and I get it. We'll have a solution.

Best regards,
-Tim

-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Christoph Hellwig]

On Sat, Mar 31, 2018 at 01:03:46PM +0200, Hannes Reinecke wrote:
> Actually I would propose to have a 'management' LUN at LUN0, who could
> handle all the device-wide commands (eg things like START STOP UNIT,
> firmware update, or even SMART commands), and ignoring them for the
> remaining LUNs.

That is in fact the only workable option at all.  Everything else
completly breaks the scsi architecture.

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Tue, Apr 3, 2018 at 1:46 AM, Christoph Hellwig <hch@infradead.org> wrote:
> On Sat, Mar 31, 2018 at 01:03:46PM +0200, Hannes Reinecke wrote:
>> Actually I would propose to have a 'management' LUN at LUN0, who could
>> handle all the device-wide commands (eg things like START STOP UNIT,
>> firmware update, or even SMART commands), and ignoring them for the
>> remaining LUNs.
>
> That is in fact the only workable option at all.  Everything else
> completly breaks the scsi architecture.

Here's an update: Seagate will eliminate the inter-LU actions from
FORMAT UNIT and SANITIZE. Probably SANITIZE will be per-LUN, but
FORMAT UNIT is trickier due to internal drive architecture, and how
FORMAT UNIT initializes on-disk metadata. Likely it will require some
sort of synchronization across LUNs, such as the command being sent to
both LUNs sequentially or something similar. We are also considering
not supporting FORMAT UNIT at all - would anybody object? Any other
suggestions?

-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Douglas Gilbert]

On 2018-04-05 07:43 PM, Tim Walker wrote:
> On Tue, Apr 3, 2018 at 1:46 AM, Christoph Hellwig <hch@infradead.org> wrote:
>> On Sat, Mar 31, 2018 at 01:03:46PM +0200, Hannes Reinecke wrote:
>>> Actually I would propose to have a 'management' LUN at LUN0, who could
>>> handle all the device-wide commands (eg things like START STOP UNIT,
>>> firmware update, or even SMART commands), and ignoring them for the
>>> remaining LUNs.
>>
>> That is in fact the only workable option at all.  Everything else
>> completly breaks the scsi architecture.
> 
> Here's an update: Seagate will eliminate the inter-LU actions from
> FORMAT UNIT and SANITIZE. Probably SANITIZE will be per-LUN, but
> FORMAT UNIT is trickier due to internal drive architecture, and how
> FORMAT UNIT initializes on-disk metadata. Likely it will require some
> sort of synchronization across LUNs, such as the command being sent to
> both LUNs sequentially or something similar. We are also considering
> not supporting FORMAT UNIT at all - would anybody object? Any other
> suggestions?

Good, that is progress. [But you still only have one spindle.]

If Protection Information (PI) or changing the logical block size between
512 and 4096 bytes per block are options, then you need FU for that.
But does it need to take 900 minutes like one I got recently from S......?
Couldn't the actual reformatting of a track be deferred until the first
block written to that track?

Doug Gilbert

Re: Multi-Actuator SAS HDD First Look

[Hannes Reinecke]

On Thu, 5 Apr 2018 17:43:46 -0600
Tim Walker <tim.t.walker@seagate.com> wrote:

> On Tue, Apr 3, 2018 at 1:46 AM, Christoph Hellwig <hch@infradead.org>
> wrote:
> > On Sat, Mar 31, 2018 at 01:03:46PM +0200, Hannes Reinecke wrote:  
> >> Actually I would propose to have a 'management' LUN at LUN0, who
> >> could handle all the device-wide commands (eg things like START
> >> STOP UNIT, firmware update, or even SMART commands), and ignoring
> >> them for the remaining LUNs.  
> >
> > That is in fact the only workable option at all.  Everything else
> > completly breaks the scsi architecture.  
> 
> Here's an update: Seagate will eliminate the inter-LU actions from
> FORMAT UNIT and SANITIZE. Probably SANITIZE will be per-LUN, but
> FORMAT UNIT is trickier due to internal drive architecture, and how
> FORMAT UNIT initializes on-disk metadata. Likely it will require some
> sort of synchronization across LUNs, such as the command being sent to
> both LUNs sequentially or something similar. We are also considering
> not supporting FORMAT UNIT at all - would anybody object? Any other
> suggestions?
> 

Ah. Far better.
What about delegating FORMAT UNIT to the control LUN, and not
implementing it for the individual disk LUNs?
That would make an even stronger case for having a control LUN;
with that there wouldn't be any problem with having to synchronize
across LUNs etc.

Cheers,

Hannes

Re: Multi-Actuator SAS HDD First Look

[Christoph Hellwig]

On Fri, Apr 06, 2018 at 08:24:18AM +0200, Hannes Reinecke wrote:
> Ah. Far better.
> What about delegating FORMAT UNIT to the control LUN, and not
> implementing it for the individual disk LUNs?
> That would make an even stronger case for having a control LUN;
> with that there wouldn't be any problem with having to synchronize
> across LUNs etc.

It sounds to me like NVMe might be a much better model for this drive
than SCSI, btw :)

Re: Multi-Actuator SAS HDD First Look

[Douglas Gilbert]

On 2018-04-06 02:42 AM, Christoph Hellwig wrote:
> On Fri, Apr 06, 2018 at 08:24:18AM +0200, Hannes Reinecke wrote:
>> Ah. Far better.
>> What about delegating FORMAT UNIT to the control LUN, and not
>> implementing it for the individual disk LUNs?
>> That would make an even stronger case for having a control LUN;
>> with that there wouldn't be any problem with having to synchronize
>> across LUNs etc.
> 
> It sounds to me like NVMe might be a much better model for this drive
> than SCSI, btw :)

So you found a document that outlines NVMe's architecture! Could you
share the url (no marketing BS, please)?


And a serious question ... How would you map NVMe's (in Linux)
subsystem number, controller device minor number, CNTLID field
(Identify ctl response) and namespace id onto the SCSI subsystem's
h:c:t:l ?

Doug Gilbert

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Fri, Apr 6, 2018 at 11:09 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>
> On 2018-04-06 02:42 AM, Christoph Hellwig wrote:
>>
>> On Fri, Apr 06, 2018 at 08:24:18AM +0200, Hannes Reinecke wrote:
>>>
>>> Ah. Far better.
>>> What about delegating FORMAT UNIT to the control LUN, and not
>>> implementing it for the individual disk LUNs?
>>> That would make an even stronger case for having a control LUN;
>>> with that there wouldn't be any problem with having to synchronize
>>> across LUNs etc.
>>
>>
>> It sounds to me like NVMe might be a much better model for this drive
>> than SCSI, btw :)
>
>
> So you found a document that outlines NVMe's architecture! Could you
> share the url (no marketing BS, please)?
>
>
> And a serious question ... How would you map NVMe's (in Linux)
> subsystem number, controller device minor number, CNTLID field
> (Identify ctl response) and namespace id onto the SCSI subsystem's
> h:c:t:l ?
>
> Doug Gilbert
>

Hannes- yes, a drive system altering operation like FORMAT UNIT is
asking for a dedicated management port, as the NVMe folks apparently
felt. But what is the least painful endpoint type for LUN0?


-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Hannes Reinecke]

On 04/09/2018 04:08 AM, Tim Walker wrote:
> On Fri, Apr 6, 2018 at 11:09 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>>
>> On 2018-04-06 02:42 AM, Christoph Hellwig wrote:
>>>
>>> On Fri, Apr 06, 2018 at 08:24:18AM +0200, Hannes Reinecke wrote:
>>>>
>>>> Ah. Far better.
>>>> What about delegating FORMAT UNIT to the control LUN, and not
>>>> implementing it for the individual disk LUNs?
>>>> That would make an even stronger case for having a control LUN;
>>>> with that there wouldn't be any problem with having to synchronize
>>>> across LUNs etc.
>>>
>>>
>>> It sounds to me like NVMe might be a much better model for this drive
>>> than SCSI, btw :)
>>
>>
>> So you found a document that outlines NVMe's architecture! Could you
>> share the url (no marketing BS, please)?
>>
>>
>> And a serious question ... How would you map NVMe's (in Linux)
>> subsystem number, controller device minor number, CNTLID field
>> (Identify ctl response) and namespace id onto the SCSI subsystem's
>> h:c:t:l ?
>>
>> Doug Gilbert
>>
> 
> Hannes- yes, a drive system altering operation like FORMAT UNIT is
> asking for a dedicated management port, as the NVMe folks apparently
> felt. But what is the least painful endpoint type for LUN0?
> 
> 
I would probably use 'processor device' (ie type 3) as it's the least
defined, so you can do basically everything you like with it.
Possibly 'Enclosure Services' (type 0x0d) works, too, but then you have
to check with the SES spec if it allows the things you'd need.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		   Teamlead Storage & Networking
hare@suse.de			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: Multi-Actuator SAS HDD First Look

[Christoph Hellwig]

On Fri, Apr 06, 2018 at 01:09:08PM -0400, Douglas Gilbert wrote:
> So you found a document that outlines NVMe's architecture! Could you
> share the url (no marketing BS, please)?

You can always take a look at the actual spec:

http://nvmexpress.org/wp-content/uploads/NVM-Express-1_3a-20171024_ratified.pdf

But in summary: while in SCSI your Nexus for any command is with the
logic unit, in NVMe it is with the controller.   Many admin commands
operate on the whole controller.

> And a serious question ... How would you map NVMe's (in Linux)
> subsystem number, controller device minor number, CNTLID field
> (Identify ctl response) and namespace id onto the SCSI subsystem's
> h:c:t:l ?

I wouldn't because the scheme already doesn't make any sense for SCSI,
nevermind should we try to map NVMe into a scsi specific worldview.

> 
> Doug Gilbert
> 
---end quoted text---

Re: Multi-Actuator SAS HDD First Look

[Douglas Gilbert]

On 2018-04-09 02:17 AM, Hannes Reinecke wrote:
> On 04/09/2018 04:08 AM, Tim Walker wrote:
>> On Fri, Apr 6, 2018 at 11:09 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>>>
>>> On 2018-04-06 02:42 AM, Christoph Hellwig wrote:
>>>>
>>>> On Fri, Apr 06, 2018 at 08:24:18AM +0200, Hannes Reinecke wrote:
>>>>>
>>>>> Ah. Far better.
>>>>> What about delegating FORMAT UNIT to the control LUN, and not
>>>>> implementing it for the individual disk LUNs?
>>>>> That would make an even stronger case for having a control LUN;
>>>>> with that there wouldn't be any problem with having to synchronize
>>>>> across LUNs etc.
>>>>
>>>>
>>>> It sounds to me like NVMe might be a much better model for this drive
>>>> than SCSI, btw :)
>>>
>>>
>>> So you found a document that outlines NVMe's architecture! Could you
>>> share the url (no marketing BS, please)?
>>>
>>>
>>> And a serious question ... How would you map NVMe's (in Linux)
>>> subsystem number, controller device minor number, CNTLID field
>>> (Identify ctl response) and namespace id onto the SCSI subsystem's
>>> h:c:t:l ?
>>>
>>> Doug Gilbert
>>>
>>
>> Hannes- yes, a drive system altering operation like FORMAT UNIT is
>> asking for a dedicated management port, as the NVMe folks apparently
>> felt. But what is the least painful endpoint type for LUN0?
>>
>>
> I would probably use 'processor device' (ie type 3) as it's the least
> defined, so you can do basically everything you like with it.
> Possibly 'Enclosure Services' (type 0x0d) works, too, but then you have
> to check with the SES spec if it allows the things you'd need.

Processor device type (0x3) please. Then you are only required to support
the mandatory commands in SPC and that is not many. And then nothing
precludes you from implementing Start Stop Unit, Sanitize and/or Format
Unit on it. And as I pointed out earlier, you could even throw in a
copy manager (see SPC). Also as far as I know Linux, FreeBSD and Windows
will leave a Processor device type LU alone and just have a SCSI
pass-through device attached to it, and that is exactly what you want.
By default only root/administrator can open those pass-through devices.

If you chose SES type (0xd) then the Linux kernel ses driver (and the
FreeBSD equivalent) would attempt to attach to it before the user space
could countermand it (as things stand). And SES additionally makes the
SEND DIAGNOSTIC and RECEIVE DIAGNOSTIC RESULTS commands mandatory and
at least one diagnostic page (0x0) mandatory. If it doesn't supply
any other SES dpages then those two drivers are going to get pretty
confused (which would be a good test for them). Also it could get
confusing from an administration point of view. I'm guessing many of
these Multi-Actuator SAS HDDs will end up in big enclosures. And
those enclosures most likely would present a SES device. Multiple dummy
enclosures within a real enclosure will look strange (especially as
SES already has a concept of a sub-enclosure).

Doug Gilbert

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Mon, Apr 9, 2018 at 10:02 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>
> On 2018-04-09 02:17 AM, Hannes Reinecke wrote:
>>
>> On 04/09/2018 04:08 AM, Tim Walker wrote:
>>>
>>> On Fri, Apr 6, 2018 at 11:09 AM, Douglas Gilbert <dgilbert@interlog.com> wrote:
>>>>
>>>>
>>>> On 2018-04-06 02:42 AM, Christoph Hellwig wrote:
>>>>>
>>>>>
>>>>> On Fri, Apr 06, 2018 at 08:24:18AM +0200, Hannes Reinecke wrote:
>>>>>>
>>>>>>
>>>>>> Ah. Far better.
>>>>>> What about delegating FORMAT UNIT to the control LUN, and not
>>>>>> implementing it for the individual disk LUNs?
>>>>>> That would make an even stronger case for having a control LUN;
>>>>>> with that there wouldn't be any problem with having to synchronize
>>>>>> across LUNs etc.
>>>>>
>>>>>
>>>>>
>>>>> It sounds to me like NVMe might be a much better model for this drive
>>>>> than SCSI, btw :)
>>>>
>>>>
>>>>
>>>> So you found a document that outlines NVMe's architecture! Could you
>>>> share the url (no marketing BS, please)?
>>>>
>>>>
>>>> And a serious question ... How would you map NVMe's (in Linux)
>>>> subsystem number, controller device minor number, CNTLID field
>>>> (Identify ctl response) and namespace id onto the SCSI subsystem's
>>>> h:c:t:l ?
>>>>
>>>> Doug Gilbert
>>>>
>>>
>>> Hannes- yes, a drive system altering operation like FORMAT UNIT is
>>> asking for a dedicated management port, as the NVMe folks apparently
>>> felt. But what is the least painful endpoint type for LUN0?
>>>
>>>
>> I would probably use 'processor device' (ie type 3) as it's the least
>> defined, so you can do basically everything you like with it.
>> Possibly 'Enclosure Services' (type 0x0d) works, too, but then you have
>> to check with the SES spec if it allows the things you'd need.
>
>
> Processor device type (0x3) please. Then you are only required to support
> the mandatory commands in SPC and that is not many. And then nothing
> precludes you from implementing Start Stop Unit, Sanitize and/or Format
> Unit on it. And as I pointed out earlier, you could even throw in a
> copy manager (see SPC). Also as far as I know Linux, FreeBSD and Windows
> will leave a Processor device type LU alone and just have a SCSI
> pass-through device attached to it, and that is exactly what you want.
> By default only root/administrator can open those pass-through devices.
>
> If you chose SES type (0xd) then the Linux kernel ses driver (and the
> FreeBSD equivalent) would attempt to attach to it before the user space
> could countermand it (as things stand). And SES additionally makes the
> SEND DIAGNOSTIC and RECEIVE DIAGNOSTIC RESULTS commands mandatory and
> at least one diagnostic page (0x0) mandatory. If it doesn't supply
> any other SES dpages then those two drivers are going to get pretty
> confused (which would be a good test for them). Also it could get
> confusing from an administration point of view. I'm guessing many of
> these Multi-Actuator SAS HDDs will end up in big enclosures. And
> those enclosures most likely would present a SES device. Multiple dummy
> enclosures within a real enclosure will look strange (especially as
> SES already has a concept of a sub-enclosure).
>
> Doug Gilbert
>
>

I also believe the dual-actuator, or any significant HDD parallelism,
would map well onto an NVMe target, or NVMe-oF behind nvmet. Maybe a
lightweight virtual NVMe controller that would efficiently present the
HDD logs/mode pages/etc via the admin queue and the LUNs as fixed
namespaces...?

Doug, I will flesh your three LUN idea out some more and send it up
the flagpole over here. Thanks for the input.

I'd like to have a conversation at LSFMM and maybe pull together a
fairly well defined consensus recommendation. Is that possible? Can we
schedule it?

Best regards,
-Tim

Re: Multi-Actuator SAS HDD First Look

[Bart Van Assche]

T24gU3VuLCAyMDE4LTA0LTE1IGF0IDE5OjM1IC0wNjAwLCBUaW0gV2Fsa2VyIHdyb3RlOg0KPiBJ
IGFsc28gYmVsaWV2ZSB0aGUgZHVhbC1hY3R1YXRvciwgb3IgYW55IHNpZ25pZmljYW50IEhERCBw
YXJhbGxlbGlzbSwNCj4gd291bGQgbWFwIHdlbGwgb250byBhbiBOVk1lIHRhcmdldCwgb3IgTlZN
ZS1vRiBiZWhpbmQgbnZtZXQuIE1heWJlIGENCj4gbGlnaHR3ZWlnaHQgdmlydHVhbCBOVk1lIGNv
bnRyb2xsZXIgdGhhdCB3b3VsZCBlZmZpY2llbnRseSBwcmVzZW50IHRoZQ0KPiBIREQgbG9ncy9t
b2RlIHBhZ2VzL2V0YyB2aWEgdGhlIGFkbWluIHF1ZXVlIGFuZCB0aGUgTFVOcyBhcyBmaXhlZA0K
PiBuYW1lc3BhY2VzLi4uPw0KPiANCj4gRG91ZywgSSB3aWxsIGZsZXNoIHlvdXIgdGhyZWUgTFVO
IGlkZWEgb3V0IHNvbWUgbW9yZSBhbmQgc2VuZCBpdCB1cA0KPiB0aGUgZmxhZ3BvbGUgb3ZlciBo
ZXJlLiBUaGFua3MgZm9yIHRoZSBpbnB1dC4NCj4gDQo+IEknZCBsaWtlIHRvIGhhdmUgYSBjb252
ZXJzYXRpb24gYXQgTFNGTU0gYW5kIG1heWJlIHB1bGwgdG9nZXRoZXIgYQ0KPiBmYWlybHkgd2Vs
bCBkZWZpbmVkIGNvbnNlbnN1cyByZWNvbW1lbmRhdGlvbi4gSXMgdGhhdCBwb3NzaWJsZT8gQ2Fu
IHdlDQo+IHNjaGVkdWxlIGl0Pw0KDQpIZWxsbyBUaW0sDQoNCkkgdGhpbmsgdGhhdCB5b3UgaGF2
ZSB0byBzdWJtaXQgYSByZXF1ZXN0IHRvIHRoZSBMU0YvTU0gcHJvZ3JhbSBjb21taXR0ZWUNCihs
c2YtcGNAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcpIHRvIGFkZCBhbiBpdGVtIHRvIHRoZSBv
ZmZpY2lhbCBhZ2VuZGEuDQpJbiBjYXNlIHRoaXMgdG9waWMgd291bGRuJ3QgYmUgYWRkZWQgdG8g
dGhlIG9mZmljaWFsIGFnZW5kYSB3ZSBjYW4gc3RpbGwNCmRpc2N1c3MgdGhpcyB0b3BpYyBpbiBh
IG1lZXRpbmcgcm9vbSBhdCB0aGUgTFNGL01NIGxvY2F0aW9uLg0KDQpCYXJ0Lg0KDQoNCg0K

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Sun, Apr 15, 2018 at 10:31 PM, Bart Van Assche
<Bart.VanAssche@wdc.com> wrote:
> On Sun, 2018-04-15 at 19:35 -0600, Tim Walker wrote:
>> I also believe the dual-actuator, or any significant HDD parallelism,
>> would map well onto an NVMe target, or NVMe-oF behind nvmet. Maybe a
>> lightweight virtual NVMe controller that would efficiently present the
>> HDD logs/mode pages/etc via the admin queue and the LUNs as fixed
>> namespaces...?
>>
>> Doug, I will flesh your three LUN idea out some more and send it up
>> the flagpole over here. Thanks for the input.
>>
>> I'd like to have a conversation at LSFMM and maybe pull together a
>> fairly well defined consensus recommendation. Is that possible? Can we
>> schedule it?
>
> Hello Tim,
>
> I think that you have to submit a request to the LSF/MM program committee
> (lsf-pc@lists.linux-foundation.org) to add an item to the official agenda.
> In case this topic wouldn't be added to the official agenda we can still
> discuss this topic in a meeting room at the LSF/MM location.
>
> Bart.
>
>
>

Hello Bart-

It would be good if we could set up an informal meeting time and
location at LSFMM to discuss these dual actuator topics. so far you,
Doug, Hannes, and Christoph have expressed the most interest, plus
Damien. Can we set an hour aside one afternoon?

-Tim

-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770

Re: Multi-Actuator SAS HDD First Look

[Bart Van Assche]

T24gV2VkLCAyMDE4LTA0LTE4IGF0IDA1OjE2IC0wNjAwLCBUaW0gV2Fsa2VyIHdyb3RlOg0KPiBJ
dCB3b3VsZCBiZSBnb29kIGlmIHdlIGNvdWxkIHNldCB1cCBhbiBpbmZvcm1hbCBtZWV0aW5nIHRp
bWUgYW5kDQo+IGxvY2F0aW9uIGF0IExTRk1NIHRvIGRpc2N1c3MgdGhlc2UgZHVhbCBhY3R1YXRv
ciB0b3BpY3MuIHNvIGZhciB5b3UsDQo+IERvdWcsIEhhbm5lcywgYW5kIENocmlzdG9waCBoYXZl
IGV4cHJlc3NlZCB0aGUgbW9zdCBpbnRlcmVzdCwgcGx1cw0KPiBEYW1pZW4uIENhbiB3ZSBzZXQg
YW4gaG91ciBhc2lkZSBvbmUgYWZ0ZXJub29uPw0KDQpIZWxsbyBUaW0sDQoNCkhhZCB5b3Ugbm90
aWNlZCB0aGF0IHRoZSBvZmZpY2lhbCBMU0YvTU0gYWdlbmRhIG1lbnRpb25zIHRoZSBmb2xsb3dp
bmc6DQpUdWVzZGF5LCAxMSAtIDExOjMwIEFNOiBKYW1lcyBCb3JkZW4sIE11bHRpLWFjdHVhdG9y
IGRpc2sgZHJpdmVzLg0KDQpUaGFua3MsDQoNCkJhcnQuDQoNCg0KDQo=

Re: Multi-Actuator SAS HDD First Look

[Tim Walker]

On Wed, Apr 18, 2018 at 10:20 AM, Bart Van Assche
<Bart.VanAssche@wdc.com> wrote:
> On Wed, 2018-04-18 at 05:16 -0600, Tim Walker wrote:
>> It would be good if we could set up an informal meeting time and
>> location at LSFMM to discuss these dual actuator topics. so far you,
>> Doug, Hannes, and Christoph have expressed the most interest, plus
>> Damien. Can we set an hour aside one afternoon?
>
> Hello Tim,
>
> Had you noticed that the official LSF/MM agenda mentions the following:
> Tuesday, 11 - 11:30 AM: James Borden, Multi-actuator disk drives.
>
> Thanks,
>
> Bart.
>
>
>

No, I hadn't checked that, yet. James probably won't be there. I guess
that solves my problem. Thanks.

-- 
Tim Walker
Product Design Systems Engineering, Seagate Technology
(303) 775-3770