* Re: [dm-crypt] Alignment issue with 4K disk
@ 2016-01-10 17:22 Eugen Rogoza
2016-01-10 19:13 ` Sven Eschenberg
0 siblings, 1 reply; 18+ messages in thread
From: Eugen Rogoza @ 2016-01-10 17:22 UTC (permalink / raw)
To: dm-crypt
> My personal opinion is: The test is semantically wrong in different
> aspects. May I ask what drive you are using and what interface it uses?
>
> Regards
>
> -Sven
Hi Sven,
drive info below:
root@nuc:~> hdparm -I /dev/sdc
/dev/sdc:
ATA device, with non-removable media
Model Number: WDC WD60EZRX-00MVLB1
Serial Number: WD-WX41D94RNKAX
Firmware Revision: 80.00A80
Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x001f)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors:11721045168
Logical Sector size: 512 bytes
Physical Sector size: 4096 bytes
device size with M = 1024*1024: 5723166 MBytes
device size with M = 1000*1000: 6001175 MBytes (6001 GB)
cache/buffer size = unknown
Nominal Media Rotation Rate: 5700
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, with device specific minimum
R/W multiple sector transfer: Max = 16 Current = 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
Power-Up In Standby feature set
* SET_FEATURES required to spinup after power up
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* 64-bit World wide name
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Host-initiated interface power management
* Phy event counters
* NCQ priority information
* unknown 76[15]
DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
unknown 206[12] (vendor specific)
unknown 206[13] (vendor specific)
* DOWNLOAD MICROCODE DMA command
* WRITE BUFFER DMA command
* READ BUFFER DMA command
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
Logical Unit WWN Device Identifier: 50014ee20b2a3e40
NAA : 5
IEEE OUI : 0014ee
Unique ID : 20b2a3e40
Checksum: correct
It is an internal 3,5" SATA-drive put into a USB3-enclosure and connected via USB3 in UAS mode:
Jan 10 01:13:26 nuc kernel: usb 2-4: new SuperSpeed USB device number 2 using xhci_hcd
Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device found, idVendor=174c, idProduct=55aa
Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device strings: Mfr=2, Product=3, SerialNumber=1
Jan 10 01:13:26 nuc kernel: usb 2-4: Product: ASM1153E
Jan 10 01:13:26 nuc kernel: usb 2-4: Manufacturer: asmedia
Jan 10 01:13:26 nuc kernel: usb 2-4: SerialNumber: 1234567891C4
Jan 10 01:13:26 nuc kernel: scsi host4: uas
Jan 10 01:13:26 nuc kernel: usbcore: registered new interface driver uas
Jan 10 01:13:26 nuc kernel: scsi 4:0:0:0: Direct-Access ASMT 2115 0 PQ: 0 ANSI: 6
Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: Attached scsi generic sg2 type 0
Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: [sdc] Spinning up disk...
Jan 10 01:13:27 nuc kernel: ............ready
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 11721045168 512-byte logical blocks: (6.00 TB/5.45 TiB)
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 4096-byte physical blocks
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write Protect is off
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Mode Sense: 43 00 00 00
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 10 01:13:38 nuc kernel: sdc: sdc1 sdc2 sdc3 sdc4
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Attached SCSI disk
As a side note: sdc4 was a dm-crypt/LUKS-encrypted partition before. To my knowledge the device mapper warnings started to appear after migration to VeraCrypt.
Cheers,
Eugen
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-10 17:22 [dm-crypt] Alignment issue with 4K disk Eugen Rogoza
@ 2016-01-10 19:13 ` Sven Eschenberg
2016-01-10 20:20 ` Eugen Rogoza
0 siblings, 1 reply; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-10 19:13 UTC (permalink / raw)
To: dm-crypt
Hi Eugen,
Quoting a document on IO-Hintig:
'Storage vendors can also supply "I/O hints" about a device's preferred
minimum unit for random I/O ('minimum_io_size') and streaming I/O
('optimal_io_size'). For example, these hints may correspond to a RAID
device's chunk size and stripe size respectively.'
Of course a RAIDs layout parameters and preferred IO sizes are
semantically completely different things.
As for your case:
Ignore the warning. I think the optimal IO size as in 'preferred size
for sequential streaming IO' is indeed correct and must not necessarily
be a multiple of physical sector size. The optimal IO size is owed to
the transport layer (USB protocol) constraints, to max out the BUS
bandwidth.
Cutting it down to a simple example:
Consider each frame in the transport layer can hold 1.9 physical
sectors. Stuffing only 1 sector into the frame (to keep the multiple
physical sector constraint) will lead to a significant rise in number of
frames/packets and thus overhead. And I am not even talking about
transport layers with fixed frame size where you'll loose nearly 50% of
bandwidth and therefore transfer rate.
Anyway, in your case everything seems properly aligned. I tried to find
a way to influence 'optimal_io_size', could not find anything. Changing
the parameters via sysfs does not work, maybe there are IOCTLs and a
suiting utility...
Regards
-Sven
Am 10.01.2016 um 18:22 schrieb Eugen Rogoza:
>> My personal opinion is: The test is semantically wrong in different
>> aspects. May I ask what drive you are using and what interface it uses?
>>
>> Regards
>>
>> -Sven
>
> Hi Sven,
>
> drive info below:
>
>
> root@nuc:~> hdparm -I /dev/sdc
> /dev/sdc:
>
> ATA device, with non-removable media
> Model Number: WDC WD60EZRX-00MVLB1
> Serial Number: WD-WX41D94RNKAX
> Firmware Revision: 80.00A80
> Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
> Standards:
> Used: unknown (minor revision code 0x001f)
> Supported: 9 8 7 6 5
> Likely used: 9
> Configuration:
> Logical max current
> cylinders 16383 16383
> heads 16 16
> sectors/track 63 63
> --
> CHS current addressable sectors: 16514064
> LBA user addressable sectors: 268435455
> LBA48 user addressable sectors:11721045168
> Logical Sector size: 512 bytes
> Physical Sector size: 4096 bytes
> device size with M = 1024*1024: 5723166 MBytes
> device size with M = 1000*1000: 6001175 MBytes (6001 GB)
> cache/buffer size = unknown
> Nominal Media Rotation Rate: 5700
> Capabilities:
> LBA, IORDY(can be disabled)
> Queue depth: 32
> Standby timer values: spec'd by Standard, with device specific minimum
> R/W multiple sector transfer: Max = 16 Current = 0
> DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
> Cycle time: min=120ns recommended=120ns
> PIO: pio0 pio1 pio2 pio3 pio4
> Cycle time: no flow control=120ns IORDY flow control=120ns
> Commands/features:
> Enabled Supported:
> * SMART feature set
> Security Mode feature set
> * Power Management feature set
> * Write cache
> * Look-ahead
> * Host Protected Area feature set
> * WRITE_BUFFER command
> * READ_BUFFER command
> * NOP cmd
> * DOWNLOAD_MICROCODE
> Power-Up In Standby feature set
> * SET_FEATURES required to spinup after power up
> SET_MAX security extension
> * 48-bit Address feature set
> * Device Configuration Overlay feature set
> * Mandatory FLUSH_CACHE
> * FLUSH_CACHE_EXT
> * SMART error logging
> * SMART self-test
> * General Purpose Logging feature set
> * 64-bit World wide name
> * WRITE_UNCORRECTABLE_EXT command
> * {READ,WRITE}_DMA_EXT_GPL commands
> * Segmented DOWNLOAD_MICROCODE
> * Gen1 signaling speed (1.5Gb/s)
> * Gen2 signaling speed (3.0Gb/s)
> * Gen3 signaling speed (6.0Gb/s)
> * Native Command Queueing (NCQ)
> * Host-initiated interface power management
> * Phy event counters
> * NCQ priority information
> * unknown 76[15]
> DMA Setup Auto-Activate optimization
> Device-initiated interface power management
> * Software settings preservation
> * SMART Command Transport (SCT) feature set
> * SCT Write Same (AC2)
> * SCT Features Control (AC4)
> * SCT Data Tables (AC5)
> unknown 206[12] (vendor specific)
> unknown 206[13] (vendor specific)
> * DOWNLOAD MICROCODE DMA command
> * WRITE BUFFER DMA command
> * READ BUFFER DMA command
> Security:
> Master password revision code = 65534
> supported
> not enabled
> not locked
> not frozen
> not expired: security count
> supported: enhanced erase
> Logical Unit WWN Device Identifier: 50014ee20b2a3e40
> NAA : 5
> IEEE OUI : 0014ee
> Unique ID : 20b2a3e40
> Checksum: correct
>
>
> It is an internal 3,5" SATA-drive put into a USB3-enclosure and connected via USB3 in UAS mode:
>
>
> Jan 10 01:13:26 nuc kernel: usb 2-4: new SuperSpeed USB device number 2 using xhci_hcd
> Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device found, idVendor=174c, idProduct=55aa
> Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device strings: Mfr=2, Product=3, SerialNumber=1
> Jan 10 01:13:26 nuc kernel: usb 2-4: Product: ASM1153E
> Jan 10 01:13:26 nuc kernel: usb 2-4: Manufacturer: asmedia
> Jan 10 01:13:26 nuc kernel: usb 2-4: SerialNumber: 1234567891C4
> Jan 10 01:13:26 nuc kernel: scsi host4: uas
> Jan 10 01:13:26 nuc kernel: usbcore: registered new interface driver uas
> Jan 10 01:13:26 nuc kernel: scsi 4:0:0:0: Direct-Access ASMT 2115 0 PQ: 0 ANSI: 6
> Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: Attached scsi generic sg2 type 0
> Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: [sdc] Spinning up disk...
> Jan 10 01:13:27 nuc kernel: ............ready
> Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 11721045168 512-byte logical blocks: (6.00 TB/5.45 TiB)
> Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 4096-byte physical blocks
> Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write Protect is off
> Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Mode Sense: 43 00 00 00
> Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
> Jan 10 01:13:38 nuc kernel: sdc: sdc1 sdc2 sdc3 sdc4
> Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Attached SCSI disk
>
>
> As a side note: sdc4 was a dm-crypt/LUKS-encrypted partition before. To my knowledge the device mapper warnings started to appear after migration to VeraCrypt.
>
> Cheers,
>
> Eugen
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-10 19:13 ` Sven Eschenberg
@ 2016-01-10 20:20 ` Eugen Rogoza
2016-01-10 21:00 ` Sven Eschenberg
0 siblings, 1 reply; 18+ messages in thread
From: Eugen Rogoza @ 2016-01-10 20:20 UTC (permalink / raw)
To: dm-crypt
> Hi Eugen,
>
> Quoting a document on IO-Hintig:
>
> 'Storage vendors can also supply "I/O hints" about a device's preferred
> minimum unit for random I/O ('minimum_io_size') and streaming I/O
> ('optimal_io_size'). For example, these hints may correspond to a RAID
> device's chunk size and stripe size respectively.'
>
> Of course a RAIDs layout parameters and preferred IO sizes are
> semantically completely different things.
>
> As for your case:
> Ignore the warning. I think the optimal IO size as in 'preferred size
> for sequential streaming IO' is indeed correct and must not necessarily
> be a multiple of physical sector size. The optimal IO size is owed to
> the transport layer (USB protocol) constraints, to max out the BUS
> bandwidth.
>
> Cutting it down to a simple example:
> Consider each frame in the transport layer can hold 1.9 physical
> sectors. Stuffing only 1 sector into the frame (to keep the multiple
> physical sector constraint) will lead to a significant rise in number of
> frames/packets and thus overhead. And I am not even talking about
> transport layers with fixed frame size where you'll loose nearly 50% of
> bandwidth and therefore transfer rate.
>
> Anyway, in your case everything seems properly aligned. I tried to find
> a way to influence 'optimal_io_size', could not find anything. Changing
> the parameters via sysfs does not work, maybe there are IOCTLs and a
> suiting utility...
Hi Sven,
thanks for the insights. If I understand the explanation correctly (and put into simpler words), the optimal_io_size is reported by USB enclosure, not by the device itself, thus confusing the device mapper layer and causing lsbkl to show misalignment (as the dm expects optimal_io_size to be multiples of physical block size). At the same time the enclosure is supposed to reassemble the sectors from the transport frames into aligned reads/writes to the physical device, thus theoretically causing no performance degradation.
Anyway, my particular issue seems to be resolved. Thanks for that again. Although it doesn't explain why a previous LUKS-container on the same partition of the same drive connected the same way didn't throw any warnings (let me redo this test to be sure).
Just a suggestion: if DM stacking tests are currently considered to be implemented in an optimal way, I would at least appreciate an additional hint somewhere in the messages that the warnings could be due to a transport layer like USB sitting in front of the physical drive, and that they could be ignored in this case.
Cheers,
Eugen
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-10 20:20 ` Eugen Rogoza
@ 2016-01-10 21:00 ` Sven Eschenberg
0 siblings, 0 replies; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-10 21:00 UTC (permalink / raw)
To: dm-crypt
Hi Eugen,
Yes, You got it right, even though I cannot tell for sure if the
optimal_io_size is reported by the enclosure (bridge) or the usb 3.0
uas(p) kernel implementation. Afterall I am just a user ;-).
Anyhow, I think I read some slides from meetings, where voices arised,
that the IO-Hinting needs to be worked over - think about non rotational
disks, think about NVMe with huge queue-depths and and huge number of
queues and better parallelization possibilities.
I checked on the IOCTLs, there's only IOCTLs to get IO-Hinting values.
And I am a little curious about your findings on LUKS.
Regards
-Sven
Am 10.01.2016 um 21:20 schrieb Eugen Rogoza:
>> Hi Eugen,
>>
>> Quoting a document on IO-Hintig:
>>
>> 'Storage vendors can also supply "I/O hints" about a device's preferred
>> minimum unit for random I/O ('minimum_io_size') and streaming I/O
>> ('optimal_io_size'). For example, these hints may correspond to a RAID
>> device's chunk size and stripe size respectively.'
>>
>> Of course a RAIDs layout parameters and preferred IO sizes are
>> semantically completely different things.
>>
>> As for your case:
>> Ignore the warning. I think the optimal IO size as in 'preferred size
>> for sequential streaming IO' is indeed correct and must not necessarily
>> be a multiple of physical sector size. The optimal IO size is owed to
>> the transport layer (USB protocol) constraints, to max out the BUS
>> bandwidth.
>>
>> Cutting it down to a simple example:
>> Consider each frame in the transport layer can hold 1.9 physical
>> sectors. Stuffing only 1 sector into the frame (to keep the multiple
>> physical sector constraint) will lead to a significant rise in number of
>> frames/packets and thus overhead. And I am not even talking about
>> transport layers with fixed frame size where you'll loose nearly 50% of
>> bandwidth and therefore transfer rate.
>>
>> Anyway, in your case everything seems properly aligned. I tried to find
>> a way to influence 'optimal_io_size', could not find anything. Changing
>> the parameters via sysfs does not work, maybe there are IOCTLs and a
>> suiting utility...
>
> Hi Sven,
>
> thanks for the insights. If I understand the explanation correctly (and put into simpler words), the optimal_io_size is reported by USB enclosure, not by the device itself, thus confusing the device mapper layer and causing lsbkl to show misalignment (as the dm expects optimal_io_size to be multiples of physical block size). At the same time the enclosure is supposed to reassemble the sectors from the transport frames into aligned reads/writes to the physical device, thus theoretically causing no performance degradation.
>
> Anyway, my particular issue seems to be resolved. Thanks for that again. Although it doesn't explain why a previous LUKS-container on the same partition of the same drive connected the same way didn't throw any warnings (let me redo this test to be sure).
>
> Just a suggestion: if DM stacking tests are currently considered to be implemented in an optimal way, I would at least appreciate an additional hint somewhere in the messages that the warnings could be due to a transport layer like USB sitting in front of the physical drive, and that they could be ignored in this case.
>
> Cheers,
>
> Eugen
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
@ 2016-01-10 15:37 Eugen Rogoza
0 siblings, 0 replies; 18+ messages in thread
From: Eugen Rogoza @ 2016-01-10 15:37 UTC (permalink / raw)
To: dm-crypt
> My personal opinion is: The test is semantically wrong in different
> aspects. May I ask what drive you are using and what interface it uses?
>
> Regards
>
> -Sven
Hi Sven,
drive info below:
root@nuc:~> hdparm -I /dev/sdc
/dev/sdc:
ATA device, with non-removable media
Model Number: WDC WD60EZRX-00MVLB1
Serial Number: WD-WX41D94RNKAX
Firmware Revision: 80.00A80
Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x001f)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors:11721045168
Logical Sector size: 512 bytes
Physical Sector size: 4096 bytes
device size with M = 1024*1024: 5723166 MBytes
device size with M = 1000*1000: 6001175 MBytes (6001 GB)
cache/buffer size = unknown
Nominal Media Rotation Rate: 5700
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, with device specific minimum
R/W multiple sector transfer: Max = 16 Current = 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
Power-Up In Standby feature set
* SET_FEATURES required to spinup after power up
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* 64-bit World wide name
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Host-initiated interface power management
* Phy event counters
* NCQ priority information
* unknown 76[15]
DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
unknown 206[12] (vendor specific)
unknown 206[13] (vendor specific)
* DOWNLOAD MICROCODE DMA command
* WRITE BUFFER DMA command
* READ BUFFER DMA command
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
Logical Unit WWN Device Identifier: 50014ee20b2a3e40
NAA : 5
IEEE OUI : 0014ee
Unique ID : 20b2a3e40
Checksum: correct
It is an internal 3,5" SATA-drive put into a USB3-enclosure and connected via USB3 in UAS mode:
Jan 10 01:13:26 nuc kernel: usb 2-4: new SuperSpeed USB device number 2 using xhci_hcd
Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device found, idVendor=174c, idProduct=55aa
Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device strings: Mfr=2, Product=3, SerialNumber=1
Jan 10 01:13:26 nuc kernel: usb 2-4: Product: ASM1153E
Jan 10 01:13:26 nuc kernel: usb 2-4: Manufacturer: asmedia
Jan 10 01:13:26 nuc kernel: usb 2-4: SerialNumber: 1234567891C4
Jan 10 01:13:26 nuc kernel: scsi host4: uas
Jan 10 01:13:26 nuc kernel: usbcore: registered new interface driver uas
Jan 10 01:13:26 nuc kernel: scsi 4:0:0:0: Direct-Access ASMT 2115 0 PQ: 0 ANSI: 6
Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: Attached scsi generic sg2 type 0
Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: [sdc] Spinning up disk...
Jan 10 01:13:27 nuc kernel: ............ready
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 11721045168 512-byte logical blocks: (6.00 TB/5.45 TiB)
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 4096-byte physical blocks
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write Protect is off
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Mode Sense: 43 00 00 00
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 10 01:13:38 nuc kernel: sdc: sdc1 sdc2 sdc3 sdc4
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Attached SCSI disk
As a side note: sdc4 was a dm-crypt/LUKS-encrypted partition before. To my knowledge the device mapper warnings started to appear after migration to VeraCrypt.
Cheers,
Eugen
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [dm-crypt] Alignment issue with 4K disk
@ 2016-01-10 15:18 Eugen Rogoza
0 siblings, 0 replies; 18+ messages in thread
From: Eugen Rogoza @ 2016-01-10 15:18 UTC (permalink / raw)
To: dm-crypt
> My personal opinion is: The test is semantically wrong in different
> aspects. May I ask what drive you are using and what interface it uses?
>
> Regards
>
> -Sven
Hi Sven,
drive info below:
/dev/sdc:
ATA device, with non-removable media
Model Number: WDC WD60EZRX-00MVLB1
Serial Number: WD-WX41D94RNKAX
Firmware Revision: 80.00A80
Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x001f)
Supported: 9 8 7 6 5
Likely used: 9
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 268435455
LBA48 user addressable sectors:11721045168
Logical Sector size: 512 bytes
Physical Sector size: 4096 bytes
device size with M = 1024*1024: 5723166 MBytes
device size with M = 1000*1000: 6001175 MBytes (6001 GB)
cache/buffer size = unknown
Nominal Media Rotation Rate: 5700
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, with device specific minimum
R/W multiple sector transfer: Max = 16 Current = 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* NOP cmd
* DOWNLOAD_MICROCODE
Power-Up In Standby feature set
* SET_FEATURES required to spinup after power up
SET_MAX security extension
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
* General Purpose Logging feature set
* 64-bit World wide name
* WRITE_UNCORRECTABLE_EXT command
* {READ,WRITE}_DMA_EXT_GPL commands
* Segmented DOWNLOAD_MICROCODE
* Gen1 signaling speed (1.5Gb/s)
* Gen2 signaling speed (3.0Gb/s)
* Gen3 signaling speed (6.0Gb/s)
* Native Command Queueing (NCQ)
* Host-initiated interface power management
* Phy event counters
* NCQ priority information
* unknown 76[15]
DMA Setup Auto-Activate optimization
Device-initiated interface power management
* Software settings preservation
* SMART Command Transport (SCT) feature set
* SCT Write Same (AC2)
* SCT Features Control (AC4)
* SCT Data Tables (AC5)
unknown 206[12] (vendor specific)
unknown 206[13] (vendor specific)
* DOWNLOAD MICROCODE DMA command
* WRITE BUFFER DMA command
* READ BUFFER DMA command
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
Logical Unit WWN Device Identifier: 50014ee20b2a3e40
NAA : 5
IEEE OUI : 0014ee
Unique ID : 20b2a3e40
Checksum: correct
It is internal 3,5" SATA-drive put into a USB3-enclosure and connected via USB3 in UAS mode:
Jan 10 01:13:26 nuc kernel: usb 2-4: new SuperSpeed USB device number 2 using xhci_hcd
Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device found, idVendor=174c, idProduct=55aa
Jan 10 01:13:26 nuc kernel: usb 2-4: New USB device strings: Mfr=2, Product=3, SerialNumber=1
Jan 10 01:13:26 nuc kernel: usb 2-4: Product: ASM1153E
Jan 10 01:13:26 nuc kernel: usb 2-4: Manufacturer: asmedia
Jan 10 01:13:26 nuc kernel: usb 2-4: SerialNumber: 1234567891C4
Jan 10 01:13:26 nuc kernel: scsi host4: uas
Jan 10 01:13:26 nuc kernel: usbcore: registered new interface driver uas
Jan 10 01:13:26 nuc kernel: scsi 4:0:0:0: Direct-Access ASMT 2115 0 PQ: 0 ANSI: 6
Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: Attached scsi generic sg2 type 0
Jan 10 01:13:26 nuc kernel: sd 4:0:0:0: [sdc] Spinning up disk...
Jan 10 01:13:27 nuc kernel: ............ready
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 11721045168 512-byte logical blocks: (6.00 TB/5.45 TiB)
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] 4096-byte physical blocks
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write Protect is off
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Mode Sense: 43 00 00 00
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Jan 10 01:13:38 nuc kernel: sdc: sdc1 sdc2 sdc3 sdc4
Jan 10 01:13:38 nuc kernel: sd 4:0:0:0: [sdc] Attached SCSI disk
As a side note: sdc4 was a dm-crypt/LUKS-encrypted partition before. To my knowledge the device mapper warnings started to appear after migration to VeraCrypt.
Cheers,
Eugen
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [dm-crypt] Alignment issue with 4K disk
@ 2016-01-10 0:25 Eugen Rogoza
2016-01-10 1:14 ` Sven Eschenberg
0 siblings, 1 reply; 18+ messages in thread
From: Eugen Rogoza @ 2016-01-10 0:25 UTC (permalink / raw)
To: dm-crypt
Hi,
> Reporter should paste output of "lsblk -t" with device activated to check what underlying
> device report alignment limits are violated in storage stack).
I am the reporter and the output of 'lsblk -t' with encrypted partition mounted is below. sda2 is the one that works fine, sdc4 is the one that throws alignment warnings.
root@nuc:~> lsblk -t
NAME ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED RQ-SIZE RA WSAME
sda 0 4096 0 4096 512 1 cfq 128 128 0B
├─sda1 0 4096 0 4096 512 1 cfq 128 128 0B
└─sda2 0 4096 0 4096 512 1 cfq 128 128 0B
└─zuluCrypt-1001-NAAN-sda2-4071757814 0 4096 0 4096 512 1 128 128 0B
sdb 0 512 0 512 512 0 noop 128 128 0B
├─sdb1 0 512 0 512 512 0 noop 128 128 0B
├─sdb2 0 512 0 512 512 0 noop 128 128 0B
├─sdb3 0 512 0 512 512 0 noop 128 128 0B
└─sdb4 0 512 0 512 512 0 noop 128 128 0B
sdc 0 4096 33553920 4096 512 1 cfq 128 128 32M
├─sdc1 0 4096 33553920 4096 512 1 cfq 128 128 32M
├─sdc2 0 4096 33553920 4096 512 1 cfq 128 128 32M
├─sdc3 0 4096 33553920 4096 512 1 cfq 128 128 32M
└─sdc4 0 4096 33553920 4096 512 1 cfq 128 128 32M
└─zuluCrypt-1001-NAAN-sdc4-1790378342 -1 4096 0 4096 512 1 128 128 0B
> That is indeed true and further info including 'dmsetup table' will be
> necessary to tell what is really going on.
Output below. Again, sdc4 is the problematic one.
root@nuc:~> dmsetup table
zuluCrypt-1001-NAAN-sdc4-1790378342: 0 11406264975 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 256 8:36 256
zuluCrypt-1001-NAAN-sda2-4071757814: 0 1324371456 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 256 8:2 256
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-10 0:25 Eugen Rogoza
@ 2016-01-10 1:14 ` Sven Eschenberg
2016-01-10 4:30 ` Sven Eschenberg
0 siblings, 1 reply; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-10 1:14 UTC (permalink / raw)
To: dm-crypt
Hi Eugen,
Indeed the optimal IO value (on sdc) seems a little weird. Yet this
should not create any alignment warnings.
zuluCrypt-1001-NAAN-sdc4-1790378342: 0 11406264975 - Now this is not 4k
aligned (sectors must be divideable by 8 to be 4k aligned). Indeed the
encrypted payload should start at 11406264976 sectors, so there is a -1
sector misalignment (See lsblk output saying so). VeraCrypt did not
properly align the payload when the container was created, I assume.
Regards
-Sven
Am 10.01.2016 um 01:25 schrieb Eugen Rogoza:
> Hi,
>
>> Reporter should paste output of "lsblk -t" with device activated to check what underlying
>> device report alignment limits are violated in storage stack).
>
> I am the reporter and the output of 'lsblk -t' with encrypted partition mounted is below. sda2 is the one that works fine, sdc4 is the one that throws alignment warnings.
>
> root@nuc:~> lsblk -t
> NAME ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED RQ-SIZE RA WSAME
> sda 0 4096 0 4096 512 1 cfq 128 128 0B
> ├─sda1 0 4096 0 4096 512 1 cfq 128 128 0B
> └─sda2 0 4096 0 4096 512 1 cfq 128 128 0B
> └─zuluCrypt-1001-NAAN-sda2-4071757814 0 4096 0 4096 512 1 128 128 0B
> sdb 0 512 0 512 512 0 noop 128 128 0B
> ├─sdb1 0 512 0 512 512 0 noop 128 128 0B
> ├─sdb2 0 512 0 512 512 0 noop 128 128 0B
> ├─sdb3 0 512 0 512 512 0 noop 128 128 0B
> └─sdb4 0 512 0 512 512 0 noop 128 128 0B
> sdc 0 4096 33553920 4096 512 1 cfq 128 128 32M
> ├─sdc1 0 4096 33553920 4096 512 1 cfq 128 128 32M
> ├─sdc2 0 4096 33553920 4096 512 1 cfq 128 128 32M
> ├─sdc3 0 4096 33553920 4096 512 1 cfq 128 128 32M
> └─sdc4 0 4096 33553920 4096 512 1 cfq 128 128 32M
> └─zuluCrypt-1001-NAAN-sdc4-1790378342 -1 4096 0 4096 512 1 128 128 0B
>
>
>
>> That is indeed true and further info including 'dmsetup table' will be
>> necessary to tell what is really going on.
>
> Output below. Again, sdc4 is the problematic one.
>
> root@nuc:~> dmsetup table
> zuluCrypt-1001-NAAN-sdc4-1790378342: 0 11406264975 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 256 8:36 256
> zuluCrypt-1001-NAAN-sda2-4071757814: 0 1324371456 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 256 8:2 256
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-10 1:14 ` Sven Eschenberg
@ 2016-01-10 4:30 ` Sven Eschenberg
0 siblings, 0 replies; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-10 4:30 UTC (permalink / raw)
To: dm-crypt
Hi Eugen,
Ignore my previous mail, don't ask me why, but I looked at the size of
the mapping. The offset is actually 256 sectors which is fine and should
not be misaligned. VeraCrypt uses 128KByte as offset, which indeed is
256 sectors and should be 4K aligned if the partition is.
The output of lsblk suggests that the partition itself is aligned (and
the output from fdisk seems to support this).
Anyhow, it is the device mapper, in dm-table.c that claims the
misalignment. Basicly geometry and io limits are compared and checked
during stacking one of the tests is:
-- Optimal I/O a multiple of the physical block size?
Which is triggered (look at the optimal IO size.
My personal opinion is: The test is semantically wrong in different
aspects. May I ask what drive you are using and what interface it uses?
Regards
-Sven
Am 10.01.2016 um 02:14 schrieb Sven Eschenberg:
> Hi Eugen,
>
> Indeed the optimal IO value (on sdc) seems a little weird. Yet this
> should not create any alignment warnings.
>
> zuluCrypt-1001-NAAN-sdc4-1790378342: 0 11406264975 - Now this is not 4k
> aligned (sectors must be divideable by 8 to be 4k aligned). Indeed the
> encrypted payload should start at 11406264976 sectors, so there is a -1
> sector misalignment (See lsblk output saying so). VeraCrypt did not
> properly align the payload when the container was created, I assume.
>
> Regards
>
> -Sven
>
>
> Am 10.01.2016 um 01:25 schrieb Eugen Rogoza:
>> Hi,
>>
>>> Reporter should paste output of "lsblk -t" with device activated to
>>> check what underlying
>>> device report alignment limits are violated in storage stack).
>>
>> I am the reporter and the output of 'lsblk -t' with encrypted
>> partition mounted is below. sda2 is the one that works fine, sdc4 is
>> the one that throws alignment warnings.
>>
>> root@nuc:~> lsblk -t
>> NAME ALIGNMENT MIN-IO OPT-IO
>> PHY-SEC LOG-SEC ROTA SCHED RQ-SIZE RA WSAME
>> sda 0 4096 0
>> 4096 512 1 cfq 128 128 0B
>> ├─sda1 0 4096 0
>> 4096 512 1 cfq 128 128 0B
>> └─sda2 0 4096 0
>> 4096 512 1 cfq 128 128 0B
>> └─zuluCrypt-1001-NAAN-sda2-4071757814 0 4096 0
>> 4096 512 1 128 128 0B
>> sdb 0 512 0
>> 512 512 0 noop 128 128 0B
>> ├─sdb1 0 512 0
>> 512 512 0 noop 128 128 0B
>> ├─sdb2 0 512 0
>> 512 512 0 noop 128 128 0B
>> ├─sdb3 0 512 0
>> 512 512 0 noop 128 128 0B
>> └─sdb4 0 512 0
>> 512 512 0 noop 128 128 0B
>> sdc 0 4096 33553920
>> 4096 512 1 cfq 128 128 32M
>> ├─sdc1 0 4096 33553920
>> 4096 512 1 cfq 128 128 32M
>> ├─sdc2 0 4096 33553920
>> 4096 512 1 cfq 128 128 32M
>> ├─sdc3 0 4096 33553920
>> 4096 512 1 cfq 128 128 32M
>> └─sdc4 0 4096 33553920
>> 4096 512 1 cfq 128 128 32M
>> └─zuluCrypt-1001-NAAN-sdc4-1790378342 -1 4096 0
>> 4096 512 1 128 128 0B
>>
>>
>>
>>> That is indeed true and further info including 'dmsetup table' will be
>>> necessary to tell what is really going on.
>>
>> Output below. Again, sdc4 is the problematic one.
>>
>> root@nuc:~> dmsetup table
>> zuluCrypt-1001-NAAN-sdc4-1790378342: 0 11406264975 crypt
>> aes-xts-plain64
>> 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
>> 256 8:36 256
>> zuluCrypt-1001-NAAN-sda2-4071757814: 0 1324371456 crypt
>> aes-xts-plain64
>> 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
>> 256 8:2 256
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
>>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
^ permalink raw reply [flat|nested] 18+ messages in thread
* [dm-crypt] Alignment issue with 4K disk
@ 2016-01-04 21:31 Yves-Alexis Perez
2016-01-04 21:54 ` Sven Eschenberg
0 siblings, 1 reply; 18+ messages in thread
From: Yves-Alexis Perez @ 2016-01-04 21:31 UTC (permalink / raw)
To: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 705 bytes --]
Hi,
I'm trying to setup an encrypted volume on an external disk which has 4k
physical sector size (with 512b logical sector size).
I've used various parted option (-a none / minimal / optimal) and various --
align-payload options to cryptsetup, but everytime I luksOpen the container, I
get a kernel log with:
janv. 04 22:25:04 scapa kernel: device-mapper: table: 253:4: adding target device sdb1 caused an alignment inconsistency: physical_block_size=4096, logical_block_size=512, alignment_offset=0, start=4194304
(argument for start= changes for different values of the --align-payload)
I'm not sure it's really serious, but any idea how to fix that?
Regards,
--
Yves-Alexis
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-04 21:31 Yves-Alexis Perez
@ 2016-01-04 21:54 ` Sven Eschenberg
2016-01-04 22:20 ` Yves-Alexis Perez
0 siblings, 1 reply; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-04 21:54 UTC (permalink / raw)
To: dm-crypt
Hi,
start=4194304 -> assumind this is in 512 byte blocks, the start sector
is aligned to MByte boundary and thus 4k boundary.
Anyway, if your device only has one partition which is encrypted with
dm-crypt, set it up to start at sector 2048 (which should be default for
parted, fdisk and so on). Pass align-payload 2048 during creation of the
LUKS header, which will give you an MByte boundary and should always be
okay.
If the kernel still complains, then something else might be going wrong.
Regards
-Sven
Am 04.01.2016 um 22:31 schrieb Yves-Alexis Perez:
> Hi,
>
> I'm trying to setup an encrypted volume on an external disk which has 4k
> physical sector size (with 512b logical sector size).
>
> I've used various parted option (-a none / minimal / optimal) and various --
> align-payload options to cryptsetup, but everytime I luksOpen the container, I
> get a kernel log with:
>
> janv. 04 22:25:04 scapa kernel: device-mapper: table: 253:4: adding target device sdb1 caused an alignment inconsistency: physical_block_size=4096, logical_block_size=512, alignment_offset=0, start=4194304
>
> (argument for start= changes for different values of the --align-payload)
>
> I'm not sure it's really serious, but any idea how to fix that?
>
> Regards,
>
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-04 21:54 ` Sven Eschenberg
@ 2016-01-04 22:20 ` Yves-Alexis Perez
2016-01-04 22:41 ` Sven Eschenberg
2016-01-04 23:02 ` Sven Eschenberg
0 siblings, 2 replies; 18+ messages in thread
From: Yves-Alexis Perez @ 2016-01-04 22:20 UTC (permalink / raw)
To: Sven Eschenberg, dm-crypt
[-- Attachment #1: Type: text/plain, Size: 2642 bytes --]
On lun., 2016-01-04 at 22:54 +0100, Sven Eschenberg wrote:
> Hi,
>
> start=4194304 -> assumind this is in 512 byte blocks, the start sector
> is aligned to MByte boundary and thus 4k boundary.
>
> Anyway, if your device only has one partition which is encrypted with
> dm-crypt, set it up to start at sector 2048 (which should be default for
> parted, fdisk and so on). Pass align-payload 2048 during creation of the
> LUKS header, which will give you an MByte boundary and should always be
> okay.
>
> If the kernel still complains, then something else might be going wrong.
Right now I have:
root@scapa:~# lsblk -o +ALIGNMENT /dev/sdc
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT ALIGNMENT
sdc 8:32 0 931.5G 0 disk 0
└─sdc1 8:33 0 931.5G 0 part 512
└─scapa-backup 253:4 0 931.4G 0 crypt /mnt/scapa-backup -1
And:
scapa-backup: 0 1953336209 crypt aes-xts-plain64
0000000000000000000000000000000000000000000000000000000000000000 0 8:33 65536
root@scapa:~# fdisk -l /dev/sdc
Disk /dev/sdc: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
Disklabel type: gpt
Disk identifier: EBA98857-89BA-4B2C-8366-4A77A527A8E0
Device Start End Sectors Size Type
/dev/sdc1 65535 1953467279 1953401745 931.5G Linux filesystem
Partition 1 does not start on physical sector boundary.
[I'm a bit concerned with that, but since I'm using a GPT and not MBR it might
be related]
root@scapa:~# gdisk -l /dev/sdc
GPT fdisk (gdisk) version 1.0.1
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sdc: 1953525168 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): EBA98857-89BA-4B2C-8366-4A77A527A8E0
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 8-sector boundaries
Total free space is 123356 sectors (60.2 MiB)
Number Start (sector) End (sector) Size Code Name
1 65535 1953467279 931.5 GiB 8300 scapa-backup
Regards,
--
Yves-Alexis
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-04 22:20 ` Yves-Alexis Perez
@ 2016-01-04 22:41 ` Sven Eschenberg
2016-01-04 23:02 ` Sven Eschenberg
1 sibling, 0 replies; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-04 22:41 UTC (permalink / raw)
To: dm-crypt
The Startsector of the Partition should NEVER be odd, rather even. 65536
would be fine, that's why lsblk gives an alignment of -1 for
scapa-backup I guess.
It does not matter if you use MBR or GPT, well, not too much for this
matter. The GPT is 34 LBA blocks (of 512 bytes). It can possibly be
longer to host more than 128 Partition entries, but no way close to 2048
which would resemble 1MByte. So 2048s would be a good place to start
your partition. It is aligned to 1 MByte, as well as 4KByte ... it even
is aligned to all major SSD Page-Sizes and SSD Erase-Block-Sizes.
Anyhow you either need to align the start of your partition or you'll
have an extremly hard time aligning the dm-crypt offset manually at the
right place.
Regards
-Sven
P.S.: If you still have a chance to change things, better do it now!
Am 04.01.2016 um 23:20 schrieb Yves-Alexis Perez:
> On lun., 2016-01-04 at 22:54 +0100, Sven Eschenberg wrote:
>> Hi,
>>
>> start=4194304 -> assumind this is in 512 byte blocks, the start sector
>> is aligned to MByte boundary and thus 4k boundary.
>>
>> Anyway, if your device only has one partition which is encrypted with
>> dm-crypt, set it up to start at sector 2048 (which should be default for
>> parted, fdisk and so on). Pass align-payload 2048 during creation of the
>> LUKS header, which will give you an MByte boundary and should always be
>> okay.
>>
>> If the kernel still complains, then something else might be going wrong.
>
> Right now I have:
>
> root@scapa:~# lsblk -o +ALIGNMENT /dev/sdc
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT ALIGNMENT
> sdc 8:32 0 931.5G 0 disk 0
> └─sdc1 8:33 0 931.5G 0 part 512
> └─scapa-backup 253:4 0 931.4G 0 crypt /mnt/scapa-backup -1
>
> And:
>
> scapa-backup: 0 1953336209 crypt aes-xts-plain64
> 0000000000000000000000000000000000000000000000000000000000000000 0 8:33 65536
>
> root@scapa:~# fdisk -l /dev/sdc
> Disk /dev/sdc: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
> Disklabel type: gpt
> Disk identifier: EBA98857-89BA-4B2C-8366-4A77A527A8E0
>
> Device Start End Sectors Size Type
> /dev/sdc1 65535 1953467279 1953401745 931.5G Linux filesystem
>
> Partition 1 does not start on physical sector boundary.
>
> [I'm a bit concerned with that, but since I'm using a GPT and not MBR it might
> be related]
>
> root@scapa:~# gdisk -l /dev/sdc
> GPT fdisk (gdisk) version 1.0.1
>
> Partition table scan:
> MBR: protective
> BSD: not present
> APM: not present
> GPT: present
>
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdc: 1953525168 sectors, 931.5 GiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): EBA98857-89BA-4B2C-8366-4A77A527A8E0
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 1953525134
> Partitions will be aligned on 8-sector boundaries
> Total free space is 123356 sectors (60.2 MiB)
>
> Number Start (sector) End (sector) Size Code Name
> 1 65535 1953467279 931.5 GiB 8300 scapa-backup
>
> Regards,
>
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-04 22:20 ` Yves-Alexis Perez
2016-01-04 22:41 ` Sven Eschenberg
@ 2016-01-04 23:02 ` Sven Eschenberg
2016-01-05 10:14 ` Yves-Alexis Perez
1 sibling, 1 reply; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-04 23:02 UTC (permalink / raw)
To: dm-crypt
Hi Alexis,
I stumbled over this line:
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
The latter resembles 65535 sectors (512 Byte logical ones), which
explains why the automated alignment was done at sector 65535. This
value seems extremely weird though.
You said the drive was external, connected how? USB maybe?
Can you take a look at hdparm -I /dev/sdc? Anything unusual there?
Regards
-Sven
Am 04.01.2016 um 23:20 schrieb Yves-Alexis Perez:
> On lun., 2016-01-04 at 22:54 +0100, Sven Eschenberg wrote:
>> Hi,
>>
>> start=4194304 -> assumind this is in 512 byte blocks, the start sector
>> is aligned to MByte boundary and thus 4k boundary.
>>
>> Anyway, if your device only has one partition which is encrypted with
>> dm-crypt, set it up to start at sector 2048 (which should be default for
>> parted, fdisk and so on). Pass align-payload 2048 during creation of the
>> LUKS header, which will give you an MByte boundary and should always be
>> okay.
>>
>> If the kernel still complains, then something else might be going wrong.
>
> Right now I have:
>
> root@scapa:~# lsblk -o +ALIGNMENT /dev/sdc
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT ALIGNMENT
> sdc 8:32 0 931.5G 0 disk 0
> └─sdc1 8:33 0 931.5G 0 part 512
> └─scapa-backup 253:4 0 931.4G 0 crypt /mnt/scapa-backup -1
>
> And:
>
> scapa-backup: 0 1953336209 crypt aes-xts-plain64
> 0000000000000000000000000000000000000000000000000000000000000000 0 8:33 65536
>
> root@scapa:~# fdisk -l /dev/sdc
> Disk /dev/sdc: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
> Disklabel type: gpt
> Disk identifier: EBA98857-89BA-4B2C-8366-4A77A527A8E0
>
> Device Start End Sectors Size Type
> /dev/sdc1 65535 1953467279 1953401745 931.5G Linux filesystem
>
> Partition 1 does not start on physical sector boundary.
>
> [I'm a bit concerned with that, but since I'm using a GPT and not MBR it might
> be related]
>
> root@scapa:~# gdisk -l /dev/sdc
> GPT fdisk (gdisk) version 1.0.1
>
> Partition table scan:
> MBR: protective
> BSD: not present
> APM: not present
> GPT: present
>
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sdc: 1953525168 sectors, 931.5 GiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): EBA98857-89BA-4B2C-8366-4A77A527A8E0
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 1953525134
> Partitions will be aligned on 8-sector boundaries
> Total free space is 123356 sectors (60.2 MiB)
>
> Number Start (sector) End (sector) Size Code Name
> 1 65535 1953467279 931.5 GiB 8300 scapa-backup
>
> Regards,
>
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-04 23:02 ` Sven Eschenberg
@ 2016-01-05 10:14 ` Yves-Alexis Perez
2016-01-07 18:00 ` .. ink ..
0 siblings, 1 reply; 18+ messages in thread
From: Yves-Alexis Perez @ 2016-01-05 10:14 UTC (permalink / raw)
To: Sven Eschenberg, dm-crypt
[-- Attachment #1: Type: text/plain, Size: 829 bytes --]
On mar., 2016-01-05 at 00:02 +0100, Sven Eschenberg wrote:
> Hi Alexis,
(Yves-Alexis actually)
>
> I stumbled over this line:
>
> I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
>
> The latter resembles 65535 sectors (512 Byte logical ones), which
> explains why the automated alignment was done at sector 65535. This
> value seems extremely weird though.
>
> You said the drive was external, connected how? USB maybe?
Yes. It's a Samsung T3 disk, so everything is integrated (disk+sata/usb3
adapter), not much chance I can do anything here.
I'll try to force the alignment to 2048s for part1 and see what happens.
>
> Can you take a look at hdparm -I /dev/sdc? Anything unusual there?
I'll try tonight (I don't have it at hand).
Thanks for the pointers though.
--
Yves-Alexis
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-05 10:14 ` Yves-Alexis Perez
@ 2016-01-07 18:00 ` .. ink ..
2016-01-09 12:30 ` Milan Broz
0 siblings, 1 reply; 18+ messages in thread
From: .. ink .. @ 2016-01-07 18:00 UTC (permalink / raw)
To: dm-crypt
There is another (i think) related problem that was reported here[1]
Are these warnings serious or can they be ignored?
[1] https://github.com/mhogomchungu/zuluCrypt/issues/25
On Tue, Jan 5, 2016 at 1:14 PM, Yves-Alexis Perez <corsac@debian.org> wrote:
> On mar., 2016-01-05 at 00:02 +0100, Sven Eschenberg wrote:
>> Hi Alexis,
>
> (Yves-Alexis actually)
>>
>> I stumbled over this line:
>>
>> I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
>>
>> The latter resembles 65535 sectors (512 Byte logical ones), which
>> explains why the automated alignment was done at sector 65535. This
>> value seems extremely weird though.
>>
>> You said the drive was external, connected how? USB maybe?
>
> Yes. It's a Samsung T3 disk, so everything is integrated (disk+sata/usb3
> adapter), not much chance I can do anything here.
>
> I'll try to force the alignment to 2048s for part1 and see what happens.
>>
>> Can you take a look at hdparm -I /dev/sdc? Anything unusual there?
>
> I'll try tonight (I don't have it at hand).
>
> Thanks for the pointers though.
> --
> Yves-Alexis
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-07 18:00 ` .. ink ..
@ 2016-01-09 12:30 ` Milan Broz
2016-01-09 14:47 ` Sven Eschenberg
0 siblings, 1 reply; 18+ messages in thread
From: Milan Broz @ 2016-01-09 12:30 UTC (permalink / raw)
To: .. ink .., dm-crypt
On 01/07/2016 07:00 PM, .. ink .. wrote:
> There is another (i think) related problem that was reported here[1]
>
> Are these warnings serious or can they be ignored?
>
>
> [1] https://github.com/mhogomchungu/zuluCrypt/issues/25
If I read this correctly, it s caused by mapping of an unaligned plaintext device-mapper
device (unlocked VeraCrypt) to underlying partition.
That's a bug (or lack of proper alignment) inside VeraCrypt header or in GPT partitioning
- it should respect underlying device alignment while creating device/partition.
Anyway, I think the only consequence should be performance drop, it should still work.
Note that output from GPT says 512B while dm kernel message says 4k block... GPT should
use 4k block as well.
Reporter should paste output of "lsblk -t" with device activated to check what underlying
device report alignment limits are violated in storage stack).
We cannot do anything with it I guess, cryptsetup just activates what is written inside
Veracrypt header using alignment limits reported by kernel.
Milan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dm-crypt] Alignment issue with 4K disk
2016-01-09 12:30 ` Milan Broz
@ 2016-01-09 14:47 ` Sven Eschenberg
0 siblings, 0 replies; 18+ messages in thread
From: Sven Eschenberg @ 2016-01-09 14:47 UTC (permalink / raw)
To: dm-crypt
Hi Milan,
Am 09.01.2016 um 13:30 schrieb Milan Broz:
>
>
> On 01/07/2016 07:00 PM, .. ink .. wrote:
>> There is another (i think) related problem that was reported here[1]
>>
>> Are these warnings serious or can they be ignored?
>>
>>
>> [1] https://github.com/mhogomchungu/zuluCrypt/issues/25
>
> If I read this correctly, it s caused by mapping of an unaligned plaintext device-mapper
> device (unlocked VeraCrypt) to underlying partition.
Looks like it...
>
> That's a bug (or lack of proper alignment) inside VeraCrypt header or in GPT partitioning
> - it should respect underlying device alignment while creating device/partition.
>
> Anyway, I think the only consequence should be performance drop, it should still work.
>
> Note that output from GPT says 512B while dm kernel message says 4k block... GPT should
> use 4k block as well.
> Reporter should paste output of "lsblk -t" with device activated to check what underlying
> device report alignment limits are violated in storage stack).
GPT fdisk claims 512 byte logical blocksize, which seems legit for a
512e drive. It even says that partitions should be aligned to 2048
sectors (which is MByte boundaries and 4k aligned). This seems all
legit, as the GPT demands to be LBA aligned and 512e drives do indeed
use a 512 byte sector addressing.
Even start=131072 is a multiple of 2048 and thus MByte aligned - It does
indeed look a little odd.
>
> We cannot do anything with it I guess, cryptsetup just activates what is written inside
> Veracrypt header using alignment limits reported by kernel.
That is indeed true and further info including 'dmsetup table' will be
necessary to tell what is really going on.
>
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
Regards
-Sven
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2016-01-10 21:00 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-10 17:22 [dm-crypt] Alignment issue with 4K disk Eugen Rogoza
2016-01-10 19:13 ` Sven Eschenberg
2016-01-10 20:20 ` Eugen Rogoza
2016-01-10 21:00 ` Sven Eschenberg
-- strict thread matches above, loose matches on Subject: below --
2016-01-10 15:37 Eugen Rogoza
2016-01-10 15:18 Eugen Rogoza
2016-01-10 0:25 Eugen Rogoza
2016-01-10 1:14 ` Sven Eschenberg
2016-01-10 4:30 ` Sven Eschenberg
2016-01-04 21:31 Yves-Alexis Perez
2016-01-04 21:54 ` Sven Eschenberg
2016-01-04 22:20 ` Yves-Alexis Perez
2016-01-04 22:41 ` Sven Eschenberg
2016-01-04 23:02 ` Sven Eschenberg
2016-01-05 10:14 ` Yves-Alexis Perez
2016-01-07 18:00 ` .. ink ..
2016-01-09 12:30 ` Milan Broz
2016-01-09 14:47 ` Sven Eschenberg
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.