[dm-crypt] dm-crypt LUKS and USB power management

[dm-crypt] dm-crypt LUKS and USB power management

[Axel Heider]

Hi,

I have a encrypted USB disk at /dev/sda that worked nicely so
far. However, recently some new (?) power management seems to
break this. Disk spin down and power up still work, but at some
point the USB disk disconnects. Searching what happens here, I
found that
https://www.kernel.org/doc/Documentation/usb/power-management.txt
claims it's not uncommon that USB devices don't support power
management properly and thus do a disconnect and reconnect there.
Seem this is happening as dmesg reports:

  [    5.459493] usb 1-1.2: new high-speed USB device number 3 using dwc_otg
  [    5.459493] usb-storage 1-1.2:1.0: USB Mass Storage device detected
  [    5.561081] scsi0 : usb-storage 1-1.2:1.0
  [    5.561081] scsi 0:0:0:0: Direct-Access Disk Name 2BC1 PQ: 0 ANSI: 2 CCS
  [    5.562474] sd 0:0:0:0: [sda] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
  ....
  [ 1659.691793] usb 1-1.2: USB disconnect, device number 3
  [ 1659.699144] scsi 0:0:0:0: rejecting I/O to offline device
  [ 1659.699215] scsi 0:0:0:0: [sda] killing request
  [ 1659.703914] scsi 0:0:0:0: [sda] Unhandled error code
  [ 1659.708993] scsi 0:0:0:0: [sda]
  [ 1659.712500] Result: hostbyte=0x01 driverbyte=0x00
  [ 1659.717332] scsi 0:0:0:0: [sda] CDB:
  [ 1659.721162] cdb[0]=0x28: 28 00 00 04 31 7a 00 00 08 00
  [ 1659.726443] end_request: I/O error, dev sda, sector 274810
  [ 1663.759509] usb 1-1.2: new high-speed USB device number 4 using dwc_otg
  [ 1663.865049] usb-storage 1-1.2:1.0: USB Mass Storage device detected
  [ 1663.866221] scsi1 : usb-storage 1-1.2:1.0
  [ 1664.869823] scsi 1:0:0:0: Direct-Access Disk Name 2BC1 PQ: 0 ANSI: 2 CCS
  [ 1664.874690] sd 1:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
  [ 1664.881544] sd 1:0:0:0: [sdb] Write Protect is off
  [ 1664.885663] sd 1:0:0:0: [sdb] Mode Sense: 28 00 00 00
  [ 1664.886284] sd 1:0:0:0: [sdb] No Caching mode page found
  [ 1664.891160] sd 1:0:0:0: [sdb] Assuming drive cache: write through
  [ 1664.899793] sd 1:0:0:0: [sdb] No Caching mode page found
  [ 1664.902859] sd 1:0:0:0: [sdb] Assuming drive cache: write through
  [ 1664.985418]  sdb: sdb1
  [ 1664.988355] sd 1:0:0:0: [sdb] No Caching mode page found
  [ 1664.988410] sd 1:0:0:0: [sdb] Assuming drive cache: write through
  [ 1664.994563] sd 1:0:0:0: [sdb] Attached SCSI disk
  [ 1665.049694] Aborting journal on device dm-0-8.
  [ 1665.050009] Buffer I/O error on device dm-0, logical block 243826688
  [ 1665.055199] lost page write due to I/O error on dm-0
  [ 1665.060425] JBD2: Error -5 detected when updating journal superblock for dm-0-8.
  [ 1689.789480] Buffer I/O error on device dm-0, logical block 0
  [ 1689.793190] lost page write due to I/O error on dm-0
  [ 1689.798373] EXT4-fs error (device dm-0): __ext4_journal_start_sb:62: Detected aborted journal
  [ 1689.806990] EXT4-fs (dm-0): Remounting filesystem read-only
  [ 1689.812846] EXT4-fs (dm-0): previous I/O error to superblock detected
  [ 1689.819660] Buffer I/O error on device dm-0, logical block 0
  [ 1689.825145] lost page write due to I/O error on dm-0
  [ 1689.830320] EXT4-fs (dm-0): ext4_da_writepages: jbd2_start: 5120 pages, ino 13; err -30


I wonder if there are any ideas how dm-crypto or LUKS can handle this
case. Or is the recommendation that the USB suspend should not happen
at all for devices that are broken (ie. that disconnect and reconnect
on resume)?
Even if all handles on /dev/sda are released internally, there is not
really a guarantee that the devices comes back as /dev/sda are the
disconnect/reconnect. However, the UUID should be the same, so that
could be used to detect it's the same device and partition then and
accesses get re-routed to it then transparently.


Thanks, Axel

Re: [dm-crypt] dm-crypt LUKS and USB power management

[Ritesh Raj Sarraf]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]

On Sat, 2016-05-21 at 03:04 +0200, Axel Heider wrote:
> 
> I wonder if there are any ideas how dm-crypto or LUKS can handle this
> case. Or is the recommendation that the USB suspend should not happen
> at all for devices that are broken (ie. that disconnect and reconnect
> on resume)?
> Even if all handles on /dev/sda are released internally, there is not
> really a guarantee that the devices comes back as /dev/sda are the
> disconnect/reconnect. However, the UUID should be the same, so that
> could be used to detect it's the same device and partition then and
> accesses get re-routed to it then transparently.

I doubt if it could be done clean. Most targets in Device Mapper ask for careful
unstacking.

Red Hat (leading Linux vendor) still seems to be recommending that.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-si
ngle/Storage_Administration_Guide/#removing_devices


I would rather investigate the (flaky) USB device. First, does it happen only
when Runtime PM is enabled ? If so, you should just blacklist it from Power
Management. Many devices, under Linux, report (false) PM capabilities.


-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [dm-crypt] dm-crypt LUKS and USB power management

[Axel Heider]

> I doubt if it could be done clean. Most targets in Device Mapper ask for
> careful unstacking.

Couldn't LUKS/cryptsetup/dm-crypt set up a monitor for the disk peripheral
at least? So it releases any connection to the device if it is diconnected.
The device is gone anyway, so there is no gain in keeping any handles open.
Internally, the higher layer file system driver should still get errors
then. But the lower lever driver stack is no longer blocked. So a new
device can become /dev/sda again and not dev/sdb because /dev/sda is still
"somehow" active.



> I would rather investigate the (flaky) USB device. First, does it happen only
> when Runtime PM is enabled ? If so, you should just blacklist it from Power
> Management. Many devices, under Linux, report (false) PM capabilities.

My current solution is using a hub with a dedicated power support to
connect the USB/SATA adapter with the HDD. Then disk spindown can
still be used, but there is no USB disconnect/reconnect. That solves
the problem practically.
I did not find a way yet to disable power/idele management on the
board USB ports. It's a Odroid C1 or C2 with Debian Jesse Kernel
3.14.29 or 3.14.65. The suggestion from
http://unix.stackexchange.com/questions/91027/how-to-disable-usb-autosuspend-on-kernel-3-7-10-or-above
does not work
and so far nobody else had a solution. Will keep searching...

Axel

Re: [dm-crypt] dm-crypt LUKS and USB power management

[Arno Wagner]

That would be the wrong place to do it. Monitoring USB
disconnect is the task of the USB driver, also because
LUKS/LVM/etc. has no clue what hardware is used on
the lower layers.

Regards,
Arno


On Tue, May 31, 2016 at 12:59:38 CEST, Axel Heider wrote:
> 
> 
> 
> > I doubt if it could be done clean. Most targets in Device Mapper ask for
> > careful unstacking.
> 
> Couldn't LUKS/cryptsetup/dm-crypt set up a monitor for the disk peripheral
> at least? So it releases any connection to the device if it is diconnected.
> The device is gone anyway, so there is no gain in keeping any handles open.
> Internally, the higher layer file system driver should still get errors
> then. But the lower lever driver stack is no longer blocked. So a new
> device can become /dev/sda again and not dev/sdb because /dev/sda is still
> "somehow" active.
> 
> 
> 
> > I would rather investigate the (flaky) USB device. First, does it happen only
> > when Runtime PM is enabled ? If so, you should just blacklist it from Power
> > Management. Many devices, under Linux, report (false) PM capabilities.
> 
> My current solution is using a hub with a dedicated power support to
> connect the USB/SATA adapter with the HDD. Then disk spindown can
> still be used, but there is no USB disconnect/reconnect. That solves
> the problem practically.
> I did not find a way yet to disable power/idele management on the
> board USB ports. It's a Odroid C1 or C2 with Debian Jesse Kernel
> 3.14.29 or 3.14.65. The suggestion from
> http://unix.stackexchange.com/questions/91027/how-to-disable-usb-autosuspend-on-kernel-3-7-10-or-above
> does not work
> and so far nobody else had a solution. Will keep searching...
> 
> Axel
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier