* Re: SCSI device probing non-deterministic in 5.3 [not found] <d2ff27ce-67b0-735e-8652-0e925d5f756c@linode.com> @ 2019-10-03 21:19 ` Randy Dunlap 2019-10-04 15:39 ` Bart Van Assche 0 siblings, 1 reply; 3+ messages in thread From: Randy Dunlap @ 2019-10-03 21:19 UTC (permalink / raw) To: Bradley LaBoon, linux-kernel, linux-scsi [add linux-scsi mailing list] On 10/3/19 1:32 PM, Bradley LaBoon wrote: > Hello, LKML! > > Beginning with kernel 5.3 the order in which SCSI devices are probed and > named has become non-deterministic. This is a result of a patch that was > submitted to add asynchronous device probing (specifically, commit > f049cf1a7b6737c75884247c3f6383ef104d255a). Previously, devices would > always be probed in the order in which they exist on the bus, resulting > in the first device being named 'sda', the second device 'sdb', and so on. > > This is important in the case of mass VM deployments where many VMs are > created from a single base image. Partition UUIDs cannot be used in the > fstab of such an image because the UUIDs will be different for each VM > and are not known in advance. Normally you can't rely on device names > being consistent between boots, but with QEMU you can set the bus order > of each block device and thus we currently use that to control the > device order in the guest. With the introduction of the aforementioned > patch this is no longer possible and the device ordering is different on > every boot, resulting in the guest booting into an emergency shell > unless the devices randomly happen to be loaded in the expected order. > > I have created a patch which reverts back to the previous behavior, but > I wanted to open this topic to discussion before posting it. I'm not > totally familiar with the low-level details of SCSI device probing, so I > don't know if the non-deterministic device order was the intended > behavior of the patch or just a side-effect. If that is the intended > behavior then is there perhaps some other way to ensure a consistent > device ordering for a guest VM? > > I am not subscribed to the list, so please CC me on any replies. > > Thank you! > Bradley LaBoon > -- ~Randy ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: SCSI device probing non-deterministic in 5.3 2019-10-03 21:19 ` SCSI device probing non-deterministic in 5.3 Randy Dunlap @ 2019-10-04 15:39 ` Bart Van Assche 2019-10-08 20:20 ` Bradley LaBoon 0 siblings, 1 reply; 3+ messages in thread From: Bart Van Assche @ 2019-10-04 15:39 UTC (permalink / raw) To: Randy Dunlap, Bradley LaBoon, linux-kernel, linux-scsi On 10/3/19 2:19 PM, Randy Dunlap wrote: > [add linux-scsi mailing list] > > On 10/3/19 1:32 PM, Bradley LaBoon wrote: >> Hello, LKML! >> >> Beginning with kernel 5.3 the order in which SCSI devices are probed and >> named has become non-deterministic. This is a result of a patch that was >> submitted to add asynchronous device probing (specifically, commit >> f049cf1a7b6737c75884247c3f6383ef104d255a). Previously, devices would >> always be probed in the order in which they exist on the bus, resulting >> in the first device being named 'sda', the second device 'sdb', and so on. >> >> This is important in the case of mass VM deployments where many VMs are >> created from a single base image. Partition UUIDs cannot be used in the >> fstab of such an image because the UUIDs will be different for each VM >> and are not known in advance. Normally you can't rely on device names >> being consistent between boots, but with QEMU you can set the bus order >> of each block device and thus we currently use that to control the >> device order in the guest. With the introduction of the aforementioned >> patch this is no longer possible and the device ordering is different on >> every boot, resulting in the guest booting into an emergency shell >> unless the devices randomly happen to be loaded in the expected order. >> >> I have created a patch which reverts back to the previous behavior, but >> I wanted to open this topic to discussion before posting it. I'm not >> totally familiar with the low-level details of SCSI device probing, so I >> don't know if the non-deterministic device order was the intended >> behavior of the patch or just a side-effect. If that is the intended >> behavior then is there perhaps some other way to ensure a consistent >> device ordering for a guest VM? Have you already had a look at the /dev/disk/by-path directory? An example of the contents of that directory: $ (cd /dev/disk/by-path && ls -l | grep /s) lrwxrwxrwx 1 root root 9 Oct 3 16:49 pci-0000:00:02.0-ata-1 -> ../../sda lrwxrwxrwx 1 root root 9 Oct 3 16:49 pci-0000:00:08.0-scsi-0:0:0:1 -> ../../sr0 Have you considered to use these soft links in /etc/fstab? In case using these links would be impractical: have you considered to add a udev rule that creates H:C:I:L soft links under a subdirectory of /dev, that makes these soft links point at the /dev/sd* device nodes and to use these soft links in /etc/fstab? That's probably a much more elegant solution than what has been proposed above. As one can see the information that is needed to implement such a udev rule is already available in sysfs: $ (cd /sys/class/scsi_device && ls -ld */device/block/*) drwxr-xr-x 9 root root 0 Oct 3 16:48 2:0:0:1/device/block/sr0 drwxr-xr-x 9 root root 0 Oct 3 16:48 3:0:0:0/device/block/sda Bart. ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: SCSI device probing non-deterministic in 5.3 2019-10-04 15:39 ` Bart Van Assche @ 2019-10-08 20:20 ` Bradley LaBoon 0 siblings, 0 replies; 3+ messages in thread From: Bradley LaBoon @ 2019-10-08 20:20 UTC (permalink / raw) To: Bart Van Assche, Randy Dunlap, linux-kernel, linux-scsi On 10/4/19 11:39 AM, Bart Van Assche wrote: > Have you already had a look at the /dev/disk/by-path directory? An > example of the contents of that directory: > > $ (cd /dev/disk/by-path && ls -l | grep /s) > lrwxrwxrwx 1 root root 9 Oct 3 16:49 pci-0000:00:02.0-ata-1 -> ../../sda > lrwxrwxrwx 1 root root 9 Oct 3 16:49 pci-0000:00:08.0-scsi-0:0:0:1 -> > ../../sr0 > > Have you considered to use these soft links in /etc/fstab? > > In case using these links would be impractical: have you considered to > add a udev rule that creates H:C:I:L soft links under a subdirectory of > /dev, that makes these soft links point at the /dev/sd* device nodes and > to use these soft links in /etc/fstab? That's probably a much more > elegant solution than what has been proposed above. > > As one can see the information that is needed to implement such a udev > rule is already available in sysfs: > > $ (cd /sys/class/scsi_device && ls -ld */device/block/*) > drwxr-xr-x 9 root root 0 Oct 3 16:48 2:0:0:1/device/block/sr0 > drwxr-xr-x 9 root root 0 Oct 3 16:48 3:0:0:0/device/block/sda > > Bart. Thanks for the reply! I wasn't aware that sysfs exposed the mapping that way. That is very useful, and we should be able to utilize that. Regards, Bradley LaBoon ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-10-08 20:20 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <d2ff27ce-67b0-735e-8652-0e925d5f756c@linode.com> 2019-10-03 21:19 ` SCSI device probing non-deterministic in 5.3 Randy Dunlap 2019-10-04 15:39 ` Bart Van Assche 2019-10-08 20:20 ` Bradley LaBoon
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).