* losetup on a image file containing (GPT) partition doesn't create partition devices @ 2014-10-03 9:44 Francis Moreau 2014-10-03 14:29 ` Dale R. Worley 0 siblings, 1 reply; 13+ messages in thread From: Francis Moreau @ 2014-10-03 9:44 UTC (permalink / raw) To: util-linux Hello, I have a disk image file which is using GPT and embeds 2 partitions. I'm setting up a loop device with: $ losetup -P -f disk.img and expect the partition devices (loop0p[12] for example) to be automatically created. During the loop setup, udev reposrts: $ udevadm monitor KERNEL[88697.611649] change /devices/virtual/block/loop0 (block) UDEV [88697.612488] change /devices/virtual/block/loop0 (block) KERNEL[88698.738369] change /devices/virtual/block/loop0 (block) UDEV [88698.744727] change /devices/virtual/block/loop0 (block) The loop kernel module was loaded with max_part=16 The weird thing is that if I'm setting up the loop device using a blank image file and create the partitions (using gdisk) on this loop dev, it works. I'm not sure if this is an issue to util-linux though and if it's not, sorry for the noise. Thanks. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-03 9:44 losetup on a image file containing (GPT) partition doesn't create partition devices Francis Moreau @ 2014-10-03 14:29 ` Dale R. Worley 2014-10-03 19:58 ` Dale R. Worley 0 siblings, 1 reply; 13+ messages in thread From: Dale R. Worley @ 2014-10-03 14:29 UTC (permalink / raw) To: Francis Moreau; +Cc: util-linux > From: Francis Moreau <francis.moro@gmail.com> > > I have a disk image file which is using GPT and embeds 2 partitions. > > I'm setting up a loop device with: > > $ losetup -P -f disk.img > > and expect the partition devices (loop0p[12] for example) to be > automatically created. I tried to reproduce this problem myself (on Fedora 19 with a 3.14.17-100.fc19.x86_64 kernel). I created a disk image file and used gdisk to install two partitions on it, then detached the file from the loopback device. The first time I re-attached it, /dev/loop0p{1,2} were created, as expected. The only message that shows up in /var/log/messages is: Oct 3 10:20:03 hobgoblin kernel: [560334.624034] loop0: p1 p2 Then I detached the loopback device, re-attached it, and this time, the partition devices were *not* created, and /var/log/messages did not have the "loop0: p1 p2" message. Given that the message starts "kernel:", I suspect that the udev behavior is triggered by the kernel detecting the partition table, and that is not happening. So it looks like it's a kernel bug. Dale ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-03 14:29 ` Dale R. Worley @ 2014-10-03 19:58 ` Dale R. Worley 2014-10-03 20:26 ` Francis Moreau 0 siblings, 1 reply; 13+ messages in thread From: Dale R. Worley @ 2014-10-03 19:58 UTC (permalink / raw) To: francis.moro, util-linux > From: worley@alum.mit.edu (Dale R. Worley) > Then I detached the loopback device, re-attached it, and this time, > the partition devices were *not* created, and /var/log/messages did > not have the "loop0: p1 p2" message. > > Given that the message starts "kernel:", I suspect that the udev > behavior is triggered by the kernel detecting the partition table, and > that is not happening. So it looks like it's a kernel bug. I tried that again, after updating to kernel 3.14.19-100.fc19.x86_64, and it worked this time. Dale ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-03 19:58 ` Dale R. Worley @ 2014-10-03 20:26 ` Francis Moreau 2014-10-04 19:35 ` Francis Moreau 0 siblings, 1 reply; 13+ messages in thread From: Francis Moreau @ 2014-10-03 20:26 UTC (permalink / raw) To: Dale R. Worley, util-linux On 10/03/2014 09:58 PM, Dale R. Worley wrote: >> From: worley@alum.mit.edu (Dale R. Worley) > >> Then I detached the loopback device, re-attached it, and this time, >> the partition devices were *not* created, and /var/log/messages did >> not have the "loop0: p1 p2" message. >> >> Given that the message starts "kernel:", I suspect that the udev >> behavior is triggered by the kernel detecting the partition table, and >> that is not happening. So it looks like it's a kernel bug. > > I tried that again, after updating to kernel 3.14.19-100.fc19.x86_64, > and it worked this time. Well I'm already running 3.14.18, and the problem is still present but not systematic. Here's a script which should trigger the problem: ----- while :; do losetup -P /dev/loop0 disk.raw || exit test -e /dev/loop0p1 || { echo "missing loop partition" losetup -d /dev/loop0 break } echo -n "losetup worked: " ls /dev/loop0* losetup -d /dev/loop0 || exit sleep 1 done ---- After looking at the kernel code, the kernel can silently fail to scan the partition table. This can happen if another process is already using the device and I think udev is the process that can interfere with 'losetup -P'. What do you think ? thanks. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-03 20:26 ` Francis Moreau @ 2014-10-04 19:35 ` Francis Moreau 2014-10-06 15:47 ` Dale R. Worley 0 siblings, 1 reply; 13+ messages in thread From: Francis Moreau @ 2014-10-04 19:35 UTC (permalink / raw) To: Dale R. Worley, util-linux On 10/03/2014 10:26 PM, Francis Moreau wrote: > On 10/03/2014 09:58 PM, Dale R. Worley wrote: >>> From: worley@alum.mit.edu (Dale R. Worley) >> >>> Then I detached the loopback device, re-attached it, and this time, >>> the partition devices were *not* created, and /var/log/messages did >>> not have the "loop0: p1 p2" message. >>> >>> Given that the message starts "kernel:", I suspect that the udev >>> behavior is triggered by the kernel detecting the partition table, and >>> that is not happening. So it looks like it's a kernel bug. >> >> I tried that again, after updating to kernel 3.14.19-100.fc19.x86_64, >> and it worked this time. > > Well I'm already running 3.14.18, and the problem is still present but > not systematic. > > Here's a script which should trigger the problem: > > ----- > while :; do > > losetup -P /dev/loop0 disk.raw || > exit > > test -e /dev/loop0p1 || { > echo "missing loop partition" > losetup -d /dev/loop0 > break > } > echo -n "losetup worked: " > ls /dev/loop0* > > losetup -d /dev/loop0 || > exit > > sleep 1 > done > ---- > > After looking at the kernel code, the kernel can silently fail to scan > the partition table. This can happen if another process is already using > the device and I think udev is the process that can interfere with > 'losetup -P'. > > What do you think ? Ok, so IMHO I think that losetup(8) should issue its own ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup since it can fail silently. This way it can check the return value of ioctl() and moght choose to do it again in case the return value is -EBUSY. Or indicates in the man page that the -P can fail silently. Thanks ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-04 19:35 ` Francis Moreau @ 2014-10-06 15:47 ` Dale R. Worley 2014-10-06 15:56 ` Francis Moreau 0 siblings, 1 reply; 13+ messages in thread From: Dale R. Worley @ 2014-10-06 15:47 UTC (permalink / raw) To: Francis Moreau; +Cc: util-linux > From: Francis Moreau <francis.moro@gmail.com> > Ok, so IMHO I think that losetup(8) should issue its own > ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup > since it can fail silently. This way it can check the return value of > ioctl() and moght choose to do it again in case the return value is -EBUSY. > > Or indicates in the man page that the -P can fail silently. It's not clear to me why the kernel would be expected to check itself for partitions in this manner. If losetup does not get the -P option, it seems like the kernel should not check. In any case, it seems like a good idea to make sure that either the kernel does things reliably, or losetup ensures that it happens reliably. Dale ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-06 15:47 ` Dale R. Worley @ 2014-10-06 15:56 ` Francis Moreau 2014-10-27 20:16 ` Mike Frysinger 0 siblings, 1 reply; 13+ messages in thread From: Francis Moreau @ 2014-10-06 15:56 UTC (permalink / raw) To: Dale R. Worley; +Cc: util-linux, Karel Zak [Adding Karel in CC] On 10/06/2014 05:47 PM, Dale R. Worley wrote: >> From: Francis Moreau <francis.moro@gmail.com> > >> Ok, so IMHO I think that losetup(8) should issue its own >> ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup >> since it can fail silently. This way it can check the return value of >> ioctl() and moght choose to do it again in case the return value is -EBUSY. >> Hm I was probably not clear, let me phase it again: Ok, so IMHO I think that losetup(8) should issue its own ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup since the kernel can fail silently. Using ioctl(BLKRRPART) allows losetup(8) to check the return value of ioctl() and might choose to do it again in case the return value is -EBUSY. >> Or indicates in the man page that the -P can fail silently. > > It's not clear to me why the kernel would be expected to check itself > for partitions in this manner. If losetup does not get the -P option, > it seems like the kernel should not check. > Thanks ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-06 15:56 ` Francis Moreau @ 2014-10-27 20:16 ` Mike Frysinger 2014-12-02 19:29 ` Phillip Susi 0 siblings, 1 reply; 13+ messages in thread From: Mike Frysinger @ 2014-10-27 20:16 UTC (permalink / raw) To: Francis Moreau; +Cc: Dale R. Worley, util-linux, Karel Zak, gabeblack, gwendal [-- Attachment #1: Type: text/plain, Size: 1732 bytes --] On 06 Oct 2014 17:56, Francis Moreau wrote: > On 10/06/2014 05:47 PM, Dale R. Worley wrote: > > From: Francis Moreau <francis.moro@gmail.com> > >> Ok, so IMHO I think that losetup(8) should issue its own > >> ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup > >> since it can fail silently. This way it can check the return value of > >> ioctl() and moght choose to do it again in case the return value is -EBUSY. > > Hm I was probably not clear, let me phase it again: > > Ok, so IMHO I think that losetup(8) should issue its own > ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup > since the kernel can fail silently. Using ioctl(BLKRRPART) allows > losetup(8) to check the return value of ioctl() and might choose to do > it again in case the return value is -EBUSY. we've been tracking the same bug in Chromium OS: http://crbug.com/411693 basically, under load, using `losetup -P` randomly fails to create the relevant partition nodes. atm we have hacks in place to call `blockdev --rereadpt` when it looks like the kernel has failed us. it would be nice if the kernel didn't ignore the result of the ioctl thus allowing all errors to go unnoticed. you can see this in drivers/block/loop.c where it calls ioctl_by_bdev multiple times and doesn't check the return value. since all current kernels are broken though, and might be for the foreseeable future, having losetup issue the ioctl itself might be a good tradeoff. i think it'd trigger unnecessary overhead (rescanning the device after it already worked), but seems better than having losetup try and check other sources (like if arbitrary partition nodes were created). -mike [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-10-27 20:16 ` Mike Frysinger @ 2014-12-02 19:29 ` Phillip Susi 2014-12-03 8:36 ` Karel Zak 2015-02-16 8:52 ` Mike Frysinger 0 siblings, 2 replies; 13+ messages in thread From: Phillip Susi @ 2014-12-02 19:29 UTC (permalink / raw) To: Francis Moreau, Dale R. Worley, util-linux, Karel Zak, gabeblack, gwendal -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/27/2014 4:16 PM, Mike Frysinger wrote: > we've been tracking the same bug in Chromium OS: > http://crbug.com/411693 basically, under load, using `losetup -P` > randomly fails to create the relevant partition nodes. atm we have > hacks in place to call `blockdev --rereadpt` when it looks like the > kernel has failed us. > > it would be nice if the kernel didn't ignore the result of the > ioctl thus allowing all errors to go unnoticed. you can see this > in drivers/block/loop.c where it calls ioctl_by_bdev multiple times > and doesn't check the return value. > > since all current kernels are broken though, and might be for the > foreseeable future, having losetup issue the ioctl itself might be > a good tradeoff. i think it'd trigger unnecessary overhead > (rescanning the device after it already worked), but seems better > than having losetup try and check other sources (like if arbitrary > partition nodes were created). As long as the kernel has a flag that is supposed to cause it to interpret the partition table, then it should *work* and not silently fail. This is a bug in the kernel that should be fixed. As a workaround though, you might consider just not using losetup -P in the first place and just run partx when you want to activate partitions. This also has the advantage of working even when the loop module is not loaded with the max_parts argument. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) iQEcBAEBAgAGBQJUfhM3AAoJENRVrw2cjl5RvhcH/iaVXICM0zOxyc7YFTeQ4Tkk 5NgLJpZGy8yp4maxJcoPJ8Eryfs6SuQdglVtfjtGLhTOVEvUObe9ALey1/oilfUI RCzBYyr+j9raD09MMA+yiNsYpeeptvWPRnPZF9TahRKWTMYSiOZoDMGQjvc1Rj3w uSLDcmcknt/YA0nOSml8xuI84k1EeaV/s9Gt5Q9Rgjw8kvv3C5xROM9JfgC7unY8 4nF3n8F5jfGeGU/NtJAYJn+WnsX69AyAvveeo5e3GdO43NP818rF2K/6d9tyyN0Z 5159ZqRur75m/pC2PqQFMeoJ/3vefbWEMrQ3D73yH22f41ignZobzKHbGcDygD4= =qPDy -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-12-02 19:29 ` Phillip Susi @ 2014-12-03 8:36 ` Karel Zak 2014-12-03 14:49 ` Phillip Susi 2015-02-16 8:52 ` Mike Frysinger 1 sibling, 1 reply; 13+ messages in thread From: Karel Zak @ 2014-12-03 8:36 UTC (permalink / raw) To: Phillip Susi Cc: Francis Moreau, Dale R. Worley, util-linux, gabeblack, gwendal On Tue, Dec 02, 2014 at 02:29:59PM -0500, Phillip Susi wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 10/27/2014 4:16 PM, Mike Frysinger wrote: > > we've been tracking the same bug in Chromium OS: > > http://crbug.com/411693 basically, under load, using `losetup -P` > > randomly fails to create the relevant partition nodes. atm we have > > hacks in place to call `blockdev --rereadpt` when it looks like the > > kernel has failed us. > > > > it would be nice if the kernel didn't ignore the result of the > > ioctl thus allowing all errors to go unnoticed. you can see this > > in drivers/block/loop.c where it calls ioctl_by_bdev multiple times > > and doesn't check the return value. > > > > since all current kernels are broken though, and might be for the > > foreseeable future, having losetup issue the ioctl itself might be > > a good tradeoff. i think it'd trigger unnecessary overhead > > (rescanning the device after it already worked), but seems better > > than having losetup try and check other sources (like if arbitrary > > partition nodes were created). > > As long as the kernel has a flag that is supposed to cause it to > interpret the partition table, then it should *work* and not silently > fail. This is a bug in the kernel that should be fixed. Yes, I don't like the idea with the ioctl workaround in losetup. It's fine to use such workarounds in stable branches or in distro specific patches, but I really don't want to use workarounds for bugs in regular releases (master branch). Karel -- Karel Zak <kzak@redhat.com> http://karelzak.blogspot.com ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-12-03 8:36 ` Karel Zak @ 2014-12-03 14:49 ` Phillip Susi 2014-12-03 16:10 ` Karel Zak 0 siblings, 1 reply; 13+ messages in thread From: Phillip Susi @ 2014-12-03 14:49 UTC (permalink / raw) To: Karel Zak; +Cc: Francis Moreau, Dale R. Worley, util-linux, gabeblack, gwendal -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/3/2014 3:36 AM, Karel Zak wrote: > Yes, I don't like the idea with the ioctl workaround in losetup. > It's fine to use such workarounds in stable branches or in distro > specific patches, but I really don't want to use workarounds for > bugs in regular releases (master branch). Then again, maybe the whole idea of the kernel handling the partition table should be abandoned ( and the feature removed from the kernel ) and losetup should just call partx and be done with it. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) iQEcBAEBAgAGBQJUfyL3AAoJENRVrw2cjl5RGIUH+wWT/HOUblnqldu4M8zgNZ9c T08BjQKPET1ylRfuVDDdEET5yaP8NI94dNisgyvWWDHbi1PA3I45dodQA3XaywWf Ue6wEUMV8tujjC7zgUup+yxIca2accrmRP4tqlVOswRMS4nFJqyRHJsDW0azJDR4 3VoaGP1WRf9vtNs4m71a/yljjirAHRBcQ8NVdgDnLe2Ja6Quy2REvVsTCO951O6i oH5AYBnvEE0iiycxUxj1EGkNRl3QC58/NIJi633MXRrL5VTOd2k37OzEZb7/P/g/ rKBbqnwMMuDVgarF7Ob8KkJb62BKca6W4Sbd806TvygCoB4K8EGW1zVpK1LUAAI= =qTsq -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-12-03 14:49 ` Phillip Susi @ 2014-12-03 16:10 ` Karel Zak 0 siblings, 0 replies; 13+ messages in thread From: Karel Zak @ 2014-12-03 16:10 UTC (permalink / raw) To: Phillip Susi Cc: Francis Moreau, Dale R. Worley, util-linux, gabeblack, gwendal On Wed, Dec 03, 2014 at 09:49:27AM -0500, Phillip Susi wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 12/3/2014 3:36 AM, Karel Zak wrote: > > Yes, I don't like the idea with the ioctl workaround in losetup. > > It's fine to use such workarounds in stable branches or in distro > > specific patches, but I really don't want to use workarounds for > > bugs in regular releases (master branch). > > Then again, maybe the whole idea of the kernel handling the partition > table should be abandoned ( and the feature removed from the kernel ) > and losetup should just call partx and be done with it. That's another story :-) And yes, I mostly agree. Karel -- Karel Zak <kzak@redhat.com> http://karelzak.blogspot.com ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: losetup on a image file containing (GPT) partition doesn't create partition devices 2014-12-02 19:29 ` Phillip Susi 2014-12-03 8:36 ` Karel Zak @ 2015-02-16 8:52 ` Mike Frysinger 1 sibling, 0 replies; 13+ messages in thread From: Mike Frysinger @ 2015-02-16 8:52 UTC (permalink / raw) To: Phillip Susi Cc: Francis Moreau, Dale R. Worley, util-linux, Karel Zak, gabeblack, gwendal [-- Attachment #1: Type: text/plain, Size: 1768 bytes --] On 02 Dec 2014 14:29, Phillip Susi wrote: > On 10/27/2014 4:16 PM, Mike Frysinger wrote: > > we've been tracking the same bug in Chromium OS: > > http://crbug.com/411693 basically, under load, using `losetup -P` > > randomly fails to create the relevant partition nodes. atm we have > > hacks in place to call `blockdev --rereadpt` when it looks like the > > kernel has failed us. > > > > it would be nice if the kernel didn't ignore the result of the > > ioctl thus allowing all errors to go unnoticed. you can see this > > in drivers/block/loop.c where it calls ioctl_by_bdev multiple times > > and doesn't check the return value. > > > > since all current kernels are broken though, and might be for the > > foreseeable future, having losetup issue the ioctl itself might be > > a good tradeoff. i think it'd trigger unnecessary overhead > > (rescanning the device after it already worked), but seems better > > than having losetup try and check other sources (like if arbitrary > > partition nodes were created). > > As long as the kernel has a flag that is supposed to cause it to > interpret the partition table, then it should *work* and not silently > fail. This is a bug in the kernel that should be fixed. i agree the kernel should be fixed. but like i said, it isn't today, and it won't be for a while, and even then it'll take a while for all those fixed kernel versions to percolate out to distros. > As a workaround though, you might consider just not using losetup -P > in the first place and just run partx when you want to activate > partitions. This also has the advantage of working even when the loop > module is not loaded with the max_parts argument. thanks, wasn't aware of the partx tool -mike [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2015-02-16 8:52 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-10-03 9:44 losetup on a image file containing (GPT) partition doesn't create partition devices Francis Moreau 2014-10-03 14:29 ` Dale R. Worley 2014-10-03 19:58 ` Dale R. Worley 2014-10-03 20:26 ` Francis Moreau 2014-10-04 19:35 ` Francis Moreau 2014-10-06 15:47 ` Dale R. Worley 2014-10-06 15:56 ` Francis Moreau 2014-10-27 20:16 ` Mike Frysinger 2014-12-02 19:29 ` Phillip Susi 2014-12-03 8:36 ` Karel Zak 2014-12-03 14:49 ` Phillip Susi 2014-12-03 16:10 ` Karel Zak 2015-02-16 8:52 ` Mike Frysinger
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.