* [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
@ 2022-04-06 15:46 ` Fabio Aiuto
0 siblings, 0 replies; 7+ messages in thread
From: Fabio Aiuto @ 2022-04-06 15:46 UTC (permalink / raw)
To: agk, snitzer; +Cc: dm-devel, linux-kernel, Michael Trimarchi
From: Michael Trimarchi <michael@amarulasolutions.com>
The device driver can be deferrable and can be a race during
the dm-init early. We need to wait all the probe are really finished
in a loop as is done in do_mounts. This is was tested on kernel 5.4
but code seems was not changed since that time
003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not found, using dummy regulator
003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not found, using dummy regulator
003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0 supported-hw 0x20 0x1
003: caam-dma caam-dma: caam dma support with 2 job rings
000: hctosys: unable to open rtc device (rtc0)
000: device-mapper: init: waiting for all devices to be available before creating mapped devices
002: device-mapper: table: 254:0: verity: Data device lookup failed
002: device-mapper: ioctl: error adding target to table
002: crng init done
003: of_cfs_init
003: of_cfs_init: OK
003: Waiting for root device /dev/dm-0...
001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
001: VSD_3V3: disabling
with the patch
003: device-mapper: init: waiting for all devices to be available before creating mapped devices
000: device-mapper: table: 254:0: verity: Data device lookup failed
000: device-mapper: ioctl: error adding target to table
002: crng init done
003: device-mapper: init: waiting for all devices to be available before creating mapped devices
003: device-mapper: table: 254:0: verity: Data device lookup failed
003: device-mapper: ioctl: error adding target to table
003: device-mapper: init: waiting for all devices to be available before creating mapped devices
000: device-mapper: table: 254:0: verity: Data device lookup failed
000: device-mapper: ioctl: error adding target to table
002: device-mapper: init: waiting for all devices to be available before creating mapped devices
002: device-mapper: table: 254:0: verity: Data device lookup failed
002: device-mapper: ioctl: error adding target to table
000: device-mapper: init: waiting for all devices to be available before creating mapped devices
000: device-mapper: table: 254:0: verity: Data device lookup failed
000: device-mapper: ioctl: error adding target to table
003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
002: device-mapper: init: waiting for all devices to be available before creating mapped devices
003: device-mapper: verity: sha256 using implementation "sha256-caam"
000: device-mapper: ioctl: dm-0 (rootfs) is ready
Wait loop is limited to 10 at the moment for our use case showed no
more than 4 loops before successfully find data device.
Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
---
Changes from v1:
- limit the loop to 10 iterations
- change variable names
- check only for -ENODEV failures
Changes from v2:
- use a limit in seconds (not in retry
number)
- add a parameter
- update docs
.../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
drivers/md/dm-init.c | 23 +++++++++++++++++--
2 files changed, 34 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst b/Documentation/admin-guide/device-mapper/dm-init.rst
index e5242ff17e9b..5c2f2bf1db03 100644
--- a/Documentation/admin-guide/device-mapper/dm-init.rst
+++ b/Documentation/admin-guide/device-mapper/dm-init.rst
@@ -123,3 +123,16 @@ Other examples (per target):
0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
+
+Delay for waiting deferred probes of block devices
+==================================================
+
+Sometimes the late initcall starting the early creation of mapped
+devices, starts too early. A loop waiting for probing of block
+devices has been added; the default maximum delay is 1 second but
+it can be set through the following kernel command::
+
+ dm-mod.delay=<seconds>
+
+This allows the procedure to retry the creation of a mapped device
+after a short wait (5 msecs).
diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
index b0c45c6ebe0b..f4c5b4a46001 100644
--- a/drivers/md/dm-init.c
+++ b/drivers/md/dm-init.c
@@ -7,7 +7,9 @@
* This file is released under the GPLv2.
*/
+#include <linux/async.h>
#include <linux/ctype.h>
+#include <linux/delay.h>
#include <linux/device.h>
#include <linux/device-mapper.h>
#include <linux/init.h>
@@ -18,8 +20,10 @@
#define DM_MAX_DEVICES 256
#define DM_MAX_TARGETS 256
#define DM_MAX_STR_SIZE 4096
+#define DM_DEFAULT_MAX_PROBE_DELAY 1
static char *create;
+static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
/*
* Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
@@ -267,6 +271,8 @@ static int __init dm_init_init(void)
LIST_HEAD(devices);
char *str;
int r;
+ int loopcnt = delay * 1000 / 5;
+ bool devnotfound = false;
if (!create)
return 0;
@@ -275,6 +281,7 @@ static int __init dm_init_init(void)
DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
return -EINVAL;
}
+retry:
str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
if (!str)
return -ENOMEM;
@@ -287,13 +294,23 @@ static int __init dm_init_init(void)
wait_for_device_probe();
list_for_each_entry(dev, &devices, list) {
- if (dm_early_create(&dev->dmi, dev->table,
- dev->target_args_array))
+ r = dm_early_create(&dev->dmi, dev->table, dev->target_args_array);
+ if (r == -ENODEV) {
+ devnotfound = true;
break;
+ }
}
+
out:
kfree(str);
dm_setup_cleanup(&devices);
+ if (devnotfound && loopcnt) {
+ msleep(5);
+ devnotfound = false;
+ loopcnt--;
+ goto retry;
+ }
+
return r;
}
@@ -301,3 +318,5 @@ late_initcall(dm_init_init);
module_param(create, charp, 0);
MODULE_PARM_DESC(create, "Create a mapped device in early boot");
+module_param(delay, int, 0);
+MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe in seconds");
--
2.30.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [dm-devel] [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
@ 2022-04-06 15:46 ` Fabio Aiuto
0 siblings, 0 replies; 7+ messages in thread
From: Fabio Aiuto @ 2022-04-06 15:46 UTC (permalink / raw)
To: agk, snitzer; +Cc: dm-devel, Michael Trimarchi, linux-kernel
[-- Attachment #1: Type: application/octet-stream, Size: 6735 bytes --]
From: Michael Trimarchi <michael@amarulasolutions.com>
The device driver can be deferrable and can be a race during
the dm-init early. We need to wait all the probe are really finished
in a loop as is done in do_mounts. This is was tested on kernel 5.4
but code seems was not changed since that time
003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not found, using dummy regulator
003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not found, using dummy regulator
003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0 supported-hw 0x20 0x1
003: caam-dma caam-dma: caam dma support with 2 job rings
000: hctosys: unable to open rtc device (rtc0)
000: device-mapper: init: waiting for all devices to be available before creating mapped devices
002: device-mapper: table: 254:0: verity: Data device lookup failed
002: device-mapper: ioctl: error adding target to table
002: crng init done
003: of_cfs_init
003: of_cfs_init: OK
003: Waiting for root device /dev/dm-0...
001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
001: VSD_3V3: disabling
with the patch
003: device-mapper: init: waiting for all devices to be available before creating mapped devices
000: device-mapper: table: 254:0: verity: Data device lookup failed
000: device-mapper: ioctl: error adding target to table
002: crng init done
003: device-mapper: init: waiting for all devices to be available before creating mapped devices
003: device-mapper: table: 254:0: verity: Data device lookup failed
003: device-mapper: ioctl: error adding target to table
003: device-mapper: init: waiting for all devices to be available before creating mapped devices
000: device-mapper: table: 254:0: verity: Data device lookup failed
000: device-mapper: ioctl: error adding target to table
002: device-mapper: init: waiting for all devices to be available before creating mapped devices
002: device-mapper: table: 254:0: verity: Data device lookup failed
002: device-mapper: ioctl: error adding target to table
000: device-mapper: init: waiting for all devices to be available before creating mapped devices
000: device-mapper: table: 254:0: verity: Data device lookup failed
000: device-mapper: ioctl: error adding target to table
003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
002: device-mapper: init: waiting for all devices to be available before creating mapped devices
003: device-mapper: verity: sha256 using implementation "sha256-caam"
000: device-mapper: ioctl: dm-0 (rootfs) is ready
Wait loop is limited to 10 at the moment for our use case showed no
more than 4 loops before successfully find data device.
Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
---
Changes from v1:
- limit the loop to 10 iterations
- change variable names
- check only for -ENODEV failures
Changes from v2:
- use a limit in seconds (not in retry
number)
- add a parameter
- update docs
.../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
drivers/md/dm-init.c | 23 +++++++++++++++++--
2 files changed, 34 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst b/Documentation/admin-guide/device-mapper/dm-init.rst
index e5242ff17e9b..5c2f2bf1db03 100644
--- a/Documentation/admin-guide/device-mapper/dm-init.rst
+++ b/Documentation/admin-guide/device-mapper/dm-init.rst
@@ -123,3 +123,16 @@ Other examples (per target):
0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
+
+Delay for waiting deferred probes of block devices
+==================================================
+
+Sometimes the late initcall starting the early creation of mapped
+devices, starts too early. A loop waiting for probing of block
+devices has been added; the default maximum delay is 1 second but
+it can be set through the following kernel command::
+
+ dm-mod.delay=<seconds>
+
+This allows the procedure to retry the creation of a mapped device
+after a short wait (5 msecs).
diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
index b0c45c6ebe0b..f4c5b4a46001 100644
--- a/drivers/md/dm-init.c
+++ b/drivers/md/dm-init.c
@@ -7,7 +7,9 @@
* This file is released under the GPLv2.
*/
+#include <linux/async.h>
#include <linux/ctype.h>
+#include <linux/delay.h>
#include <linux/device.h>
#include <linux/device-mapper.h>
#include <linux/init.h>
@@ -18,8 +20,10 @@
#define DM_MAX_DEVICES 256
#define DM_MAX_TARGETS 256
#define DM_MAX_STR_SIZE 4096
+#define DM_DEFAULT_MAX_PROBE_DELAY 1
static char *create;
+static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
/*
* Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
@@ -267,6 +271,8 @@ static int __init dm_init_init(void)
LIST_HEAD(devices);
char *str;
int r;
+ int loopcnt = delay * 1000 / 5;
+ bool devnotfound = false;
if (!create)
return 0;
@@ -275,6 +281,7 @@ static int __init dm_init_init(void)
DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
return -EINVAL;
}
+retry:
str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
if (!str)
return -ENOMEM;
@@ -287,13 +294,23 @@ static int __init dm_init_init(void)
wait_for_device_probe();
list_for_each_entry(dev, &devices, list) {
- if (dm_early_create(&dev->dmi, dev->table,
- dev->target_args_array))
+ r = dm_early_create(&dev->dmi, dev->table, dev->target_args_array);
+ if (r == -ENODEV) {
+ devnotfound = true;
break;
+ }
}
+
out:
kfree(str);
dm_setup_cleanup(&devices);
+ if (devnotfound && loopcnt) {
+ msleep(5);
+ devnotfound = false;
+ loopcnt--;
+ goto retry;
+ }
+
return r;
}
@@ -301,3 +318,5 @@ late_initcall(dm_init_init);
module_param(create, charp, 0);
MODULE_PARM_DESC(create, "Create a mapped device in early boot");
+module_param(delay, int, 0);
+MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe in seconds");
--
2.30.2
[-- Attachment #2: Type: text/plain, Size: 98 bytes --]
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
2022-04-06 15:46 ` [dm-devel] " Fabio Aiuto
@ 2022-04-23 8:13 ` Michael Nazzareno Trimarchi
-1 siblings, 0 replies; 7+ messages in thread
From: Michael Nazzareno Trimarchi @ 2022-04-23 8:13 UTC (permalink / raw)
To: Fabio Aiuto; +Cc: agk, snitzer, dm-devel, linux-kernel
Hi all
On Wed, Apr 6, 2022 at 5:46 PM Fabio Aiuto
<fabio.aiuto@amarulasolutions.com> wrote:
>
> From: Michael Trimarchi <michael@amarulasolutions.com>
>
> The device driver can be deferrable and can be a race during
> the dm-init early. We need to wait all the probe are really finished
> in a loop as is done in do_mounts. This is was tested on kernel 5.4
> but code seems was not changed since that time
>
Any follow up?
Michael
> 003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0 supported-hw 0x20 0x1
> 003: caam-dma caam-dma: caam dma support with 2 job rings
> 000: hctosys: unable to open rtc device (rtc0)
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: of_cfs_init
> 003: of_cfs_init: OK
> 003: Waiting for root device /dev/dm-0...
> 001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
> 001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
> 001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
> 001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
> 001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 001: VSD_3V3: disabling
>
> with the patch
>
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
>
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: table: 254:0: verity: Data device lookup failed
> 003: device-mapper: ioctl: error adding target to table
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
> 003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
> 003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
> 003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
> 003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: verity: sha256 using implementation "sha256-caam"
> 000: device-mapper: ioctl: dm-0 (rootfs) is ready
>
> Wait loop is limited to 10 at the moment for our use case showed no
> more than 4 loops before successfully find data device.
>
> Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
> Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
> ---
> Changes from v1:
> - limit the loop to 10 iterations
> - change variable names
> - check only for -ENODEV failures
>
> Changes from v2:
> - use a limit in seconds (not in retry
> number)
> - add a parameter
> - update docs
>
> .../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
> drivers/md/dm-init.c | 23 +++++++++++++++++--
> 2 files changed, 34 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst b/Documentation/admin-guide/device-mapper/dm-init.rst
> index e5242ff17e9b..5c2f2bf1db03 100644
> --- a/Documentation/admin-guide/device-mapper/dm-init.rst
> +++ b/Documentation/admin-guide/device-mapper/dm-init.rst
> @@ -123,3 +123,16 @@ Other examples (per target):
> 0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
> fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
> 51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
> +
> +Delay for waiting deferred probes of block devices
> +==================================================
> +
> +Sometimes the late initcall starting the early creation of mapped
> +devices, starts too early. A loop waiting for probing of block
> +devices has been added; the default maximum delay is 1 second but
> +it can be set through the following kernel command::
> +
> + dm-mod.delay=<seconds>
> +
> +This allows the procedure to retry the creation of a mapped device
> +after a short wait (5 msecs).
> diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
> index b0c45c6ebe0b..f4c5b4a46001 100644
> --- a/drivers/md/dm-init.c
> +++ b/drivers/md/dm-init.c
> @@ -7,7 +7,9 @@
> * This file is released under the GPLv2.
> */
>
> +#include <linux/async.h>
> #include <linux/ctype.h>
> +#include <linux/delay.h>
> #include <linux/device.h>
> #include <linux/device-mapper.h>
> #include <linux/init.h>
> @@ -18,8 +20,10 @@
> #define DM_MAX_DEVICES 256
> #define DM_MAX_TARGETS 256
> #define DM_MAX_STR_SIZE 4096
> +#define DM_DEFAULT_MAX_PROBE_DELAY 1
>
> static char *create;
> +static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
>
> /*
> * Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
> @@ -267,6 +271,8 @@ static int __init dm_init_init(void)
> LIST_HEAD(devices);
> char *str;
> int r;
> + int loopcnt = delay * 1000 / 5;
> + bool devnotfound = false;
>
> if (!create)
> return 0;
> @@ -275,6 +281,7 @@ static int __init dm_init_init(void)
> DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
> return -EINVAL;
> }
> +retry:
> str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
> if (!str)
> return -ENOMEM;
> @@ -287,13 +294,23 @@ static int __init dm_init_init(void)
> wait_for_device_probe();
>
> list_for_each_entry(dev, &devices, list) {
> - if (dm_early_create(&dev->dmi, dev->table,
> - dev->target_args_array))
> + r = dm_early_create(&dev->dmi, dev->table, dev->target_args_array);
> + if (r == -ENODEV) {
> + devnotfound = true;
> break;
> + }
> }
> +
> out:
> kfree(str);
> dm_setup_cleanup(&devices);
> + if (devnotfound && loopcnt) {
> + msleep(5);
> + devnotfound = false;
> + loopcnt--;
> + goto retry;
> + }
> +
> return r;
> }
>
> @@ -301,3 +318,5 @@ late_initcall(dm_init_init);
>
> module_param(create, charp, 0);
> MODULE_PARM_DESC(create, "Create a mapped device in early boot");
> +module_param(delay, int, 0);
> +MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe in seconds");
> --
> 2.30.2
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dm-devel] [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
@ 2022-04-23 8:13 ` Michael Nazzareno Trimarchi
0 siblings, 0 replies; 7+ messages in thread
From: Michael Nazzareno Trimarchi @ 2022-04-23 8:13 UTC (permalink / raw)
To: Fabio Aiuto; +Cc: dm-devel, snitzer, linux-kernel, agk
Hi all
On Wed, Apr 6, 2022 at 5:46 PM Fabio Aiuto
<fabio.aiuto@amarulasolutions.com> wrote:
>
> From: Michael Trimarchi <michael@amarulasolutions.com>
>
> The device driver can be deferrable and can be a race during
> the dm-init early. We need to wait all the probe are really finished
> in a loop as is done in do_mounts. This is was tested on kernel 5.4
> but code seems was not changed since that time
>
Any follow up?
Michael
> 003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0 supported-hw 0x20 0x1
> 003: caam-dma caam-dma: caam dma support with 2 job rings
> 000: hctosys: unable to open rtc device (rtc0)
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: of_cfs_init
> 003: of_cfs_init: OK
> 003: Waiting for root device /dev/dm-0...
> 001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
> 001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
> 001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
> 001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
> 001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 001: VSD_3V3: disabling
>
> with the patch
>
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
>
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: table: 254:0: verity: Data device lookup failed
> 003: device-mapper: ioctl: error adding target to table
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
> 003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
> 003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
> 003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
> 003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: verity: sha256 using implementation "sha256-caam"
> 000: device-mapper: ioctl: dm-0 (rootfs) is ready
>
> Wait loop is limited to 10 at the moment for our use case showed no
> more than 4 loops before successfully find data device.
>
> Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
> Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
> ---
> Changes from v1:
> - limit the loop to 10 iterations
> - change variable names
> - check only for -ENODEV failures
>
> Changes from v2:
> - use a limit in seconds (not in retry
> number)
> - add a parameter
> - update docs
>
> .../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
> drivers/md/dm-init.c | 23 +++++++++++++++++--
> 2 files changed, 34 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst b/Documentation/admin-guide/device-mapper/dm-init.rst
> index e5242ff17e9b..5c2f2bf1db03 100644
> --- a/Documentation/admin-guide/device-mapper/dm-init.rst
> +++ b/Documentation/admin-guide/device-mapper/dm-init.rst
> @@ -123,3 +123,16 @@ Other examples (per target):
> 0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
> fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
> 51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
> +
> +Delay for waiting deferred probes of block devices
> +==================================================
> +
> +Sometimes the late initcall starting the early creation of mapped
> +devices, starts too early. A loop waiting for probing of block
> +devices has been added; the default maximum delay is 1 second but
> +it can be set through the following kernel command::
> +
> + dm-mod.delay=<seconds>
> +
> +This allows the procedure to retry the creation of a mapped device
> +after a short wait (5 msecs).
> diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
> index b0c45c6ebe0b..f4c5b4a46001 100644
> --- a/drivers/md/dm-init.c
> +++ b/drivers/md/dm-init.c
> @@ -7,7 +7,9 @@
> * This file is released under the GPLv2.
> */
>
> +#include <linux/async.h>
> #include <linux/ctype.h>
> +#include <linux/delay.h>
> #include <linux/device.h>
> #include <linux/device-mapper.h>
> #include <linux/init.h>
> @@ -18,8 +20,10 @@
> #define DM_MAX_DEVICES 256
> #define DM_MAX_TARGETS 256
> #define DM_MAX_STR_SIZE 4096
> +#define DM_DEFAULT_MAX_PROBE_DELAY 1
>
> static char *create;
> +static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
>
> /*
> * Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
> @@ -267,6 +271,8 @@ static int __init dm_init_init(void)
> LIST_HEAD(devices);
> char *str;
> int r;
> + int loopcnt = delay * 1000 / 5;
> + bool devnotfound = false;
>
> if (!create)
> return 0;
> @@ -275,6 +281,7 @@ static int __init dm_init_init(void)
> DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
> return -EINVAL;
> }
> +retry:
> str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
> if (!str)
> return -ENOMEM;
> @@ -287,13 +294,23 @@ static int __init dm_init_init(void)
> wait_for_device_probe();
>
> list_for_each_entry(dev, &devices, list) {
> - if (dm_early_create(&dev->dmi, dev->table,
> - dev->target_args_array))
> + r = dm_early_create(&dev->dmi, dev->table, dev->target_args_array);
> + if (r == -ENODEV) {
> + devnotfound = true;
> break;
> + }
> }
> +
> out:
> kfree(str);
> dm_setup_cleanup(&devices);
> + if (devnotfound && loopcnt) {
> + msleep(5);
> + devnotfound = false;
> + loopcnt--;
> + goto retry;
> + }
> +
> return r;
> }
>
> @@ -301,3 +318,5 @@ late_initcall(dm_init_init);
>
> module_param(create, charp, 0);
> MODULE_PARM_DESC(create, "Create a mapped device in early boot");
> +module_param(delay, int, 0);
> +MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe in seconds");
> --
> 2.30.2
>
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
2022-04-06 15:46 ` [dm-devel] " Fabio Aiuto
@ 2022-09-09 20:12 ` Sean Anderson
-1 siblings, 0 replies; 7+ messages in thread
From: Sean Anderson @ 2022-09-09 20:12 UTC (permalink / raw)
To: Fabio Aiuto, agk, snitzer; +Cc: dm-devel, linux-kernel, Michael Trimarchi
I ran into this issue today. This patch fixes things, so
Tested-by: Sean Anderson <sean.anderson@seco.com>
However, I think the implementation leaves a bit to be desired...
On 4/6/22 11:46 AM, Fabio Aiuto wrote:
> From: Michael Trimarchi <michael@amarulasolutions.com>
>
> The device driver can be deferrable and can be a race during
> the dm-init early. We need to wait all the probe are really finished
> in a loop as is done in do_mounts. This is was tested on kernel 5.4
> but code seems was not changed since that time
>
> 003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0 supported-hw 0x20 0x1
> 003: caam-dma caam-dma: caam dma support with 2 job rings
> 000: hctosys: unable to open rtc device (rtc0)
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: of_cfs_init
> 003: of_cfs_init: OK
> 003: Waiting for root device /dev/dm-0...
> 001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
> 001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
> 001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
> 001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
> 001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 001: VSD_3V3: disabling
>
> with the patch
>
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
>
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: table: 254:0: verity: Data device lookup failed
> 003: device-mapper: ioctl: error adding target to table
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
For example, we repeatedly print these errors, even though there is not
really an error condition. Additionally, we still loop even if (e.g.)
there is a syntax error in the dm-mod.create parameter. It also
introduces similar problems to rootdelay, where you have to choose a
suitable maximum value.
I would much rather see something closer to rootwait (as implemented in
prepare_namespace). To do that, I think it is better to modify
dm_get_device or dm_get_dev_t, which are closer to the source of the
error. One issue is that these functions are not just called from
dm_early_create, but also at runtime. So we can't just blindly start
waiting for devices to show up if they don't exist and dm-mod.wait (or
whatever) is missing. We could solve this by introducing a variable
which is cleared at the end of dm_early_create. I think that solution is
OK. We could also modify dm_early_create to pass a parameter to
dm_table_add_target which indicates that the target is being created
"early." However, I think this would be fairly disruptive to existing
code.
--Sean
> 003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
> 003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
> 003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
> 003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
> 003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: verity: sha256 using implementation "sha256-caam"
> 000: device-mapper: ioctl: dm-0 (rootfs) is ready
>
> Wait loop is limited to 10 at the moment for our use case showed no
> more than 4 loops before successfully find data device.
>
> Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
> Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
> ---
> Changes from v1:
> - limit the loop to 10 iterations
> - change variable names
> - check only for -ENODEV failures
>
> Changes from v2:
> - use a limit in seconds (not in retry
> number)
> - add a parameter
> - update docs
>
> .../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
> drivers/md/dm-init.c | 23 +++++++++++++++++--
> 2 files changed, 34 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst b/Documentation/admin-guide/device-mapper/dm-init.rst
> index e5242ff17e9b..5c2f2bf1db03 100644
> --- a/Documentation/admin-guide/device-mapper/dm-init.rst
> +++ b/Documentation/admin-guide/device-mapper/dm-init.rst
> @@ -123,3 +123,16 @@ Other examples (per target):
> 0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
> fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
> 51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
> +
> +Delay for waiting deferred probes of block devices
> +==================================================
> +
> +Sometimes the late initcall starting the early creation of mapped
> +devices, starts too early. A loop waiting for probing of block
> +devices has been added; the default maximum delay is 1 second but
> +it can be set through the following kernel command::
> +
> + dm-mod.delay=<seconds>
> +
> +This allows the procedure to retry the creation of a mapped device
> +after a short wait (5 msecs).
> diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
> index b0c45c6ebe0b..f4c5b4a46001 100644
> --- a/drivers/md/dm-init.c
> +++ b/drivers/md/dm-init.c
> @@ -7,7 +7,9 @@
> * This file is released under the GPLv2.
> */
>
> +#include <linux/async.h>
> #include <linux/ctype.h>
> +#include <linux/delay.h>
> #include <linux/device.h>
> #include <linux/device-mapper.h>
> #include <linux/init.h>
> @@ -18,8 +20,10 @@
> #define DM_MAX_DEVICES 256
> #define DM_MAX_TARGETS 256
> #define DM_MAX_STR_SIZE 4096
> +#define DM_DEFAULT_MAX_PROBE_DELAY 1
>
> static char *create;
> +static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
>
> /*
> * Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
> @@ -267,6 +271,8 @@ static int __init dm_init_init(void)
> LIST_HEAD(devices);
> char *str;
> int r;
> + int loopcnt = delay * 1000 / 5;
> + bool devnotfound = false;
>
> if (!create)
> return 0;
> @@ -275,6 +281,7 @@ static int __init dm_init_init(void)
> DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
> return -EINVAL;
> }
> +retry:
> str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
> if (!str)
> return -ENOMEM;
> @@ -287,13 +294,23 @@ static int __init dm_init_init(void)
> wait_for_device_probe();
>
> list_for_each_entry(dev, &devices, list) {
> - if (dm_early_create(&dev->dmi, dev->table,
> - dev->target_args_array))
> + r = dm_early_create(&dev->dmi, dev->table, dev->target_args_array);
> + if (r == -ENODEV) {
> + devnotfound = true;
> break;
> + }
> }
> +
> out:
> kfree(str);
> dm_setup_cleanup(&devices);
> + if (devnotfound && loopcnt) {
> + msleep(5);
> + devnotfound = false;
> + loopcnt--;
> + goto retry;
> + }
> +
> return r;
> }
>
> @@ -301,3 +318,5 @@ late_initcall(dm_init_init);
>
> module_param(create, charp, 0);
> MODULE_PARM_DESC(create, "Create a mapped device in early boot");
> +module_param(delay, int, 0);
> +MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe in seconds");
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dm-devel] [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
@ 2022-09-09 20:12 ` Sean Anderson
0 siblings, 0 replies; 7+ messages in thread
From: Sean Anderson @ 2022-09-09 20:12 UTC (permalink / raw)
To: Fabio Aiuto, agk, snitzer; +Cc: dm-devel, Michael Trimarchi, linux-kernel
I ran into this issue today. This patch fixes things, so
Tested-by: Sean Anderson <sean.anderson@seco.com>
However, I think the implementation leaves a bit to be desired...
On 4/6/22 11:46 AM, Fabio Aiuto wrote:
> From: Michael Trimarchi <michael@amarulasolutions.com>
>
> The device driver can be deferrable and can be a race during
> the dm-init early. We need to wait all the probe are really finished
> in a loop as is done in do_mounts. This is was tested on kernel 5.4
> but code seems was not changed since that time
>
> 003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not found, using dummy regulator
> 003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0 supported-hw 0x20 0x1
> 003: caam-dma caam-dma: caam dma support with 2 job rings
> 000: hctosys: unable to open rtc device (rtc0)
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: of_cfs_init
> 003: of_cfs_init: OK
> 003: Waiting for root device /dev/dm-0...
> 001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
> 001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
> 001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
> 001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
> 001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 001: VSD_3V3: disabling
>
> with the patch
>
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
>
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: crng init done
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: table: 254:0: verity: Data device lookup failed
> 003: device-mapper: ioctl: error adding target to table
> 003: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 002: device-mapper: table: 254:0: verity: Data device lookup failed
> 002: device-mapper: ioctl: error adding target to table
> 000: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 000: device-mapper: table: 254:0: verity: Data device lookup failed
> 000: device-mapper: ioctl: error adding target to table
For example, we repeatedly print these errors, even though there is not
really an error condition. Additionally, we still loop even if (e.g.)
there is a syntax error in the dm-mod.create parameter. It also
introduces similar problems to rootdelay, where you have to choose a
suitable maximum value.
I would much rather see something closer to rootwait (as implemented in
prepare_namespace). To do that, I think it is better to modify
dm_get_device or dm_get_dev_t, which are closer to the source of the
error. One issue is that these functions are not just called from
dm_early_create, but also at runtime. So we can't just blindly start
waiting for devices to show up if they don't exist and dm-mod.wait (or
whatever) is missing. We could solve this by introducing a variable
which is cleared at the end of dm_early_create. I think that solution is
OK. We could also modify dm_early_create to pass a parameter to
dm_table_add_target which indicates that the target is being created
"early." However, I think this would be fairly disruptive to existing
code.
--Sean
> 003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> 003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
> 003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
> 003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
> 003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
> 003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> 002: device-mapper: init: waiting for all devices to be available before creating mapped devices
> 003: device-mapper: verity: sha256 using implementation "sha256-caam"
> 000: device-mapper: ioctl: dm-0 (rootfs) is ready
>
> Wait loop is limited to 10 at the moment for our use case showed no
> more than 4 loops before successfully find data device.
>
> Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
> Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
> ---
> Changes from v1:
> - limit the loop to 10 iterations
> - change variable names
> - check only for -ENODEV failures
>
> Changes from v2:
> - use a limit in seconds (not in retry
> number)
> - add a parameter
> - update docs
>
> .../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
> drivers/md/dm-init.c | 23 +++++++++++++++++--
> 2 files changed, 34 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst b/Documentation/admin-guide/device-mapper/dm-init.rst
> index e5242ff17e9b..5c2f2bf1db03 100644
> --- a/Documentation/admin-guide/device-mapper/dm-init.rst
> +++ b/Documentation/admin-guide/device-mapper/dm-init.rst
> @@ -123,3 +123,16 @@ Other examples (per target):
> 0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
> fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
> 51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
> +
> +Delay for waiting deferred probes of block devices
> +==================================================
> +
> +Sometimes the late initcall starting the early creation of mapped
> +devices, starts too early. A loop waiting for probing of block
> +devices has been added; the default maximum delay is 1 second but
> +it can be set through the following kernel command::
> +
> + dm-mod.delay=<seconds>
> +
> +This allows the procedure to retry the creation of a mapped device
> +after a short wait (5 msecs).
> diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
> index b0c45c6ebe0b..f4c5b4a46001 100644
> --- a/drivers/md/dm-init.c
> +++ b/drivers/md/dm-init.c
> @@ -7,7 +7,9 @@
> * This file is released under the GPLv2.
> */
>
> +#include <linux/async.h>
> #include <linux/ctype.h>
> +#include <linux/delay.h>
> #include <linux/device.h>
> #include <linux/device-mapper.h>
> #include <linux/init.h>
> @@ -18,8 +20,10 @@
> #define DM_MAX_DEVICES 256
> #define DM_MAX_TARGETS 256
> #define DM_MAX_STR_SIZE 4096
> +#define DM_DEFAULT_MAX_PROBE_DELAY 1
>
> static char *create;
> +static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
>
> /*
> * Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
> @@ -267,6 +271,8 @@ static int __init dm_init_init(void)
> LIST_HEAD(devices);
> char *str;
> int r;
> + int loopcnt = delay * 1000 / 5;
> + bool devnotfound = false;
>
> if (!create)
> return 0;
> @@ -275,6 +281,7 @@ static int __init dm_init_init(void)
> DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
> return -EINVAL;
> }
> +retry:
> str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
> if (!str)
> return -ENOMEM;
> @@ -287,13 +294,23 @@ static int __init dm_init_init(void)
> wait_for_device_probe();
>
> list_for_each_entry(dev, &devices, list) {
> - if (dm_early_create(&dev->dmi, dev->table,
> - dev->target_args_array))
> + r = dm_early_create(&dev->dmi, dev->table, dev->target_args_array);
> + if (r == -ENODEV) {
> + devnotfound = true;
> break;
> + }
> }
> +
> out:
> kfree(str);
> dm_setup_cleanup(&devices);
> + if (devnotfound && loopcnt) {
> + msleep(5);
> + devnotfound = false;
> + loopcnt--;
> + goto retry;
> + }
> +
> return r;
> }
>
> @@ -301,3 +318,5 @@ late_initcall(dm_init_init);
>
> module_param(create, charp, 0);
> MODULE_PARM_DESC(create, "Create a mapped device in early boot");
> +module_param(delay, int, 0);
> +MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe in seconds");
>
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [dm-devel] [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet
2022-09-09 20:12 ` [dm-devel] " Sean Anderson
(?)
@ 2022-09-15 21:20 ` Michael Nazzareno Trimarchi
-1 siblings, 0 replies; 7+ messages in thread
From: Michael Nazzareno Trimarchi @ 2022-09-15 21:20 UTC (permalink / raw)
To: Sean Anderson; +Cc: Fabio Aiuto, dm-devel, snitzer, linux-kernel, agk
[-- Attachment #1.1: Type: text/plain, Size: 9500 bytes --]
Hi Sean
On Fri, Sep 9, 2022 at 10:12 PM Sean Anderson <sean.anderson@seco.com>
wrote:
> I ran into this issue today. This patch fixes things, so
>
> Tested-by: Sean Anderson <sean.anderson@seco.com>
>
> However, I think the implementation leaves a bit to be desired...
>
> On 4/6/22 11:46 AM, Fabio Aiuto wrote:
> > From: Michael Trimarchi <michael@amarulasolutions.com>
> >
> > The device driver can be deferrable and can be a race during
> > the dm-init early. We need to wait all the probe are really finished
> > in a loop as is done in do_mounts. This is was tested on kernel 5.4
> > but code seems was not changed since that time
> >
>
Ok, I know, I was waiting for some feedback. I will try to rework
Michael
> > 003: imx8mq-usb-phy 381f0040.usb-phy: 381f0040.usb-phy supply vbus not
> found, using dummy regulator
> > 003: imx8mq-usb-phy 382f0040.usb-phy: 382f0040.usb-phy supply vbus not
> found, using dummy regulator
> > 003: imx-cpufreq-dt imx-cpufreq-dt: cpu speed grade 5 mkt segment 0
> supported-hw 0x20 0x1
> > 003: caam-dma caam-dma: caam dma support with 2 job rings
> > 000: hctosys: unable to open rtc device (rtc0)
> > 000: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> > 002: device-mapper: table: 254:0: verity: Data device lookup failed
> > 002: device-mapper: ioctl: error adding target to table
> > 002: crng init done
> > 003: of_cfs_init
> > 003: of_cfs_init: OK
> > 003: Waiting for root device /dev/dm-0...
> > 001: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> > 001: mmcblk2: mmc2:0001 IB2916 14.6 GiB
> > 001: mmcblk2boot0: mmc2:0001 IB2916 partition 1 4.00 MiB
> > 001: mmcblk2boot1: mmc2:0001 IB2916 partition 2 4.00 MiB
> > 001: mmcblk2rpmb: mmc2:0001 IB2916 partition 3 4.00 MiB, chardev (249:0)
> > 001: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> > 001: VSD_3V3: disabling
> >
> > with the patch
> >
> > 003: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> >
> > 000: device-mapper: table: 254:0: verity: Data device lookup failed
> > 000: device-mapper: ioctl: error adding target to table
> > 002: crng init done
> > 003: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> > 003: device-mapper: table: 254:0: verity: Data device lookup failed
> > 003: device-mapper: ioctl: error adding target to table
> > 003: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> > 000: device-mapper: table: 254:0: verity: Data device lookup failed
> > 000: device-mapper: ioctl: error adding target to table
> > 002: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> > 002: device-mapper: table: 254:0: verity: Data device lookup failed
> > 002: device-mapper: ioctl: error adding target to table
> > 000: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> > 000: device-mapper: table: 254:0: verity: Data device lookup failed
> > 000: device-mapper: ioctl: error adding target to table
>
> For example, we repeatedly print these errors, even though there is not
> really an error condition. Additionally, we still loop even if (e.g.)
> there is a syntax error in the dm-mod.create parameter. It also
> introduces similar problems to rootdelay, where you have to choose a
> suitable maximum value.
>
> I would much rather see something closer to rootwait (as implemented in
> prepare_namespace). To do that, I think it is better to modify
> dm_get_device or dm_get_dev_t, which are closer to the source of the
> error. One issue is that these functions are not just called from
> dm_early_create, but also at runtime. So we can't just blindly start
> waiting for devices to show up if they don't exist and dm-mod.wait (or
> whatever) is missing. We could solve this by introducing a variable
> which is cleared at the end of dm_early_create. I think that solution is
> OK. We could also modify dm_early_create to pass a parameter to
> dm_table_add_target which indicates that the target is being created
> "early." However, I think this would be fairly disruptive to existing
> code.
>
> --Sean
>
> > 003: mmc2: new HS400 Enhanced strobe MMC card at address 0001
> > 003: mmcblk2: mmc2:0001 DG4016 14.7 GiB
> > 003: mmcblk2boot0: mmc2:0001 DG4016 partition 1 4.00 MiB
> > 003: mmcblk2boot1: mmc2:0001 DG4016 partition 2 4.00 MiB
> > 003: mmcblk2rpmb: mmc2:0001 DG4016 partition 3 4.00 MiB, chardev (249:0)
> > 003: mmcblk2: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> > 002: device-mapper: init: waiting for all devices to be available before
> creating mapped devices
> > 003: device-mapper: verity: sha256 using implementation "sha256-caam"
> > 000: device-mapper: ioctl: dm-0 (rootfs) is ready
> >
> > Wait loop is limited to 10 at the moment for our use case showed no
> > more than 4 loops before successfully find data device.
> >
> > Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
> > Signed-off-by: Fabio Aiuto <fabio.aiuto@amarulasolutions.com>
> > ---
> > Changes from v1:
> > - limit the loop to 10 iterations
> > - change variable names
> > - check only for -ENODEV failures
> >
> > Changes from v2:
> > - use a limit in seconds (not in retry
> > number)
> > - add a parameter
> > - update docs
> >
> > .../admin-guide/device-mapper/dm-init.rst | 13 +++++++++++
> > drivers/md/dm-init.c | 23 +++++++++++++++++--
> > 2 files changed, 34 insertions(+), 2 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/device-mapper/dm-init.rst
> b/Documentation/admin-guide/device-mapper/dm-init.rst
> > index e5242ff17e9b..5c2f2bf1db03 100644
> > --- a/Documentation/admin-guide/device-mapper/dm-init.rst
> > +++ b/Documentation/admin-guide/device-mapper/dm-init.rst
> > @@ -123,3 +123,16 @@ Other examples (per target):
> > 0 1638400 verity 1 8:1 8:2 4096 4096 204800 1 sha256
> > fb1a5a0f00deb908d8b53cb270858975e76cf64105d412ce764225d53b8f3cfd
> > 51934789604d1b92399c52e7cb149d1b3a1b74bbbcb103b2a0aaacbed5c08584
> > +
> > +Delay for waiting deferred probes of block devices
> > +==================================================
> > +
> > +Sometimes the late initcall starting the early creation of mapped
> > +devices, starts too early. A loop waiting for probing of block
> > +devices has been added; the default maximum delay is 1 second but
> > +it can be set through the following kernel command::
> > +
> > + dm-mod.delay=<seconds>
> > +
> > +This allows the procedure to retry the creation of a mapped device
> > +after a short wait (5 msecs).
> > diff --git a/drivers/md/dm-init.c b/drivers/md/dm-init.c
> > index b0c45c6ebe0b..f4c5b4a46001 100644
> > --- a/drivers/md/dm-init.c
> > +++ b/drivers/md/dm-init.c
> > @@ -7,7 +7,9 @@
> > * This file is released under the GPLv2.
> > */
> >
> > +#include <linux/async.h>
> > #include <linux/ctype.h>
> > +#include <linux/delay.h>
> > #include <linux/device.h>
> > #include <linux/device-mapper.h>
> > #include <linux/init.h>
> > @@ -18,8 +20,10 @@
> > #define DM_MAX_DEVICES 256
> > #define DM_MAX_TARGETS 256
> > #define DM_MAX_STR_SIZE 4096
> > +#define DM_DEFAULT_MAX_PROBE_DELAY 1
> >
> > static char *create;
> > +static int delay = DM_DEFAULT_MAX_PROBE_DELAY;
> >
> > /*
> > * Format:
> dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
> > @@ -267,6 +271,8 @@ static int __init dm_init_init(void)
> > LIST_HEAD(devices);
> > char *str;
> > int r;
> > + int loopcnt = delay * 1000 / 5;
> > + bool devnotfound = false;
> >
> > if (!create)
> > return 0;
> > @@ -275,6 +281,7 @@ static int __init dm_init_init(void)
> > DMERR("Argument is too big. Limit is %d", DM_MAX_STR_SIZE);
> > return -EINVAL;
> > }
> > +retry:
> > str = kstrndup(create, DM_MAX_STR_SIZE, GFP_KERNEL);
> > if (!str)
> > return -ENOMEM;
> > @@ -287,13 +294,23 @@ static int __init dm_init_init(void)
> > wait_for_device_probe();
> >
> > list_for_each_entry(dev, &devices, list) {
> > - if (dm_early_create(&dev->dmi, dev->table,
> > - dev->target_args_array))
> > + r = dm_early_create(&dev->dmi, dev->table,
> dev->target_args_array);
> > + if (r == -ENODEV) {
> > + devnotfound = true;
> > break;
> > + }
> > }
> > +
> > out:
> > kfree(str);
> > dm_setup_cleanup(&devices);
> > + if (devnotfound && loopcnt) {
> > + msleep(5);
> > + devnotfound = false;
> > + loopcnt--;
> > + goto retry;
> > + }
> > +
> > return r;
> > }
> >
> > @@ -301,3 +318,5 @@ late_initcall(dm_init_init);
> >
> > module_param(create, charp, 0);
> > MODULE_PARM_DESC(create, "Create a mapped device in early boot");
> > +module_param(delay, int, 0);
> > +MODULE_PARM_DESC(delay, "Max delay to wait for data/hash device probe
> in seconds");
> >
>
--
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael@amarulasolutions.com
__________________________________
Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info@amarulasolutions.com
www.amarulasolutions.com
[-- Attachment #1.2: Type: text/html, Size: 12243 bytes --]
[-- Attachment #2: Type: text/plain, Size: 98 bytes --]
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-09-16 8:45 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-06 15:46 [PATCH v3] md: dm-init: Wait devices if it's not find on first adpet Fabio Aiuto
2022-04-06 15:46 ` [dm-devel] " Fabio Aiuto
2022-04-23 8:13 ` Michael Nazzareno Trimarchi
2022-04-23 8:13 ` [dm-devel] " Michael Nazzareno Trimarchi
2022-09-09 20:12 ` Sean Anderson
2022-09-09 20:12 ` [dm-devel] " Sean Anderson
2022-09-15 21:20 ` Michael Nazzareno Trimarchi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.