* [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-09-02 23:00 ` Douglas Anderson 0 siblings, 0 replies; 18+ messages in thread From: Douglas Anderson @ 2020-09-02 23:00 UTC (permalink / raw) To: Tudor Ambarus Cc: Douglas Anderson, Miquel Raynal, Richard Weinberger, Vignesh Raghavendra, linux-kernel, linux-mtd On my system the spi_nor_probe() took ~6 ms at bootup. That's not a lot, but every little bit adds up to a slow bootup. While we can get this out of the boot path by making it a module, there are times where it is convenient (or even required) for this to be builtin the kernel. Let's set that we prefer async probe so that we don't block other drivers from probing while we are probing. This is a tiny little change that is almost guaranteed to be safe for anything that is able to run as a module, which SPI_NOR is. Specifically modules are already probed asynchronously. Also: since other things in the system may have enabled asynchronous probe the system may already be doing other things during our probe. There is a small possibility that some other driver that was a client of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe ordering and only worked when the SPI_NOR and the SPI bus were builtin. In that case the other driver has a bug that's waiting to hit and the other driver should be fixed. Signed-off-by: Douglas Anderson <dianders@chromium.org> --- drivers/mtd/spi-nor/core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c index 65eff4ce6ab1..756da93f0f16 100644 --- a/drivers/mtd/spi-nor/core.c +++ b/drivers/mtd/spi-nor/core.c @@ -3470,6 +3470,7 @@ static struct spi_mem_driver spi_nor_driver = { .driver = { .name = "spi-nor", .of_match_table = spi_nor_of_table, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, .id_table = spi_nor_dev_ids, }, -- 2.28.0.402.g5ffc5be6b7-goog ^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-09-02 23:00 ` Douglas Anderson 0 siblings, 0 replies; 18+ messages in thread From: Douglas Anderson @ 2020-09-02 23:00 UTC (permalink / raw) To: Tudor Ambarus Cc: Vignesh Raghavendra, Richard Weinberger, Douglas Anderson, linux-kernel, linux-mtd, Miquel Raynal On my system the spi_nor_probe() took ~6 ms at bootup. That's not a lot, but every little bit adds up to a slow bootup. While we can get this out of the boot path by making it a module, there are times where it is convenient (or even required) for this to be builtin the kernel. Let's set that we prefer async probe so that we don't block other drivers from probing while we are probing. This is a tiny little change that is almost guaranteed to be safe for anything that is able to run as a module, which SPI_NOR is. Specifically modules are already probed asynchronously. Also: since other things in the system may have enabled asynchronous probe the system may already be doing other things during our probe. There is a small possibility that some other driver that was a client of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe ordering and only worked when the SPI_NOR and the SPI bus were builtin. In that case the other driver has a bug that's waiting to hit and the other driver should be fixed. Signed-off-by: Douglas Anderson <dianders@chromium.org> --- drivers/mtd/spi-nor/core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/mtd/spi-nor/core.c b/drivers/mtd/spi-nor/core.c index 65eff4ce6ab1..756da93f0f16 100644 --- a/drivers/mtd/spi-nor/core.c +++ b/drivers/mtd/spi-nor/core.c @@ -3470,6 +3470,7 @@ static struct spi_mem_driver spi_nor_driver = { .driver = { .name = "spi-nor", .of_match_table = spi_nor_of_table, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, .id_table = spi_nor_dev_ids, }, -- 2.28.0.402.g5ffc5be6b7-goog ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-09-02 23:00 ` Douglas Anderson @ 2020-09-30 8:34 ` Vignesh Raghavendra -1 siblings, 0 replies; 18+ messages in thread From: Vignesh Raghavendra @ 2020-09-30 8:34 UTC (permalink / raw) To: Douglas Anderson, Tudor Ambarus Cc: Vignesh Raghavendra, linux-mtd, Miquel Raynal, Richard Weinberger, linux-kernel On Wed, 2 Sep 2020 16:00:40 -0700, Douglas Anderson wrote: > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > lot, but every little bit adds up to a slow bootup. While we can get > this out of the boot path by making it a module, there are times where > it is convenient (or even required) for this to be builtin the kernel. > Let's set that we prefer async probe so that we don't block other > drivers from probing while we are probing. > > [...] Applied to https://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git spi-nor/next, thanks! [1/1] mtd: spi-nor: Prefer asynchronous probe https://git.kernel.org/mtd/c/03edda0e1e -- Regards Vignesh ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-09-30 8:34 ` Vignesh Raghavendra 0 siblings, 0 replies; 18+ messages in thread From: Vignesh Raghavendra @ 2020-09-30 8:34 UTC (permalink / raw) To: Douglas Anderson, Tudor Ambarus Cc: Richard Weinberger, linux-mtd, Vignesh Raghavendra, linux-kernel, Miquel Raynal On Wed, 2 Sep 2020 16:00:40 -0700, Douglas Anderson wrote: > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > lot, but every little bit adds up to a slow bootup. While we can get > this out of the boot path by making it a module, there are times where > it is convenient (or even required) for this to be builtin the kernel. > Let's set that we prefer async probe so that we don't block other > drivers from probing while we are probing. > > [...] Applied to https://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git spi-nor/next, thanks! [1/1] mtd: spi-nor: Prefer asynchronous probe https://git.kernel.org/mtd/c/03edda0e1e -- Regards Vignesh ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-09-02 23:00 ` Douglas Anderson @ 2020-10-03 15:06 ` Michael Walle -1 siblings, 0 replies; 18+ messages in thread From: Michael Walle @ 2020-10-03 15:06 UTC (permalink / raw) To: dianders Cc: linux-kernel, linux-mtd, miquel.raynal, richard, tudor.ambarus, vigneshr, Michael Walle Hi Douglas, > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > lot, but every little bit adds up to a slow bootup. While we can get > this out of the boot path by making it a module, there are times where > it is convenient (or even required) for this to be builtin the kernel. > Let's set that we prefer async probe so that we don't block other > drivers from probing while we are probing. > > This is a tiny little change that is almost guaranteed to be safe for > anything that is able to run as a module, which SPI_NOR is. > Specifically modules are already probed asynchronously. Also: since > other things in the system may have enabled asynchronous probe the > system may already be doing other things during our probe. > > There is a small possibility that some other driver that was a client > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe > ordering and only worked when the SPI_NOR and the SPI bus were > builtin. In that case the other driver has a bug that's waiting to > hit and the other driver should be fixed. linux-next now triggers the following warning in kernel/kmod.c:136 on my board. I've bisected this to this patch. kmod.c: /* * We don't allow synchronous module loading from async. Module * init may invoke async_synchronize_full() which will end up * waiting for this task which already is waiting for the module * loading to complete, leading to a deadlock. */ WARN_ON_ONCE(wait && current_is_async()); [ 1.849801] ------------[ cut here ]------------ [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 __request_module+0x3a4/0x568 [ 1.858755] Modules linked in: [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted 5.9.0-rc6-00001-g03edda0e1eda #113 [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT) [ 1.872647] Workqueue: events_unbound async_run_entry_fn [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) [ 1.881297] pc : __request_module+0x3a4/0x568 [ 1.881299] lr : __request_module+0x39c/0x568 [ 1.881302] sp : ffff8000113a3920 [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 [ 1.975241] 10 fixed-partitions partitions found on MTD device 20c0000.spi [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 [ 1.979564] Call trace: [ 1.979571] __request_module+0x3a4/0x568 [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 [ 1.997133] 0x000000000000-0x000000010000 : "rcw" [ 2.002454] spi_nor_probe+0x29c/0x2f0 [ 2.002458] spi_mem_probe+0x74/0xb0 [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" [ 2.018433] spi_drv_probe+0x88/0xe8 [ 2.018439] really_probe+0xec/0x3c0 [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" [ 2.035555] driver_probe_device+0x60/0xc0 [ 2.035559] __device_attach_driver+0x8c/0xd0 [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted firmware" [ 2.044642] bus_for_each_drv+0x84/0xd8 [ 2.044645] __device_attach_async_helper+0xc4/0xe8 [ 2.044648] async_run_entry_fn+0x4c/0x150 [ 2.044653] process_one_work+0x1f4/0x4b8 [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" [ 2.062814] worker_thread+0x50/0x480 [ 2.062817] kthread+0x160/0x168 [ 2.062821] ret_from_fork+0x10/0x34 [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- -michael ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-03 15:06 ` Michael Walle 0 siblings, 0 replies; 18+ messages in thread From: Michael Walle @ 2020-10-03 15:06 UTC (permalink / raw) To: dianders Cc: vigneshr, tudor.ambarus, richard, linux-kernel, Michael Walle, linux-mtd, miquel.raynal Hi Douglas, > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > lot, but every little bit adds up to a slow bootup. While we can get > this out of the boot path by making it a module, there are times where > it is convenient (or even required) for this to be builtin the kernel. > Let's set that we prefer async probe so that we don't block other > drivers from probing while we are probing. > > This is a tiny little change that is almost guaranteed to be safe for > anything that is able to run as a module, which SPI_NOR is. > Specifically modules are already probed asynchronously. Also: since > other things in the system may have enabled asynchronous probe the > system may already be doing other things during our probe. > > There is a small possibility that some other driver that was a client > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe > ordering and only worked when the SPI_NOR and the SPI bus were > builtin. In that case the other driver has a bug that's waiting to > hit and the other driver should be fixed. linux-next now triggers the following warning in kernel/kmod.c:136 on my board. I've bisected this to this patch. kmod.c: /* * We don't allow synchronous module loading from async. Module * init may invoke async_synchronize_full() which will end up * waiting for this task which already is waiting for the module * loading to complete, leading to a deadlock. */ WARN_ON_ONCE(wait && current_is_async()); [ 1.849801] ------------[ cut here ]------------ [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 __request_module+0x3a4/0x568 [ 1.858755] Modules linked in: [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted 5.9.0-rc6-00001-g03edda0e1eda #113 [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT) [ 1.872647] Workqueue: events_unbound async_run_entry_fn [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) [ 1.881297] pc : __request_module+0x3a4/0x568 [ 1.881299] lr : __request_module+0x39c/0x568 [ 1.881302] sp : ffff8000113a3920 [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 [ 1.975241] 10 fixed-partitions partitions found on MTD device 20c0000.spi [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 [ 1.979564] Call trace: [ 1.979571] __request_module+0x3a4/0x568 [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 [ 1.997133] 0x000000000000-0x000000010000 : "rcw" [ 2.002454] spi_nor_probe+0x29c/0x2f0 [ 2.002458] spi_mem_probe+0x74/0xb0 [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" [ 2.018433] spi_drv_probe+0x88/0xe8 [ 2.018439] really_probe+0xec/0x3c0 [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" [ 2.035555] driver_probe_device+0x60/0xc0 [ 2.035559] __device_attach_driver+0x8c/0xd0 [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted firmware" [ 2.044642] bus_for_each_drv+0x84/0xd8 [ 2.044645] __device_attach_async_helper+0xc4/0xe8 [ 2.044648] async_run_entry_fn+0x4c/0x150 [ 2.044653] process_one_work+0x1f4/0x4b8 [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" [ 2.062814] worker_thread+0x50/0x480 [ 2.062817] kthread+0x160/0x168 [ 2.062821] ret_from_fork+0x10/0x34 [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- -michael ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-10-03 15:06 ` Michael Walle @ 2020-10-03 16:27 ` Doug Anderson -1 siblings, 0 replies; 18+ messages in thread From: Doug Anderson @ 2020-10-03 16:27 UTC (permalink / raw) To: Michael Walle Cc: LKML, linux-mtd, miquel.raynal, richard, tudor.ambarus, vigneshr Hi, On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: > > Hi Douglas, > > > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > > lot, but every little bit adds up to a slow bootup. While we can get > > this out of the boot path by making it a module, there are times where > > it is convenient (or even required) for this to be builtin the kernel. > > Let's set that we prefer async probe so that we don't block other > > drivers from probing while we are probing. > > > > This is a tiny little change that is almost guaranteed to be safe for > > anything that is able to run as a module, which SPI_NOR is. > > Specifically modules are already probed asynchronously. Also: since > > other things in the system may have enabled asynchronous probe the > > system may already be doing other things during our probe. > > > > There is a small possibility that some other driver that was a client > > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe > > ordering and only worked when the SPI_NOR and the SPI bus were > > builtin. In that case the other driver has a bug that's waiting to > > hit and the other driver should be fixed. > > linux-next now triggers the following warning in kernel/kmod.c:136 on my > board. I've bisected this to this patch. > > kmod.c: > /* > * We don't allow synchronous module loading from async. Module > * init may invoke async_synchronize_full() which will end up > * waiting for this task which already is waiting for the module > * loading to complete, leading to a deadlock. > */ > WARN_ON_ONCE(wait && current_is_async()); > > [ 1.849801] ------------[ cut here ]------------ > [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping > [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 __request_module+0x3a4/0x568 > [ 1.858755] Modules linked in: > [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 > [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted 5.9.0-rc6-00001-g03edda0e1eda #113 > [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT) > [ 1.872647] Workqueue: events_unbound async_run_entry_fn > [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) > [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) > [ 1.881297] pc : __request_module+0x3a4/0x568 > [ 1.881299] lr : __request_module+0x39c/0x568 > [ 1.881302] sp : ffff8000113a3920 > [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 > [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 > [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 > [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 > [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc > [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 > [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) > [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 > [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed > [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 > [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 > [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 > [ 1.975241] 10 fixed-partitions partitions found on MTD device 20c0000.spi > [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 > [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 > [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 > [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 > [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 > [ 1.979564] Call trace: > [ 1.979571] __request_module+0x3a4/0x568 > [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": > [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 > [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 > [ 1.997133] 0x000000000000-0x000000010000 : "rcw" > [ 2.002454] spi_nor_probe+0x29c/0x2f0 > [ 2.002458] spi_mem_probe+0x74/0xb0 > [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" > [ 2.018433] spi_drv_probe+0x88/0xe8 > [ 2.018439] really_probe+0xec/0x3c0 > [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" > [ 2.035555] driver_probe_device+0x60/0xc0 > [ 2.035559] __device_attach_driver+0x8c/0xd0 > [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted firmware" > [ 2.044642] bus_for_each_drv+0x84/0xd8 > [ 2.044645] __device_attach_async_helper+0xc4/0xe8 > [ 2.044648] async_run_entry_fn+0x4c/0x150 > [ 2.044653] process_one_work+0x1f4/0x4b8 > [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" > [ 2.062814] worker_thread+0x50/0x480 > [ 2.062817] kthread+0x160/0x168 > [ 2.062821] ret_from_fork+0x10/0x34 > [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" > [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- > > -michael Thanks for your report! My vote would be to revert my patch and then this would need to be resolved before it could be added back in. Without doing tons of research, maybe the right answer here is that mtd_device_parse_register() should be moved into a separate task so it's not blocking probe? I probably won't try to tackle this immediately, but the eventual goal is that async is default, so I think this would need to be resolved before then. -Doug ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-03 16:27 ` Doug Anderson 0 siblings, 0 replies; 18+ messages in thread From: Doug Anderson @ 2020-10-03 16:27 UTC (permalink / raw) To: Michael Walle Cc: vigneshr, tudor.ambarus, richard, LKML, linux-mtd, miquel.raynal Hi, On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: > > Hi Douglas, > > > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > > lot, but every little bit adds up to a slow bootup. While we can get > > this out of the boot path by making it a module, there are times where > > it is convenient (or even required) for this to be builtin the kernel. > > Let's set that we prefer async probe so that we don't block other > > drivers from probing while we are probing. > > > > This is a tiny little change that is almost guaranteed to be safe for > > anything that is able to run as a module, which SPI_NOR is. > > Specifically modules are already probed asynchronously. Also: since > > other things in the system may have enabled asynchronous probe the > > system may already be doing other things during our probe. > > > > There is a small possibility that some other driver that was a client > > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe > > ordering and only worked when the SPI_NOR and the SPI bus were > > builtin. In that case the other driver has a bug that's waiting to > > hit and the other driver should be fixed. > > linux-next now triggers the following warning in kernel/kmod.c:136 on my > board. I've bisected this to this patch. > > kmod.c: > /* > * We don't allow synchronous module loading from async. Module > * init may invoke async_synchronize_full() which will end up > * waiting for this task which already is waiting for the module > * loading to complete, leading to a deadlock. > */ > WARN_ON_ONCE(wait && current_is_async()); > > [ 1.849801] ------------[ cut here ]------------ > [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping > [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 __request_module+0x3a4/0x568 > [ 1.858755] Modules linked in: > [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 > [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted 5.9.0-rc6-00001-g03edda0e1eda #113 > [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on SMARC Eval 2.0 carrier (DT) > [ 1.872647] Workqueue: events_unbound async_run_entry_fn > [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) > [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) > [ 1.881297] pc : __request_module+0x3a4/0x568 > [ 1.881299] lr : __request_module+0x39c/0x568 > [ 1.881302] sp : ffff8000113a3920 > [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 > [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 > [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 > [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 > [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc > [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 > [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) > [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 > [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed > [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 > [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 > [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 > [ 1.975241] 10 fixed-partitions partitions found on MTD device 20c0000.spi > [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 > [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 > [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 > [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 > [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 > [ 1.979564] Call trace: > [ 1.979571] __request_module+0x3a4/0x568 > [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": > [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 > [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 > [ 1.997133] 0x000000000000-0x000000010000 : "rcw" > [ 2.002454] spi_nor_probe+0x29c/0x2f0 > [ 2.002458] spi_mem_probe+0x74/0xb0 > [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" > [ 2.018433] spi_drv_probe+0x88/0xe8 > [ 2.018439] really_probe+0xec/0x3c0 > [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" > [ 2.035555] driver_probe_device+0x60/0xc0 > [ 2.035559] __device_attach_driver+0x8c/0xd0 > [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted firmware" > [ 2.044642] bus_for_each_drv+0x84/0xd8 > [ 2.044645] __device_attach_async_helper+0xc4/0xe8 > [ 2.044648] async_run_entry_fn+0x4c/0x150 > [ 2.044653] process_one_work+0x1f4/0x4b8 > [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" > [ 2.062814] worker_thread+0x50/0x480 > [ 2.062817] kthread+0x160/0x168 > [ 2.062821] ret_from_fork+0x10/0x34 > [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" > [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- > > -michael Thanks for your report! My vote would be to revert my patch and then this would need to be resolved before it could be added back in. Without doing tons of research, maybe the right answer here is that mtd_device_parse_register() should be moved into a separate task so it's not blocking probe? I probably won't try to tackle this immediately, but the eventual goal is that async is default, so I think this would need to be resolved before then. -Doug ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-10-03 16:27 ` Doug Anderson @ 2020-10-03 16:54 ` Michael Walle -1 siblings, 0 replies; 18+ messages in thread From: Michael Walle @ 2020-10-03 16:54 UTC (permalink / raw) To: Doug Anderson Cc: LKML, linux-mtd, miquel.raynal, richard, tudor.ambarus, vigneshr Hi Douglas, Am 2020-10-03 18:27, schrieb Doug Anderson: > Hi, > > On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: >> >> Hi Douglas, >> >> > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a >> > lot, but every little bit adds up to a slow bootup. While we can get >> > this out of the boot path by making it a module, there are times where >> > it is convenient (or even required) for this to be builtin the kernel. >> > Let's set that we prefer async probe so that we don't block other >> > drivers from probing while we are probing. >> > >> > This is a tiny little change that is almost guaranteed to be safe for >> > anything that is able to run as a module, which SPI_NOR is. >> > Specifically modules are already probed asynchronously. Also: since >> > other things in the system may have enabled asynchronous probe the >> > system may already be doing other things during our probe. >> > >> > There is a small possibility that some other driver that was a client >> > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe >> > ordering and only worked when the SPI_NOR and the SPI bus were >> > builtin. In that case the other driver has a bug that's waiting to >> > hit and the other driver should be fixed. >> >> linux-next now triggers the following warning in kernel/kmod.c:136 on >> my >> board. I've bisected this to this patch. >> >> kmod.c: >> /* >> * We don't allow synchronous module loading from async. >> Module >> * init may invoke async_synchronize_full() which will end up >> * waiting for this task which already is waiting for the >> module >> * loading to complete, leading to a deadlock. >> */ >> WARN_ON_ONCE(wait && current_is_async()); >> >> [ 1.849801] ------------[ cut here ]------------ >> [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping >> [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 >> __request_module+0x3a4/0x568 >> [ 1.858755] Modules linked in: >> [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 >> [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted >> 5.9.0-rc6-00001-g03edda0e1eda #113 >> [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on >> SMARC Eval 2.0 carrier (DT) >> [ 1.872647] Workqueue: events_unbound async_run_entry_fn >> [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) >> [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) >> [ 1.881297] pc : __request_module+0x3a4/0x568 >> [ 1.881299] lr : __request_module+0x39c/0x568 >> [ 1.881302] sp : ffff8000113a3920 >> [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 >> [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 >> [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 >> [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 >> [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc >> [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 >> [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) >> [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 >> [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed >> [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 >> [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 >> [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 >> [ 1.975241] 10 fixed-partitions partitions found on MTD device >> 20c0000.spi >> [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 >> [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 >> [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 >> [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 >> [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 >> [ 1.979564] Call trace: >> [ 1.979571] __request_module+0x3a4/0x568 >> [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": >> [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 >> [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 >> [ 1.997133] 0x000000000000-0x000000010000 : "rcw" >> [ 2.002454] spi_nor_probe+0x29c/0x2f0 >> [ 2.002458] spi_mem_probe+0x74/0xb0 >> [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" >> [ 2.018433] spi_drv_probe+0x88/0xe8 >> [ 2.018439] really_probe+0xec/0x3c0 >> [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" >> [ 2.035555] driver_probe_device+0x60/0xc0 >> [ 2.035559] __device_attach_driver+0x8c/0xd0 >> [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted >> firmware" >> [ 2.044642] bus_for_each_drv+0x84/0xd8 >> [ 2.044645] __device_attach_async_helper+0xc4/0xe8 >> [ 2.044648] async_run_entry_fn+0x4c/0x150 >> [ 2.044653] process_one_work+0x1f4/0x4b8 >> [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" >> [ 2.062814] worker_thread+0x50/0x480 >> [ 2.062817] kthread+0x160/0x168 >> [ 2.062821] ret_from_fork+0x10/0x34 >> [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" >> [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- >> >> -michael > > Thanks for your report! My vote would be to revert my patch and then > this would need to be resolved before it could be added back in. > Without doing tons of research, maybe the right answer here is that > mtd_device_parse_register() should be moved into a separate task so > it's not blocking probe? I probably won't try to tackle this > immediately, but the eventual goal is that async is default, so I > think this would need to be resolved before then. Ok. Vignesh, will you take care of that? While debugging another issue I also noticed that sometimes my /dev/mtdN devices were reordered. Note that I have two SPI flashes. Might this also be connected to the async probe? -michael ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-03 16:54 ` Michael Walle 0 siblings, 0 replies; 18+ messages in thread From: Michael Walle @ 2020-10-03 16:54 UTC (permalink / raw) To: Doug Anderson Cc: vigneshr, tudor.ambarus, richard, LKML, linux-mtd, miquel.raynal Hi Douglas, Am 2020-10-03 18:27, schrieb Doug Anderson: > Hi, > > On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: >> >> Hi Douglas, >> >> > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a >> > lot, but every little bit adds up to a slow bootup. While we can get >> > this out of the boot path by making it a module, there are times where >> > it is convenient (or even required) for this to be builtin the kernel. >> > Let's set that we prefer async probe so that we don't block other >> > drivers from probing while we are probing. >> > >> > This is a tiny little change that is almost guaranteed to be safe for >> > anything that is able to run as a module, which SPI_NOR is. >> > Specifically modules are already probed asynchronously. Also: since >> > other things in the system may have enabled asynchronous probe the >> > system may already be doing other things during our probe. >> > >> > There is a small possibility that some other driver that was a client >> > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe >> > ordering and only worked when the SPI_NOR and the SPI bus were >> > builtin. In that case the other driver has a bug that's waiting to >> > hit and the other driver should be fixed. >> >> linux-next now triggers the following warning in kernel/kmod.c:136 on >> my >> board. I've bisected this to this patch. >> >> kmod.c: >> /* >> * We don't allow synchronous module loading from async. >> Module >> * init may invoke async_synchronize_full() which will end up >> * waiting for this task which already is waiting for the >> module >> * loading to complete, leading to a deadlock. >> */ >> WARN_ON_ONCE(wait && current_is_async()); >> >> [ 1.849801] ------------[ cut here ]------------ >> [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping >> [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 >> __request_module+0x3a4/0x568 >> [ 1.858755] Modules linked in: >> [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 >> [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted >> 5.9.0-rc6-00001-g03edda0e1eda #113 >> [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on >> SMARC Eval 2.0 carrier (DT) >> [ 1.872647] Workqueue: events_unbound async_run_entry_fn >> [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) >> [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) >> [ 1.881297] pc : __request_module+0x3a4/0x568 >> [ 1.881299] lr : __request_module+0x39c/0x568 >> [ 1.881302] sp : ffff8000113a3920 >> [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 >> [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 >> [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 >> [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 >> [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc >> [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 >> [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) >> [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 >> [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed >> [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 >> [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 >> [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 >> [ 1.975241] 10 fixed-partitions partitions found on MTD device >> 20c0000.spi >> [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 >> [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 >> [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 >> [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 >> [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 >> [ 1.979564] Call trace: >> [ 1.979571] __request_module+0x3a4/0x568 >> [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": >> [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 >> [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 >> [ 1.997133] 0x000000000000-0x000000010000 : "rcw" >> [ 2.002454] spi_nor_probe+0x29c/0x2f0 >> [ 2.002458] spi_mem_probe+0x74/0xb0 >> [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" >> [ 2.018433] spi_drv_probe+0x88/0xe8 >> [ 2.018439] really_probe+0xec/0x3c0 >> [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" >> [ 2.035555] driver_probe_device+0x60/0xc0 >> [ 2.035559] __device_attach_driver+0x8c/0xd0 >> [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted >> firmware" >> [ 2.044642] bus_for_each_drv+0x84/0xd8 >> [ 2.044645] __device_attach_async_helper+0xc4/0xe8 >> [ 2.044648] async_run_entry_fn+0x4c/0x150 >> [ 2.044653] process_one_work+0x1f4/0x4b8 >> [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" >> [ 2.062814] worker_thread+0x50/0x480 >> [ 2.062817] kthread+0x160/0x168 >> [ 2.062821] ret_from_fork+0x10/0x34 >> [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" >> [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- >> >> -michael > > Thanks for your report! My vote would be to revert my patch and then > this would need to be resolved before it could be added back in. > Without doing tons of research, maybe the right answer here is that > mtd_device_parse_register() should be moved into a separate task so > it's not blocking probe? I probably won't try to tackle this > immediately, but the eventual goal is that async is default, so I > think this would need to be resolved before then. Ok. Vignesh, will you take care of that? While debugging another issue I also noticed that sometimes my /dev/mtdN devices were reordered. Note that I have two SPI flashes. Might this also be connected to the async probe? -michael ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-10-03 16:54 ` Michael Walle @ 2020-10-03 17:00 ` Doug Anderson -1 siblings, 0 replies; 18+ messages in thread From: Doug Anderson @ 2020-10-03 17:00 UTC (permalink / raw) To: Michael Walle Cc: LKML, linux-mtd, miquel.raynal, richard, tudor.ambarus, vigneshr Hi, On Sat, Oct 3, 2020 at 9:54 AM Michael Walle <michael@walle.cc> wrote: > > Hi Douglas, > > Am 2020-10-03 18:27, schrieb Doug Anderson: > > Hi, > > > > On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: > >> > >> Hi Douglas, > >> > >> > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > >> > lot, but every little bit adds up to a slow bootup. While we can get > >> > this out of the boot path by making it a module, there are times where > >> > it is convenient (or even required) for this to be builtin the kernel. > >> > Let's set that we prefer async probe so that we don't block other > >> > drivers from probing while we are probing. > >> > > >> > This is a tiny little change that is almost guaranteed to be safe for > >> > anything that is able to run as a module, which SPI_NOR is. > >> > Specifically modules are already probed asynchronously. Also: since > >> > other things in the system may have enabled asynchronous probe the > >> > system may already be doing other things during our probe. > >> > > >> > There is a small possibility that some other driver that was a client > >> > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe > >> > ordering and only worked when the SPI_NOR and the SPI bus were > >> > builtin. In that case the other driver has a bug that's waiting to > >> > hit and the other driver should be fixed. > >> > >> linux-next now triggers the following warning in kernel/kmod.c:136 on > >> my > >> board. I've bisected this to this patch. > >> > >> kmod.c: > >> /* > >> * We don't allow synchronous module loading from async. > >> Module > >> * init may invoke async_synchronize_full() which will end up > >> * waiting for this task which already is waiting for the > >> module > >> * loading to complete, leading to a deadlock. > >> */ > >> WARN_ON_ONCE(wait && current_is_async()); > >> > >> [ 1.849801] ------------[ cut here ]------------ > >> [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping > >> [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 > >> __request_module+0x3a4/0x568 > >> [ 1.858755] Modules linked in: > >> [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 > >> [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted > >> 5.9.0-rc6-00001-g03edda0e1eda #113 > >> [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on > >> SMARC Eval 2.0 carrier (DT) > >> [ 1.872647] Workqueue: events_unbound async_run_entry_fn > >> [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) > >> [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) > >> [ 1.881297] pc : __request_module+0x3a4/0x568 > >> [ 1.881299] lr : __request_module+0x39c/0x568 > >> [ 1.881302] sp : ffff8000113a3920 > >> [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 > >> [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 > >> [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 > >> [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 > >> [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc > >> [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 > >> [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) > >> [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 > >> [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed > >> [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 > >> [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 > >> [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 > >> [ 1.975241] 10 fixed-partitions partitions found on MTD device > >> 20c0000.spi > >> [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 > >> [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 > >> [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 > >> [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 > >> [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 > >> [ 1.979564] Call trace: > >> [ 1.979571] __request_module+0x3a4/0x568 > >> [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": > >> [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 > >> [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 > >> [ 1.997133] 0x000000000000-0x000000010000 : "rcw" > >> [ 2.002454] spi_nor_probe+0x29c/0x2f0 > >> [ 2.002458] spi_mem_probe+0x74/0xb0 > >> [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" > >> [ 2.018433] spi_drv_probe+0x88/0xe8 > >> [ 2.018439] really_probe+0xec/0x3c0 > >> [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" > >> [ 2.035555] driver_probe_device+0x60/0xc0 > >> [ 2.035559] __device_attach_driver+0x8c/0xd0 > >> [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted > >> firmware" > >> [ 2.044642] bus_for_each_drv+0x84/0xd8 > >> [ 2.044645] __device_attach_async_helper+0xc4/0xe8 > >> [ 2.044648] async_run_entry_fn+0x4c/0x150 > >> [ 2.044653] process_one_work+0x1f4/0x4b8 > >> [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" > >> [ 2.062814] worker_thread+0x50/0x480 > >> [ 2.062817] kthread+0x160/0x168 > >> [ 2.062821] ret_from_fork+0x10/0x34 > >> [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" > >> [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- > >> > >> -michael > > > > Thanks for your report! My vote would be to revert my patch and then > > this would need to be resolved before it could be added back in. > > Without doing tons of research, maybe the right answer here is that > > mtd_device_parse_register() should be moved into a separate task so > > it's not blocking probe? I probably won't try to tackle this > > immediately, but the eventual goal is that async is default, so I > > think this would need to be resolved before then. > > Ok. Vignesh, will you take care of that? > > While debugging another issue I also noticed that sometimes my > /dev/mtdN devices were reordered. Note that I have two SPI flashes. > Might this also be connected to the async probe? It's likely. My guess is that you shouldn't really be depending on the numbering. If you need to depend on the numbering, there should be something that guarantees it like a device tree alias. We have struggled with similar things on MMC for years and I guess Ulf finally decided that we weren't going to get a better solution than the device tree aliases. See: https://lore.kernel.org/r/CAPDyKFoNFEa4ssv3TXbj4KM0UGjJgcp3dtypwicxJ-bExBL28A@mail.gmail.com ...for some details. -Doug ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-03 17:00 ` Doug Anderson 0 siblings, 0 replies; 18+ messages in thread From: Doug Anderson @ 2020-10-03 17:00 UTC (permalink / raw) To: Michael Walle Cc: vigneshr, tudor.ambarus, richard, LKML, linux-mtd, miquel.raynal Hi, On Sat, Oct 3, 2020 at 9:54 AM Michael Walle <michael@walle.cc> wrote: > > Hi Douglas, > > Am 2020-10-03 18:27, schrieb Doug Anderson: > > Hi, > > > > On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: > >> > >> Hi Douglas, > >> > >> > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a > >> > lot, but every little bit adds up to a slow bootup. While we can get > >> > this out of the boot path by making it a module, there are times where > >> > it is convenient (or even required) for this to be builtin the kernel. > >> > Let's set that we prefer async probe so that we don't block other > >> > drivers from probing while we are probing. > >> > > >> > This is a tiny little change that is almost guaranteed to be safe for > >> > anything that is able to run as a module, which SPI_NOR is. > >> > Specifically modules are already probed asynchronously. Also: since > >> > other things in the system may have enabled asynchronous probe the > >> > system may already be doing other things during our probe. > >> > > >> > There is a small possibility that some other driver that was a client > >> > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe > >> > ordering and only worked when the SPI_NOR and the SPI bus were > >> > builtin. In that case the other driver has a bug that's waiting to > >> > hit and the other driver should be fixed. > >> > >> linux-next now triggers the following warning in kernel/kmod.c:136 on > >> my > >> board. I've bisected this to this patch. > >> > >> kmod.c: > >> /* > >> * We don't allow synchronous module loading from async. > >> Module > >> * init may invoke async_synchronize_full() which will end up > >> * waiting for this task which already is waiting for the > >> module > >> * loading to complete, leading to a deadlock. > >> */ > >> WARN_ON_ONCE(wait && current_is_async()); > >> > >> [ 1.849801] ------------[ cut here ]------------ > >> [ 1.854271] mscc_felix 0000:00:00.5: device is disabled, skipping > >> [ 1.858753] WARNING: CPU: 1 PID: 7 at kernel/kmod.c:136 > >> __request_module+0x3a4/0x568 > >> [ 1.858755] Modules linked in: > >> [ 1.865028] fsl_enetc 0000:00:00.0: Adding to iommu group 1 > >> [ 1.872640] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted > >> 5.9.0-rc6-00001-g03edda0e1eda #113 > >> [ 1.872642] Hardware name: Kontron SMARC-sAL28 (Single PHY) on > >> SMARC Eval 2.0 carrier (DT) > >> [ 1.872647] Workqueue: events_unbound async_run_entry_fn > >> [ 1.876013] spi-nor spi0.0: w25q32dw (4096 Kbytes) > >> [ 1.881294] pstate: 00000005 (nzcv daif -PAN -UAO BTYPE=--) > >> [ 1.881297] pc : __request_module+0x3a4/0x568 > >> [ 1.881299] lr : __request_module+0x39c/0x568 > >> [ 1.881302] sp : ffff8000113a3920 > >> [ 1.925739] x29: ffff8000113a3920 x28: ffff800010c7b000 > >> [ 1.931068] x27: ffff00207ae05648 x26: ffff800010a41a88 > >> [ 1.936397] x25: 0000000000000000 x24: 0000000000000000 > >> [ 1.941727] x23: ffff800010c35140 x22: 0000000000000001 > >> [ 1.947055] x21: ffff800011149948 x20: ffff800010615bdc > >> [ 1.952383] x19: 00000000ffffffff x18: 0000000000000000 > >> [ 1.957447] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) > >> [ 1.957711] x17: ffff800010a3e618 x16: ffff800010a3e5f8 > >> [ 1.964175] libphy: Freescale ENETC MDIO Bus: probed > >> [ 1.969238] x15: ffffffffffffffff x14: ffff800011149948 > >> [ 1.969241] x13: ffff8000113a3918 x12: 0000000000000018 > >> [ 1.969245] x11: 0000000000000005 x10: 0101010101010101 > >> [ 1.975241] 10 fixed-partitions partitions found on MTD device > >> 20c0000.spi > >> [ 1.979550] x9 : ffff80001005f6a4 x8 : 0000000000000000 > >> [ 1.979553] x7 : 606f2c6364776865 x6 : 05041c090d431511 > >> [ 1.979556] x5 : 1115430d091c0405 x4 : 0000000000000000 > >> [ 1.979558] x3 : 6dac8d8d2dccae00 x2 : ffff800010c956e8 > >> [ 1.979561] x1 : ffff80001005fa58 x0 : 0000000000000001 > >> [ 1.979564] Call trace: > >> [ 1.979571] __request_module+0x3a4/0x568 > >> [ 1.984914] Creating 10 MTD partitions on "20c0000.spi": > >> [ 1.990227] parse_mtd_partitions+0x2ec/0x3c0 > >> [ 1.990232] mtd_device_parse_register+0xdc/0x1c8 > >> [ 1.997133] 0x000000000000-0x000000010000 : "rcw" > >> [ 2.002454] spi_nor_probe+0x29c/0x2f0 > >> [ 2.002458] spi_mem_probe+0x74/0xb0 > >> [ 2.017759] 0x000000010000-0x000000100000 : "failsafe bootloader" > >> [ 2.018433] spi_drv_probe+0x88/0xe8 > >> [ 2.018439] really_probe+0xec/0x3c0 > >> [ 2.033744] 0x000000100000-0x000000140000 : "failsafe DP firmware" > >> [ 2.035555] driver_probe_device+0x60/0xc0 > >> [ 2.035559] __device_attach_driver+0x8c/0xd0 > >> [ 2.040455] 0x000000140000-0x0000001e0000 : "failsafe trusted > >> firmware" > >> [ 2.044642] bus_for_each_drv+0x84/0xd8 > >> [ 2.044645] __device_attach_async_helper+0xc4/0xe8 > >> [ 2.044648] async_run_entry_fn+0x4c/0x150 > >> [ 2.044653] process_one_work+0x1f4/0x4b8 > >> [ 2.057751] 0x0000001e0000-0x000000200000 : "reserved" > >> [ 2.062814] worker_thread+0x50/0x480 > >> [ 2.062817] kthread+0x160/0x168 > >> [ 2.062821] ret_from_fork+0x10/0x34 > >> [ 2.073748] 0x000000200000-0x000000210000 : "configuration store" > >> [ 2.076185] ---[ end trace 44224cc02e4e53d2 ]--- > >> > >> -michael > > > > Thanks for your report! My vote would be to revert my patch and then > > this would need to be resolved before it could be added back in. > > Without doing tons of research, maybe the right answer here is that > > mtd_device_parse_register() should be moved into a separate task so > > it's not blocking probe? I probably won't try to tackle this > > immediately, but the eventual goal is that async is default, so I > > think this would need to be resolved before then. > > Ok. Vignesh, will you take care of that? > > While debugging another issue I also noticed that sometimes my > /dev/mtdN devices were reordered. Note that I have two SPI flashes. > Might this also be connected to the async probe? It's likely. My guess is that you shouldn't really be depending on the numbering. If you need to depend on the numbering, there should be something that guarantees it like a device tree alias. We have struggled with similar things on MMC for years and I guess Ulf finally decided that we weren't going to get a better solution than the device tree aliases. See: https://lore.kernel.org/r/CAPDyKFoNFEa4ssv3TXbj4KM0UGjJgcp3dtypwicxJ-bExBL28A@mail.gmail.com ...for some details. -Doug ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-10-03 17:00 ` Doug Anderson @ 2020-10-04 14:10 ` Michael Walle -1 siblings, 0 replies; 18+ messages in thread From: Michael Walle @ 2020-10-04 14:10 UTC (permalink / raw) To: Doug Anderson Cc: LKML, linux-mtd, miquel.raynal, richard, tudor.ambarus, vigneshr Hi, Am 2020-10-03 19:00, schrieb Doug Anderson: > On Sat, Oct 3, 2020 at 9:54 AM Michael Walle <michael@walle.cc> wrote: >> While debugging another issue I also noticed that sometimes my >> /dev/mtdN devices were reordered. Note that I have two SPI flashes. >> Might this also be connected to the async probe? > > It's likely. My guess is that you shouldn't really be depending on > the numbering. If you need to depend on the numbering, there should > be something that guarantees it like a device tree alias. We have > struggled with similar things on MMC for years and I guess Ulf finally > decided that we weren't going to get a better solution than the device > tree aliases. But this has to be supported by spi-nor first, right? So that would also be something which has to be added before we can make the probe async. And as far as I know there is no such mechanism like /dev/disk/by-X for /dev/mtdN. -michael ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-04 14:10 ` Michael Walle 0 siblings, 0 replies; 18+ messages in thread From: Michael Walle @ 2020-10-04 14:10 UTC (permalink / raw) To: Doug Anderson Cc: vigneshr, tudor.ambarus, richard, LKML, linux-mtd, miquel.raynal Hi, Am 2020-10-03 19:00, schrieb Doug Anderson: > On Sat, Oct 3, 2020 at 9:54 AM Michael Walle <michael@walle.cc> wrote: >> While debugging another issue I also noticed that sometimes my >> /dev/mtdN devices were reordered. Note that I have two SPI flashes. >> Might this also be connected to the async probe? > > It's likely. My guess is that you shouldn't really be depending on > the numbering. If you need to depend on the numbering, there should > be something that guarantees it like a device tree alias. We have > struggled with similar things on MMC for years and I guess Ulf finally > decided that we weren't going to get a better solution than the device > tree aliases. But this has to be supported by spi-nor first, right? So that would also be something which has to be added before we can make the probe async. And as far as I know there is no such mechanism like /dev/disk/by-X for /dev/mtdN. -michael ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-10-04 14:10 ` Michael Walle @ 2020-10-05 14:53 ` Doug Anderson -1 siblings, 0 replies; 18+ messages in thread From: Doug Anderson @ 2020-10-05 14:53 UTC (permalink / raw) To: Michael Walle Cc: LKML, linux-mtd, miquel.raynal, richard, Tudor Ambarus, Vignesh Raghavendra Hi, On Sun, Oct 4, 2020 at 7:10 AM Michael Walle <michael@walle.cc> wrote: > > Hi, > > Am 2020-10-03 19:00, schrieb Doug Anderson: > > On Sat, Oct 3, 2020 at 9:54 AM Michael Walle <michael@walle.cc> wrote: > >> While debugging another issue I also noticed that sometimes my > >> /dev/mtdN devices were reordered. Note that I have two SPI flashes. > >> Might this also be connected to the async probe? > > > > It's likely. My guess is that you shouldn't really be depending on > > the numbering. If you need to depend on the numbering, there should > > be something that guarantees it like a device tree alias. We have > > struggled with similar things on MMC for years and I guess Ulf finally > > decided that we weren't going to get a better solution than the device > > tree aliases. > > But this has to be supported by spi-nor first, right? So that would also > be something which has to be added before we can make the probe async. > And as far as I know there is no such mechanism like /dev/disk/by-X for > /dev/mtdN. I did a quick skim and didn't see any mechanism like this for the MTD layer, so I think you're right. Something like this would almost certainly need to be added. As per the discussion in the MMC threads about this, though, probably whatever you have working right now is "by luck". It may be stable from boot to boot, but I don't think there are any guarantees that a new kernel version won't come along and shuffle your IDs around. Presumably it would be a good idea for MTD folks to come up with some sort of solution here... -Doug ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-05 14:53 ` Doug Anderson 0 siblings, 0 replies; 18+ messages in thread From: Doug Anderson @ 2020-10-05 14:53 UTC (permalink / raw) To: Michael Walle Cc: Vignesh Raghavendra, Tudor Ambarus, richard, LKML, linux-mtd, miquel.raynal Hi, On Sun, Oct 4, 2020 at 7:10 AM Michael Walle <michael@walle.cc> wrote: > > Hi, > > Am 2020-10-03 19:00, schrieb Doug Anderson: > > On Sat, Oct 3, 2020 at 9:54 AM Michael Walle <michael@walle.cc> wrote: > >> While debugging another issue I also noticed that sometimes my > >> /dev/mtdN devices were reordered. Note that I have two SPI flashes. > >> Might this also be connected to the async probe? > > > > It's likely. My guess is that you shouldn't really be depending on > > the numbering. If you need to depend on the numbering, there should > > be something that guarantees it like a device tree alias. We have > > struggled with similar things on MMC for years and I guess Ulf finally > > decided that we weren't going to get a better solution than the device > > tree aliases. > > But this has to be supported by spi-nor first, right? So that would also > be something which has to be added before we can make the probe async. > And as far as I know there is no such mechanism like /dev/disk/by-X for > /dev/mtdN. I did a quick skim and didn't see any mechanism like this for the MTD layer, so I think you're right. Something like this would almost certainly need to be added. As per the discussion in the MMC threads about this, though, probably whatever you have working right now is "by luck". It may be stable from boot to boot, but I don't think there are any guarantees that a new kernel version won't come along and shuffle your IDs around. Presumably it would be a good idea for MTD folks to come up with some sort of solution here... -Doug ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe 2020-10-03 16:54 ` Michael Walle @ 2020-10-05 9:06 ` Vignesh Raghavendra -1 siblings, 0 replies; 18+ messages in thread From: Vignesh Raghavendra @ 2020-10-05 9:06 UTC (permalink / raw) To: Michael Walle, Doug Anderson Cc: LKML, linux-mtd, miquel.raynal, richard, tudor.ambarus Hi Michael, On 10/3/20 10:24 PM, Michael Walle wrote: > Hi Douglas, > > Am 2020-10-03 18:27, schrieb Doug Anderson: >> Hi, >> >> On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: >>> >>> Hi Douglas, >>> >>> > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a >>> > lot, but every little bit adds up to a slow bootup. While we can get >>> > this out of the boot path by making it a module, there are times where >>> > it is convenient (or even required) for this to be builtin the kernel. >>> > Let's set that we prefer async probe so that we don't block other >>> > drivers from probing while we are probing. >>> > >>> > This is a tiny little change that is almost guaranteed to be safe for >>> > anything that is able to run as a module, which SPI_NOR is. >>> > Specifically modules are already probed asynchronously. Also: since >>> > other things in the system may have enabled asynchronous probe the >>> > system may already be doing other things during our probe. >>> > >>> > There is a small possibility that some other driver that was a client >>> > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe >>> > ordering and only worked when the SPI_NOR and the SPI bus were >>> > builtin. In that case the other driver has a bug that's waiting to >>> > hit and the other driver should be fixed. >>> >>> linux-next now triggers the following warning in kernel/kmod.c:136 on my >>> board. I've bisected this to this patch. >>> [...] >> >> Thanks for your report! My vote would be to revert my patch and then >> this would need to be resolved before it could be added back in. >> Without doing tons of research, maybe the right answer here is that >> mtd_device_parse_register() should be moved into a separate task so >> it's not blocking probe? I probably won't try to tackle this >> immediately, but the eventual goal is that async is default, so I >> think this would need to be resolved before then. > > Ok. Vignesh, will you take care of that? Thanks for the report! I have posted a patch reverting this commit. Will merge into spi-nor/next shortly Regards Vignesh ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH] mtd: spi-nor: Prefer asynchronous probe @ 2020-10-05 9:06 ` Vignesh Raghavendra 0 siblings, 0 replies; 18+ messages in thread From: Vignesh Raghavendra @ 2020-10-05 9:06 UTC (permalink / raw) To: Michael Walle, Doug Anderson Cc: richard, tudor.ambarus, linux-mtd, LKML, miquel.raynal Hi Michael, On 10/3/20 10:24 PM, Michael Walle wrote: > Hi Douglas, > > Am 2020-10-03 18:27, schrieb Doug Anderson: >> Hi, >> >> On Sat, Oct 3, 2020 at 8:22 AM Michael Walle <michael@walle.cc> wrote: >>> >>> Hi Douglas, >>> >>> > On my system the spi_nor_probe() took ~6 ms at bootup. That's not a >>> > lot, but every little bit adds up to a slow bootup. While we can get >>> > this out of the boot path by making it a module, there are times where >>> > it is convenient (or even required) for this to be builtin the kernel. >>> > Let's set that we prefer async probe so that we don't block other >>> > drivers from probing while we are probing. >>> > >>> > This is a tiny little change that is almost guaranteed to be safe for >>> > anything that is able to run as a module, which SPI_NOR is. >>> > Specifically modules are already probed asynchronously. Also: since >>> > other things in the system may have enabled asynchronous probe the >>> > system may already be doing other things during our probe. >>> > >>> > There is a small possibility that some other driver that was a client >>> > of SPI_NOR didn't handle -EPROBE_DEFER and was relying on probe >>> > ordering and only worked when the SPI_NOR and the SPI bus were >>> > builtin. In that case the other driver has a bug that's waiting to >>> > hit and the other driver should be fixed. >>> >>> linux-next now triggers the following warning in kernel/kmod.c:136 on my >>> board. I've bisected this to this patch. >>> [...] >> >> Thanks for your report! My vote would be to revert my patch and then >> this would need to be resolved before it could be added back in. >> Without doing tons of research, maybe the right answer here is that >> mtd_device_parse_register() should be moved into a separate task so >> it's not blocking probe? I probably won't try to tackle this >> immediately, but the eventual goal is that async is default, so I >> think this would need to be resolved before then. > > Ok. Vignesh, will you take care of that? Thanks for the report! I have posted a patch reverting this commit. Will merge into spi-nor/next shortly Regards Vignesh ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2020-10-05 14:54 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-09-02 23:00 [PATCH] mtd: spi-nor: Prefer asynchronous probe Douglas Anderson 2020-09-02 23:00 ` Douglas Anderson 2020-09-30 8:34 ` Vignesh Raghavendra 2020-09-30 8:34 ` Vignesh Raghavendra 2020-10-03 15:06 ` Michael Walle 2020-10-03 15:06 ` Michael Walle 2020-10-03 16:27 ` Doug Anderson 2020-10-03 16:27 ` Doug Anderson 2020-10-03 16:54 ` Michael Walle 2020-10-03 16:54 ` Michael Walle 2020-10-03 17:00 ` Doug Anderson 2020-10-03 17:00 ` Doug Anderson 2020-10-04 14:10 ` Michael Walle 2020-10-04 14:10 ` Michael Walle 2020-10-05 14:53 ` Doug Anderson 2020-10-05 14:53 ` Doug Anderson 2020-10-05 9:06 ` Vignesh Raghavendra 2020-10-05 9:06 ` Vignesh Raghavendra
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.