* INFO: memory leak in gpio device in 6.2-rc6
@ 2023-01-31 9:36 Mirsad Goran Todorovac
2023-02-08 19:55 ` INFO: REPRODUCED: " Mirsad Goran Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-01-31 9:36 UTC (permalink / raw)
To: linux-gpio
Cc: Linus Walleij, Bartosz Golaszewski, linux-kernel, Thorsten Leemhuis
Hi all,
I came across this memory leak apparently in the GPIO device driver.
It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
enabled.
If you think this bug is significant, I can attempt the bug bisect in the
environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
Here is the /sys/kernel/debug/kmemleak output:
unreferenced object 0xffff9e67ad71f160 (size 32):
comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
[<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
[<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
[<00000000bd682ecc>] kvasprintf+0x6b/0xd0
[<00000000a3431d55>] kasprintf+0x4e/0x70
[<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<00000000673fc6df>] configfs_write_iter+0xcc/0x130
[<000000001d5d0829>] vfs_write+0x2b4/0x3d0
[<00000000d2336251>] ksys_write+0x61/0xe0
[<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
[<000000008ac743d2>] do_syscall_64+0x58/0x80
[<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[root@pc-mtodorov marvin]#
Regards,
Mirsad
--
Mirsad Todorovac
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb
Republic of Croatia, the European Union
--
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-01-31 9:36 INFO: memory leak in gpio device in 6.2-rc6 Mirsad Goran Todorovac
@ 2023-02-08 19:55 ` Mirsad Goran Todorovac
2023-02-12 14:19 ` Andy Shevchenko
0 siblings, 1 reply; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-08 19:55 UTC (permalink / raw)
To: linux-gpio
Cc: Linus Walleij, Bartosz Golaszewski, linux-kernel,
Thorsten Leemhuis, Bartosz Golaszewski
[-- Attachment #1: Type: text/plain, Size: 6736 bytes --]
Hi all,
On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
> Hi all,
>
> I came across this memory leak apparently in the GPIO device driver.
> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
>
> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
> enabled.
>
> If you think this bug is significant, I can attempt the bug bisect in the
> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
>
> Here is the /sys/kernel/debug/kmemleak output:
>
> unreferenced object 0xffff9e67ad71f160 (size 32):
> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
> hex dump (first 32 bytes):
> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
> [<00000000a3431d55>] kasprintf+0x4e/0x70
> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
> [<00000000d2336251>] ksys_write+0x61/0xe0
> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
> [<000000008ac743d2>] do_syscall_64+0x58/0x80
> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> [root@pc-mtodorov marvin]#
The new development on the bug is that it probably requires some superuser privileges
or some capability to be exploited, for it requires access to configfs,
but it was reproduced on different hardware as well.
The minimum reproducing script is attached, with its output log.
From the testing "age" of the unreferenced object I assume that it is allocated earlier
in a part of script I am unable to locate or specify, but orphaned at the end of the script.
root@/home/user/kernel_bugs/gpio-sim# time ./gpio-reproduce-min.sh
2.14. Lines can be hogged
Scanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.7 clean.
Rescanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
unreferenced object 0xffff9593b9d16bc0 (size 32):
comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 136.184s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
real 2m16.398s
user 0m0.023s
sys 0m16.360s
root@/home/user/kernel_bugs/gpio-sim#
The effect is cummulative:
root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim# cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff9593b9d16bc0 (size 32):
comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 520.296s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
unreferenced object 0xffff95938918fb40 (size 32):
comm "gpio-reproduce-", pid 7675, jiffies 4295954327 (age 164.832s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
unreferenced object 0xffff9594a3cf1820 (size 32):
comm "gpio-reproduce-", pid 7721, jiffies 4295976853 (age 74.728s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
[<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
[<00000000c4efe87f>] kvasprintf+0x6b/0xd0
[<000000000c0f91cd>] kasprintf+0x4e/0x70
[<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
[<0000000052ce6759>] configfs_write_iter+0xcc/0x130
[<0000000006087fd2>] vfs_write+0x2b4/0x3d0
[<000000008a17e041>] ksys_write+0x61/0xe0
[<000000007bded8ea>] __x64_sys_write+0x1a/0x20
[<00000000e1220148>] do_syscall_64+0x58/0x80
[<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim#
With sufficient privileges (or possibly even without them), an exploit can be devised
to automate allocation of orphaned objects, at a rate of a couple per second (PoC attached).
Regards,
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
[-- Attachment #2: gpio-trace.log --]
[-- Type: text/x-log, Size: 2418 bytes --]
2.14. Lines can be hogged
Scanning stage 2.14.0 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.0 clean.
Rescanning stage 2.14.0 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.0 re-check clean.
Re-rescanning stage 2.14.0 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.0 re-check clean.
Scanning stage 2.14.1 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.1 clean.
Rescanning stage 2.14.1 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.1 re-check clean.
Re-rescanning stage 2.14.1 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.1 re-check clean.
Scanning stage 2.14.2 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.2 clean.
Rescanning stage 2.14.2 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.2 re-check clean.
Re-rescanning stage 2.14.2 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.2 re-check clean.
Scanning stage 2.14.3 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.3 clean.
Rescanning stage 2.14.3 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.3 re-check clean.
Re-rescanning stage 2.14.3 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.3 re-check clean.
Scanning stage 2.14.4 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.4 clean.
Rescanning stage 2.14.4 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.4 re-check clean.
Re-rescanning stage 2.14.4 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.4 re-check clean.
Scanning stage 2.14.5 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.5 clean.
Rescanning stage 2.14.5 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.5 re-check clean.
Re-rescanning stage 2.14.5 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.5 re-check clean.
Scanning stage 2.14.6 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.6 clean.
Rescanning stage 2.14.6 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.6 re-check clean.
Re-rescanning stage 2.14.6 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.6 re-check clean.
Scanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
Stage 2.14.7 clean.
Rescanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
comm "gpio-sim-reprod", pid 74359, jiffies 4317617873 (age 531.940s)
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
Re-rescanning stage 2.14.7 ... done.
Sleeping 60 seconds ... done.
comm "gpio-sim-reprod", pid 74359, jiffies 4317617873 (age 598.480s)
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
[-- Attachment #3: gpio-reproduce-min.sh --]
[-- Type: application/x-shellscript, Size: 5067 bytes --]
[-- Attachment #4: gpio-min-iterate.sh --]
[-- Type: application/x-shellscript, Size: 5166 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-08 19:55 ` INFO: REPRODUCED: " Mirsad Goran Todorovac
@ 2023-02-12 14:19 ` Andy Shevchenko
2023-02-14 18:19 ` Mirsad Goran Todorovac
2023-02-14 20:54 ` INFO: BISECTED: " Mirsad Goran Todorovac
0 siblings, 2 replies; 23+ messages in thread
From: Andy Shevchenko @ 2023-02-12 14:19 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: linux-gpio, Linus Walleij, Bartosz Golaszewski, linux-kernel,
Thorsten Leemhuis
On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
> > I came across this memory leak apparently in the GPIO device driver.
> > It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
> >
> > This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
> > enabled.
> >
> > If you think this bug is significant, I can attempt the bug bisect in the
> > environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
> > with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
> >
> > Here is the /sys/kernel/debug/kmemleak output:
> >
> > unreferenced object 0xffff9e67ad71f160 (size 32):
> > comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
> > hex dump (first 32 bytes):
> > 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> > backtrace:
> > [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
> > [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
> > [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
> > [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
> > [<00000000a3431d55>] kasprintf+0x4e/0x70
> > [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> > [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
> > [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
> > [<00000000d2336251>] ksys_write+0x61/0xe0
> > [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
> > [<000000008ac743d2>] do_syscall_64+0x58/0x80
> > [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> > [root@pc-mtodorov marvin]#
>
> The new development on the bug is that it probably requires some superuser privileges
> or some capability to be exploited, for it requires access to configfs,
> but it was reproduced on different hardware as well.
>
> The minimum reproducing script is attached, with its output log.
>
> From the testing "age" of the unreferenced object I assume that it is allocated earlier
> in a part of script I am unable to locate or specify, but orphaned at the end of the script.
>
> root@/home/user/kernel_bugs/gpio-sim# time ./gpio-reproduce-min.sh
> 2.14. Lines can be hogged
> Scanning stage 2.14.7 ... done.
> Sleeping 60 seconds ... done.
> Stage 2.14.7 clean.
> Rescanning stage 2.14.7 ... done.
> Sleeping 60 seconds ... done.
> unreferenced object 0xffff9593b9d16bc0 (size 32):
> comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 136.184s)
> hex dump (first 32 bytes):
> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
> [<000000000c0f91cd>] kasprintf+0x4e/0x70
> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
> [<000000008a17e041>] ksys_write+0x61/0xe0
> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
> [<00000000e1220148>] do_syscall_64+0x58/0x80
> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>
> real 2m16.398s
> user 0m0.023s
> sys 0m16.360s
> root@/home/user/kernel_bugs/gpio-sim#
>
> The effect is cummulative:
>
> root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim# cat /sys/kernel/debug/kmemleak
> unreferenced object 0xffff9593b9d16bc0 (size 32):
> comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 520.296s)
> hex dump (first 32 bytes):
> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
> [<000000000c0f91cd>] kasprintf+0x4e/0x70
> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
> [<000000008a17e041>] ksys_write+0x61/0xe0
> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
> [<00000000e1220148>] do_syscall_64+0x58/0x80
> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> unreferenced object 0xffff95938918fb40 (size 32):
> comm "gpio-reproduce-", pid 7675, jiffies 4295954327 (age 164.832s)
> hex dump (first 32 bytes):
> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
> [<000000000c0f91cd>] kasprintf+0x4e/0x70
> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
> [<000000008a17e041>] ksys_write+0x61/0xe0
> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
> [<00000000e1220148>] do_syscall_64+0x58/0x80
> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> unreferenced object 0xffff9594a3cf1820 (size 32):
> comm "gpio-reproduce-", pid 7721, jiffies 4295976853 (age 74.728s)
> hex dump (first 32 bytes):
> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
> [<000000000c0f91cd>] kasprintf+0x4e/0x70
> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
> [<000000008a17e041>] ksys_write+0x61/0xe0
> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
> [<00000000e1220148>] do_syscall_64+0x58/0x80
> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim#
>
> With sufficient privileges (or possibly even without them), an exploit can be devised
> to automate allocation of orphaned objects, at a rate of a couple per second (PoC attached).
Looking into the code I found one inconsistency. It might be that that brings
an issue, dunno. Can you try the patch below if it helps (but TBH I'm a bit
sceptical)?
From 499cfb52aa7de67a8bbb56ce183d9528b2376db0 Mon Sep 17 00:00:00 2001
From: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Date: Sun, 12 Feb 2023 16:15:00 +0200
Subject: [PATCH 1/1] gpio: sim: Deactivate device in reversed order
Run the steps to deactivate device in the reserved order to what
it has been done in gpio_sim_device_activate_unlocked().
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
drivers/gpio/gpio-sim.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
index a51b5ea38ad5..b0111d18808c 100644
--- a/drivers/gpio/gpio-sim.c
+++ b/drivers/gpio/gpio-sim.c
@@ -953,9 +953,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
swnode = dev_fwnode(&dev->pdev->dev);
platform_device_unregister(dev->pdev);
+ gpio_sim_remove_hogs(dev);
gpio_sim_remove_swnode_recursive(swnode);
dev->pdev = NULL;
- gpio_sim_remove_hogs(dev);
}
static ssize_t
--
2.39.1
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-12 14:19 ` Andy Shevchenko
@ 2023-02-14 18:19 ` Mirsad Goran Todorovac
2023-02-15 10:53 ` Andy Shevchenko
2023-02-14 20:54 ` INFO: BISECTED: " Mirsad Goran Todorovac
1 sibling, 1 reply; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-14 18:19 UTC (permalink / raw)
To: Andy Shevchenko
Cc: linux-gpio, Linus Walleij, Bartosz Golaszewski, linux-kernel,
Thorsten Leemhuis
On 12. 02. 2023. 15:19, Andy Shevchenko wrote:
> On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
>> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
>>> I came across this memory leak apparently in the GPIO device driver.
>>> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
>>>
>>> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
>>> enabled.
>>>
>>> If you think this bug is significant, I can attempt the bug bisect in the
>>> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
>>> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
>>>
>>> Here is the /sys/kernel/debug/kmemleak output:
>>>
>>> unreferenced object 0xffff9e67ad71f160 (size 32):
>>> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
>>> hex dump (first 32 bytes):
>>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> backtrace:
>>> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
>>> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
>>> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
>>> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
>>> [<00000000a3431d55>] kasprintf+0x4e/0x70
>>> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>>> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
>>> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
>>> [<00000000d2336251>] ksys_write+0x61/0xe0
>>> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
>>> [<000000008ac743d2>] do_syscall_64+0x58/0x80
>>> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>> [root@pc-mtodorov marvin]#
>>
>> The new development on the bug is that it probably requires some superuser privileges
>> or some capability to be exploited, for it requires access to configfs,
>> but it was reproduced on different hardware as well.
>>
>> The minimum reproducing script is attached, with its output log.
>>
>> From the testing "age" of the unreferenced object I assume that it is allocated earlier
>> in a part of script I am unable to locate or specify, but orphaned at the end of the script.
>>
>> root@/home/user/kernel_bugs/gpio-sim# time ./gpio-reproduce-min.sh
>> 2.14. Lines can be hogged
>> Scanning stage 2.14.7 ... done.
>> Sleeping 60 seconds ... done.
>> Stage 2.14.7 clean.
>> Rescanning stage 2.14.7 ... done.
>> Sleeping 60 seconds ... done.
>> unreferenced object 0xffff9593b9d16bc0 (size 32):
>> comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 136.184s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>
>> real 2m16.398s
>> user 0m0.023s
>> sys 0m16.360s
>> root@/home/user/kernel_bugs/gpio-sim#
>>
>> The effect is cummulative:
>>
>> root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim# cat /sys/kernel/debug/kmemleak
>> unreferenced object 0xffff9593b9d16bc0 (size 32):
>> comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 520.296s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>> unreferenced object 0xffff95938918fb40 (size 32):
>> comm "gpio-reproduce-", pid 7675, jiffies 4295954327 (age 164.832s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>> unreferenced object 0xffff9594a3cf1820 (size 32):
>> comm "gpio-reproduce-", pid 7721, jiffies 4295976853 (age 74.728s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>> root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim#
>>
>> With sufficient privileges (or possibly even without them), an exploit can be devised
>> to automate allocation of orphaned objects, at a rate of a couple per second (PoC attached).
>
> Looking into the code I found one inconsistency. It might be that that brings
> an issue, dunno. Can you try the patch below if it helps (but TBH I'm a bit
> sceptical)?
>
>>From 499cfb52aa7de67a8bbb56ce183d9528b2376db0 Mon Sep 17 00:00:00 2001
> From: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Date: Sun, 12 Feb 2023 16:15:00 +0200
> Subject: [PATCH 1/1] gpio: sim: Deactivate device in reversed order
>
> Run the steps to deactivate device in the reserved order to what
> it has been done in gpio_sim_device_activate_unlocked().
>
> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> ---
> drivers/gpio/gpio-sim.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
> index a51b5ea38ad5..b0111d18808c 100644
> --- a/drivers/gpio/gpio-sim.c
> +++ b/drivers/gpio/gpio-sim.c
> @@ -953,9 +953,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
>
> swnode = dev_fwnode(&dev->pdev->dev);
> platform_device_unregister(dev->pdev);
> + gpio_sim_remove_hogs(dev);
> gpio_sim_remove_swnode_recursive(swnode);
> dev->pdev = NULL;
> - gpio_sim_remove_hogs(dev);
> }
>
> static ssize_t
Hi, Andy,
Sorry, but unfortunately this patch didn't fix the memleak. Please see the result:
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo clear > /sys/kernel/debug/kmemleak
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# time ./gpio-sim.sh
trap: SIGTERM: bad trap
1. chip_name and dev_name attributes
1.1. Chip name is communicated to user
1.2. chip_name returns 'none' if the chip is still pending
1.3. Device name is communicated to user
2. Creating and configuring simulated chips
2.1. Default number of lines is 1
2.2. Number of lines can be specified
2.3. Label can be set
2.4. Label can be left empty
2.5. Line names can be configured
2.6. Line config can remain unused if offset is greater than number of lines
2.7. Line configfs directory names are sanitized
2.8. Multiple chips can be created
2.9. Can't modify settings when chip is live
2.10. Can't create line items when chip is live
2.11. Probe errors are propagated to user-space
2.12. Cannot enable a chip without any GPIO banks
2.13. Duplicate chip labels are not allowed
2.14. Lines can be hogged
3. Controlling simulated chips
3.1. Pull can be set over sysfs
3.2. Pull can be read from sysfs
3.3. Incorrect input in sysfs is rejected
3.4. Can't write to value
4. Simulated GPIO chips are functional
4.1. Values can be read from sysfs
4.2. Bias settings work correctly
GPIO gpio-sim test PASS
real 0m1.120s
user 0m0.283s
sys 0m0.842s
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo scan > /sys/kernel/debug/kmemleak
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff91f019219660 (size 32):
comm "gpio-sim.sh", pid 11223, jiffies 4295028142 (age 87.304s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<ffffffff86fcd408>] __kmem_cache_alloc_node+0x1d8/0x330
[<ffffffff86f4c271>] __kmalloc_node_track_caller+0x51/0xd0
[<ffffffff872fb359>] kvasprintf+0x69/0xd0
[<ffffffff872fb4ce>] kasprintf+0x4e/0x70
[<ffffffffc1864553>] gpio_sim_device_config_live_store+0x483/0x6b5 [gpio_sim]
[<ffffffff870dde9c>] configfs_write_iter+0xcc/0x130
[<ffffffff87016869>] vfs_write+0x1f9/0x3b0
[<ffffffff87016dab>] ksys_write+0x6b/0xf0
[<ffffffff87016e59>] __x64_sys_write+0x19/0x20
[<ffffffff87b91188>] do_syscall_64+0x58/0x80
[<ffffffff87c000aa>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# uname -rms
Linux 6.2.0-rc8-lru-km-andy-00015-gf6feea56f66d-dirty x86_64
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio#
I can verify that I build the right patch:
marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$ git diff
diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
index 60514bc5454f..7f79e49b23d7 100644
--- a/drivers/gpio/gpio-sim.c
+++ b/drivers/gpio/gpio-sim.c
@@ -954,9 +954,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
swnode = dev_fwnode(&dev->pdev->dev);
platform_device_unregister(dev->pdev);
+ gpio_sim_remove_hogs(dev);
gpio_sim_remove_swnode_recursive(swnode);
dev->pdev = NULL;
- gpio_sim_remove_hogs(dev);
}
static ssize_t
marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$
Alternatively, I could try to bisect if you think it's prudent to try that.
But first I need a stroll after this kernel build :-)
Do you think that knowing when the bug was introduced might help find the culprit?
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: INFO: BISECTED: memory leak in gpio device in 6.2-rc6
2023-02-12 14:19 ` Andy Shevchenko
2023-02-14 18:19 ` Mirsad Goran Todorovac
@ 2023-02-14 20:54 ` Mirsad Goran Todorovac
1 sibling, 0 replies; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-14 20:54 UTC (permalink / raw)
To: Andy Shevchenko
Cc: linux-gpio, Linus Walleij, Bartosz Golaszewski, linux-kernel,
Thorsten Leemhuis
On 12. 02. 2023. 15:19, Andy Shevchenko wrote:
> On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
>> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
>>> I came across this memory leak apparently in the GPIO device driver.
>>> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
>>>
>>> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
>>> enabled.
>>>
>>> If you think this bug is significant, I can attempt the bug bisect in the
>>> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
>>> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
>>>
>>> Here is the /sys/kernel/debug/kmemleak output:
>>>
>>> unreferenced object 0xffff9e67ad71f160 (size 32):
>>> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
>>> hex dump (first 32 bytes):
>>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>> backtrace:
>>> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
>>> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
>>> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
>>> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
>>> [<00000000a3431d55>] kasprintf+0x4e/0x70
>>> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>>> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
>>> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
>>> [<00000000d2336251>] ksys_write+0x61/0xe0
>>> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
>>> [<000000008ac743d2>] do_syscall_64+0x58/0x80
>>> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>> [root@pc-mtodorov marvin]#
>>
>> The new development on the bug is that it probably requires some superuser privileges
>> or some capability to be exploited, for it requires access to configfs,
>> but it was reproduced on different hardware as well.
>>
>> The minimum reproducing script is attached, with its output log.
>>
>> From the testing "age" of the unreferenced object I assume that it is allocated earlier
>> in a part of script I am unable to locate or specify, but orphaned at the end of the script.
>>
>> root@/home/user/kernel_bugs/gpio-sim# time ./gpio-reproduce-min.sh
>> 2.14. Lines can be hogged
>> Scanning stage 2.14.7 ... done.
>> Sleeping 60 seconds ... done.
>> Stage 2.14.7 clean.
>> Rescanning stage 2.14.7 ... done.
>> Sleeping 60 seconds ... done.
>> unreferenced object 0xffff9593b9d16bc0 (size 32):
>> comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 136.184s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>
>> real 2m16.398s
>> user 0m0.023s
>> sys 0m16.360s
>> root@/home/user/kernel_bugs/gpio-sim#
>>
>> The effect is cummulative:
>>
>> root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim# cat /sys/kernel/debug/kmemleak
>> unreferenced object 0xffff9593b9d16bc0 (size 32):
>> comm "gpio-reproduce-", pid 7594, jiffies 4295865460 (age 520.296s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>> unreferenced object 0xffff95938918fb40 (size 32):
>> comm "gpio-reproduce-", pid 7675, jiffies 4295954327 (age 164.832s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>> unreferenced object 0xffff9594a3cf1820 (size 32):
>> comm "gpio-reproduce-", pid 7721, jiffies 4295976853 (age 74.728s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<00000000fe76444b>] __kmem_cache_alloc_node+0x380/0x4e0
>> [<0000000099f63f55>] __kmalloc_node_track_caller+0x55/0x140
>> [<00000000c4efe87f>] kvasprintf+0x6b/0xd0
>> [<000000000c0f91cd>] kasprintf+0x4e/0x70
>> [<000000003434d9b5>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>> [<0000000052ce6759>] configfs_write_iter+0xcc/0x130
>> [<0000000006087fd2>] vfs_write+0x2b4/0x3d0
>> [<000000008a17e041>] ksys_write+0x61/0xe0
>> [<000000007bded8ea>] __x64_sys_write+0x1a/0x20
>> [<00000000e1220148>] do_syscall_64+0x58/0x80
>> [<0000000006093069>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>> root@marvin-IdeaPad-3-15ITL6:/home/user/kernel_bugs/gpio-sim#
>>
>> With sufficient privileges (or possibly even without them), an exploit can be devised
>> to automate allocation of orphaned objects, at a rate of a couple per second (PoC attached).
>
> Looking into the code I found one inconsistency. It might be that that brings
> an issue, dunno. Can you try the patch below if it helps (but TBH I'm a bit
> sceptical)?
>
>>From 499cfb52aa7de67a8bbb56ce183d9528b2376db0 Mon Sep 17 00:00:00 2001
> From: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Date: Sun, 12 Feb 2023 16:15:00 +0200
> Subject: [PATCH 1/1] gpio: sim: Deactivate device in reversed order
>
> Run the steps to deactivate device in the reserved order to what
> it has been done in gpio_sim_device_activate_unlocked().
>
> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> ---
> drivers/gpio/gpio-sim.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
> index a51b5ea38ad5..b0111d18808c 100644
> --- a/drivers/gpio/gpio-sim.c
> +++ b/drivers/gpio/gpio-sim.c
> @@ -953,9 +953,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
>
> swnode = dev_fwnode(&dev->pdev->dev);
> platform_device_unregister(dev->pdev);
> + gpio_sim_remove_hogs(dev);
> gpio_sim_remove_swnode_recursive(swnode);
> dev->pdev = NULL;
> - gpio_sim_remove_hogs(dev);
> }
>
> static ssize_t
P.S.
Hi, Andy,
Lucky or unlucky, bisect was rather quick: the gpio-sim module seems to appear in
the 5.17-rc1 kernel, and it seems that this first version is already "bad", that is,
leaking memory.
I have managed to automate the exploit to create cca. 330,000 leaks, but the kernel
did not crash yet, though much of the memory was used. But the leak is rather small,
32 bytes + overhead.
Here is the result of the run of tools/testing/selftests/gpio/gpio-sim.sh:
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# cat /sys/kernel/debug/kmemleak
unreferenced object 0xffff9ed2ca929f40 (size 32):
comm "gpio-sim.sh", pid 15074, jiffies 4294959639 (age 25.848s)
hex dump (first 32 bytes):
67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000008ae1eb47>] slab_post_alloc_hook+0x80/0x2a0
[<000000000d18725c>] __kmalloc_track_caller+0x118/0x290
[<0000000031cbee97>] kvasprintf+0x66/0xd0
[<00000000f315aa2a>] kasprintf+0x4e/0x70
[<00000000a996121f>] gpio_sim_device_config_live_store+0x372/0x578 [gpio_sim]
[<00000000fd890a9e>] configfs_write_iter+0xc8/0x130
[<00000000e3ef7485>] new_sync_write+0x117/0x1b0
[<000000006dc1f988>] vfs_write+0x1aa/0x290
[<0000000027668ff9>] ksys_write+0x61/0xe0
[<00000000ac7bd963>] __x64_sys_write+0x1a/0x20
[<00000000ed9d842c>] do_syscall_64+0x58/0x80
[<00000000c81cc832>] entry_SYSCALL_64_after_hwframe+0x44/0xae
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# uname -rms
Linux 5.17.0-rc1-mglru-kmlk-andy x86_64
root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio#
Basically, this is the similar output to the one with 6.2-rc6 leak.
I hope this can help narrow down the culprit code.
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-14 18:19 ` Mirsad Goran Todorovac
@ 2023-02-15 10:53 ` Andy Shevchenko
2023-02-15 14:45 ` Mirsad Goran Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Andy Shevchenko @ 2023-02-15 10:53 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: linux-gpio, Linus Walleij, Bartosz Golaszewski, linux-kernel,
Thorsten Leemhuis
On Tue, Feb 14, 2023 at 07:19:16PM +0100, Mirsad Goran Todorovac wrote:
> On 12. 02. 2023. 15:19, Andy Shevchenko wrote:
> > On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
> >> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
> >>> I came across this memory leak apparently in the GPIO device driver.
> >>> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
> >>>
> >>> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
> >>> enabled.
> >>>
> >>> If you think this bug is significant, I can attempt the bug bisect in the
> >>> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
> >>> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
> >>>
> >>> Here is the /sys/kernel/debug/kmemleak output:
> >>>
> >>> unreferenced object 0xffff9e67ad71f160 (size 32):
> >>> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
> >>> hex dump (first 32 bytes):
> >>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> >>> backtrace:
> >>> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
> >>> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
> >>> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
> >>> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
> >>> [<00000000a3431d55>] kasprintf+0x4e/0x70
> >>> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> >>> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
> >>> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
> >>> [<00000000d2336251>] ksys_write+0x61/0xe0
> >>> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
> >>> [<000000008ac743d2>] do_syscall_64+0x58/0x80
> >>> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> Sorry, but unfortunately this patch didn't fix the memleak. Please see the result:
Thank you for give a try!
Yeah, that's why I put that I'm skeptical, because while patch is correct per
se it wouldn't prevent the initial leakage (it seems it happens due to other
circumstances).
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo clear > /sys/kernel/debug/kmemleak
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# time ./gpio-sim.sh
> trap: SIGTERM: bad trap
> 1. chip_name and dev_name attributes
> 1.1. Chip name is communicated to user
> 1.2. chip_name returns 'none' if the chip is still pending
> 1.3. Device name is communicated to user
> 2. Creating and configuring simulated chips
> 2.1. Default number of lines is 1
> 2.2. Number of lines can be specified
> 2.3. Label can be set
> 2.4. Label can be left empty
> 2.5. Line names can be configured
> 2.6. Line config can remain unused if offset is greater than number of lines
> 2.7. Line configfs directory names are sanitized
> 2.8. Multiple chips can be created
> 2.9. Can't modify settings when chip is live
> 2.10. Can't create line items when chip is live
> 2.11. Probe errors are propagated to user-space
> 2.12. Cannot enable a chip without any GPIO banks
> 2.13. Duplicate chip labels are not allowed
> 2.14. Lines can be hogged
> 3. Controlling simulated chips
> 3.1. Pull can be set over sysfs
> 3.2. Pull can be read from sysfs
> 3.3. Incorrect input in sysfs is rejected
> 3.4. Can't write to value
> 4. Simulated GPIO chips are functional
> 4.1. Values can be read from sysfs
> 4.2. Bias settings work correctly
> GPIO gpio-sim test PASS
>
> real 0m1.120s
> user 0m0.283s
> sys 0m0.842s
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo scan > /sys/kernel/debug/kmemleak
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# cat /sys/kernel/debug/kmemleak
> unreferenced object 0xffff91f019219660 (size 32):
> comm "gpio-sim.sh", pid 11223, jiffies 4295028142 (age 87.304s)
> hex dump (first 32 bytes):
> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<ffffffff86fcd408>] __kmem_cache_alloc_node+0x1d8/0x330
> [<ffffffff86f4c271>] __kmalloc_node_track_caller+0x51/0xd0
> [<ffffffff872fb359>] kvasprintf+0x69/0xd0
> [<ffffffff872fb4ce>] kasprintf+0x4e/0x70
> [<ffffffffc1864553>] gpio_sim_device_config_live_store+0x483/0x6b5 [gpio_sim]
> [<ffffffff870dde9c>] configfs_write_iter+0xcc/0x130
> [<ffffffff87016869>] vfs_write+0x1f9/0x3b0
> [<ffffffff87016dab>] ksys_write+0x6b/0xf0
> [<ffffffff87016e59>] __x64_sys_write+0x19/0x20
> [<ffffffff87b91188>] do_syscall_64+0x58/0x80
> [<ffffffff87c000aa>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# uname -rms
> Linux 6.2.0-rc8-lru-km-andy-00015-gf6feea56f66d-dirty x86_64
> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio#
>
> I can verify that I build the right patch:
>
> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$ git diff
> diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
> index 60514bc5454f..7f79e49b23d7 100644
> --- a/drivers/gpio/gpio-sim.c
> +++ b/drivers/gpio/gpio-sim.c
> @@ -954,9 +954,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
>
> swnode = dev_fwnode(&dev->pdev->dev);
> platform_device_unregister(dev->pdev);
> + gpio_sim_remove_hogs(dev);
> gpio_sim_remove_swnode_recursive(swnode);
> dev->pdev = NULL;
> - gpio_sim_remove_hogs(dev);
> }
>
> static ssize_t
> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$
>
> Alternatively, I could try to bisect if you think it's prudent to try that.
> But first I need a stroll after this kernel build :-)
>
> Do you think that knowing when the bug was introduced might help find the culprit?
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-15 10:53 ` Andy Shevchenko
@ 2023-02-15 14:45 ` Mirsad Goran Todorovac
2023-02-16 14:16 ` Bartosz Golaszewski
0 siblings, 1 reply; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-15 14:45 UTC (permalink / raw)
To: Andy Shevchenko
Cc: linux-gpio, Linus Walleij, Bartosz Golaszewski, linux-kernel,
Thorsten Leemhuis
On 15.2.2023. 11:53, Andy Shevchenko wrote:
> On Tue, Feb 14, 2023 at 07:19:16PM +0100, Mirsad Goran Todorovac wrote:
>> On 12. 02. 2023. 15:19, Andy Shevchenko wrote:
>>> On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
>>>> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
>>>>> I came across this memory leak apparently in the GPIO device driver.
>>>>> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
>>>>>
>>>>> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
>>>>> enabled.
>>>>>
>>>>> If you think this bug is significant, I can attempt the bug bisect in the
>>>>> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
>>>>> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
>>>>>
>>>>> Here is the /sys/kernel/debug/kmemleak output:
>>>>>
>>>>> unreferenced object 0xffff9e67ad71f160 (size 32):
>>>>> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
>>>>> hex dump (first 32 bytes):
>>>>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>>> backtrace:
>>>>> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
>>>>> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
>>>>> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
>>>>> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
>>>>> [<00000000a3431d55>] kasprintf+0x4e/0x70
>>>>> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>>>>> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
>>>>> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
>>>>> [<00000000d2336251>] ksys_write+0x61/0xe0
>>>>> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
>>>>> [<000000008ac743d2>] do_syscall_64+0x58/0x80
>>>>> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>
>> Sorry, but unfortunately this patch didn't fix the memleak. Please see the result:
>
> Thank you for give a try!
No sweat. It was worth the effort.
> Yeah, that's why I put that I'm skeptical, because while patch is correct per
> se it wouldn't prevent the initial leakage (it seems it happens due to other
> circumstances).
I must admit that is looks like hieroglyphs to me. My learning curve had not
yet reached that point of debugging memory leaks in the kernel drivers ...
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo clear > /sys/kernel/debug/kmemleak
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# time ./gpio-sim.sh
>> trap: SIGTERM: bad trap
>> 1. chip_name and dev_name attributes
>> 1.1. Chip name is communicated to user
>> 1.2. chip_name returns 'none' if the chip is still pending
>> 1.3. Device name is communicated to user
>> 2. Creating and configuring simulated chips
>> 2.1. Default number of lines is 1
>> 2.2. Number of lines can be specified
>> 2.3. Label can be set
>> 2.4. Label can be left empty
>> 2.5. Line names can be configured
>> 2.6. Line config can remain unused if offset is greater than number of lines
>> 2.7. Line configfs directory names are sanitized
>> 2.8. Multiple chips can be created
>> 2.9. Can't modify settings when chip is live
>> 2.10. Can't create line items when chip is live
>> 2.11. Probe errors are propagated to user-space
>> 2.12. Cannot enable a chip without any GPIO banks
>> 2.13. Duplicate chip labels are not allowed
>> 2.14. Lines can be hogged
>> 3. Controlling simulated chips
>> 3.1. Pull can be set over sysfs
>> 3.2. Pull can be read from sysfs
>> 3.3. Incorrect input in sysfs is rejected
>> 3.4. Can't write to value
>> 4. Simulated GPIO chips are functional
>> 4.1. Values can be read from sysfs
>> 4.2. Bias settings work correctly
>> GPIO gpio-sim test PASS
>>
>> real 0m1.120s
>> user 0m0.283s
>> sys 0m0.842s
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo scan > /sys/kernel/debug/kmemleak
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# cat /sys/kernel/debug/kmemleak
>> unreferenced object 0xffff91f019219660 (size 32):
>> comm "gpio-sim.sh", pid 11223, jiffies 4295028142 (age 87.304s)
>> hex dump (first 32 bytes):
>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>> backtrace:
>> [<ffffffff86fcd408>] __kmem_cache_alloc_node+0x1d8/0x330
>> [<ffffffff86f4c271>] __kmalloc_node_track_caller+0x51/0xd0
>> [<ffffffff872fb359>] kvasprintf+0x69/0xd0
>> [<ffffffff872fb4ce>] kasprintf+0x4e/0x70
>> [<ffffffffc1864553>] gpio_sim_device_config_live_store+0x483/0x6b5 [gpio_sim]
>> [<ffffffff870dde9c>] configfs_write_iter+0xcc/0x130
>> [<ffffffff87016869>] vfs_write+0x1f9/0x3b0
>> [<ffffffff87016dab>] ksys_write+0x6b/0xf0
>> [<ffffffff87016e59>] __x64_sys_write+0x19/0x20
>> [<ffffffff87b91188>] do_syscall_64+0x58/0x80
>> [<ffffffff87c000aa>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# uname -rms
>> Linux 6.2.0-rc8-lru-km-andy-00015-gf6feea56f66d-dirty x86_64
>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio#
>>
>> I can verify that I build the right patch:
>>
>> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$ git diff
>> diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
>> index 60514bc5454f..7f79e49b23d7 100644
>> --- a/drivers/gpio/gpio-sim.c
>> +++ b/drivers/gpio/gpio-sim.c
>> @@ -954,9 +954,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
>>
>> swnode = dev_fwnode(&dev->pdev->dev);
>> platform_device_unregister(dev->pdev);
>> + gpio_sim_remove_hogs(dev);
>> gpio_sim_remove_swnode_recursive(swnode);
>> dev->pdev = NULL;
>> - gpio_sim_remove_hogs(dev);
>> }
>>
>> static ssize_t
>> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$
>>
>> Alternatively, I could try to bisect if you think it's prudent to try that.
>> But first I need a stroll after this kernel build :-)
>>
>> Do you think that knowing when the bug was introduced might help find the culprit?
>
--
Mirsad Todorovac
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb
Republic of Croatia, the European Union
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-15 14:45 ` Mirsad Goran Todorovac
@ 2023-02-16 14:16 ` Bartosz Golaszewski
2023-02-20 13:10 ` Mirsad Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Bartosz Golaszewski @ 2023-02-16 14:16 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: Andy Shevchenko, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On Wed, Feb 15, 2023 at 3:45 PM Mirsad Goran Todorovac
<mirsad.todorovac@alu.unizg.hr> wrote:
>
> On 15.2.2023. 11:53, Andy Shevchenko wrote:
> > On Tue, Feb 14, 2023 at 07:19:16PM +0100, Mirsad Goran Todorovac wrote:
> >> On 12. 02. 2023. 15:19, Andy Shevchenko wrote:
> >>> On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
> >>>> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
> >>>>> I came across this memory leak apparently in the GPIO device driver.
> >>>>> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
> >>>>>
> >>>>> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
> >>>>> enabled.
> >>>>>
> >>>>> If you think this bug is significant, I can attempt the bug bisect in the
> >>>>> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
> >>>>> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
> >>>>>
> >>>>> Here is the /sys/kernel/debug/kmemleak output:
> >>>>>
> >>>>> unreferenced object 0xffff9e67ad71f160 (size 32):
> >>>>> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
> >>>>> hex dump (first 32 bytes):
> >>>>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> >>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> >>>>> backtrace:
> >>>>> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
> >>>>> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
> >>>>> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
> >>>>> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
> >>>>> [<00000000a3431d55>] kasprintf+0x4e/0x70
> >>>>> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
> >>>>> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
> >>>>> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
> >>>>> [<00000000d2336251>] ksys_write+0x61/0xe0
> >>>>> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
> >>>>> [<000000008ac743d2>] do_syscall_64+0x58/0x80
> >>>>> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
> >
> >> Sorry, but unfortunately this patch didn't fix the memleak. Please see the result:
> >
> > Thank you for give a try!
>
> No sweat. It was worth the effort.
>
> > Yeah, that's why I put that I'm skeptical, because while patch is correct per
> > se it wouldn't prevent the initial leakage (it seems it happens due to other
> > circumstances).
>
> I must admit that is looks like hieroglyphs to me. My learning curve had not
> yet reached that point of debugging memory leaks in the kernel drivers ...
>
> >> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo clear > /sys/kernel/debug/kmemleak
> >> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# time ./gpio-sim.sh
> >> trap: SIGTERM: bad trap
> >> 1. chip_name and dev_name attributes
> >> 1.1. Chip name is communicated to user
> >> 1.2. chip_name returns 'none' if the chip is still pending
> >> 1.3. Device name is communicated to user
> >> 2. Creating and configuring simulated chips
> >> 2.1. Default number of lines is 1
> >> 2.2. Number of lines can be specified
> >> 2.3. Label can be set
> >> 2.4. Label can be left empty
> >> 2.5. Line names can be configured
> >> 2.6. Line config can remain unused if offset is greater than number of lines
> >> 2.7. Line configfs directory names are sanitized
> >> 2.8. Multiple chips can be created
> >> 2.9. Can't modify settings when chip is live
> >> 2.10. Can't create line items when chip is live
> >> 2.11. Probe errors are propagated to user-space
> >> 2.12. Cannot enable a chip without any GPIO banks
> >> 2.13. Duplicate chip labels are not allowed
> >> 2.14. Lines can be hogged
> >> 3. Controlling simulated chips
> >> 3.1. Pull can be set over sysfs
> >> 3.2. Pull can be read from sysfs
> >> 3.3. Incorrect input in sysfs is rejected
> >> 3.4. Can't write to value
> >> 4. Simulated GPIO chips are functional
> >> 4.1. Values can be read from sysfs
> >> 4.2. Bias settings work correctly
> >> GPIO gpio-sim test PASS
> >>
> >> real 0m1.120s
> >> user 0m0.283s
> >> sys 0m0.842s
> >> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo scan > /sys/kernel/debug/kmemleak
> >> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# cat /sys/kernel/debug/kmemleak
> >> unreferenced object 0xffff91f019219660 (size 32):
> >> comm "gpio-sim.sh", pid 11223, jiffies 4295028142 (age 87.304s)
> >> hex dump (first 32 bytes):
> >> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
> >> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> >> backtrace:
> >> [<ffffffff86fcd408>] __kmem_cache_alloc_node+0x1d8/0x330
> >> [<ffffffff86f4c271>] __kmalloc_node_track_caller+0x51/0xd0
> >> [<ffffffff872fb359>] kvasprintf+0x69/0xd0
> >> [<ffffffff872fb4ce>] kasprintf+0x4e/0x70
> >> [<ffffffffc1864553>] gpio_sim_device_config_live_store+0x483/0x6b5 [gpio_sim]
> >> [<ffffffff870dde9c>] configfs_write_iter+0xcc/0x130
> >> [<ffffffff87016869>] vfs_write+0x1f9/0x3b0
> >> [<ffffffff87016dab>] ksys_write+0x6b/0xf0
> >> [<ffffffff87016e59>] __x64_sys_write+0x19/0x20
> >> [<ffffffff87b91188>] do_syscall_64+0x58/0x80
> >> [<ffffffff87c000aa>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
> >> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# uname -rms
> >> Linux 6.2.0-rc8-lru-km-andy-00015-gf6feea56f66d-dirty x86_64
> >> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio#
> >>
> >> I can verify that I build the right patch:
> >>
> >> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$ git diff
> >> diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
> >> index 60514bc5454f..7f79e49b23d7 100644
> >> --- a/drivers/gpio/gpio-sim.c
> >> +++ b/drivers/gpio/gpio-sim.c
> >> @@ -954,9 +954,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
> >>
> >> swnode = dev_fwnode(&dev->pdev->dev);
> >> platform_device_unregister(dev->pdev);
> >> + gpio_sim_remove_hogs(dev);
> >> gpio_sim_remove_swnode_recursive(swnode);
> >> dev->pdev = NULL;
> >> - gpio_sim_remove_hogs(dev);
> >> }
> >>
> >> static ssize_t
> >> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$
> >>
> >> Alternatively, I could try to bisect if you think it's prudent to try that.
> >> But first I need a stroll after this kernel build :-)
> >>
> >> Do you think that knowing when the bug was introduced might help find the culprit?
> >
>
> --
> Mirsad Todorovac
> System engineer
> Faculty of Graphic Arts | Academy of Fine Arts
> University of Zagreb
> Republic of Croatia, the European Union
>
> Sistem inženjer
> Grafički fakultet | Akademija likovnih umjetnosti
> Sveučilište u Zagrebu
>
Thanks for the report Mirsad, just sent out a fix.
Bart
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-16 14:16 ` Bartosz Golaszewski
@ 2023-02-20 13:10 ` Mirsad Todorovac
2023-02-20 13:43 ` Andy Shevchenko
0 siblings, 1 reply; 23+ messages in thread
From: Mirsad Todorovac @ 2023-02-20 13:10 UTC (permalink / raw)
To: Bartosz Golaszewski
Cc: Andy Shevchenko, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 2/16/23 15:16, Bartosz Golaszewski wrote:
> On Wed, Feb 15, 2023 at 3:45 PM Mirsad Goran Todorovac
> <mirsad.todorovac@alu.unizg.hr> wrote:
>>
>> On 15.2.2023. 11:53, Andy Shevchenko wrote:
>>> On Tue, Feb 14, 2023 at 07:19:16PM +0100, Mirsad Goran Todorovac wrote:
>>>> On 12. 02. 2023. 15:19, Andy Shevchenko wrote:
>>>>> On Wed, Feb 08, 2023 at 08:55:24PM +0100, Mirsad Goran Todorovac wrote:
>>>>>> On 31. 01. 2023. 10:36, Mirsad Goran Todorovac wrote:
>>>>>>> I came across this memory leak apparently in the GPIO device driver.
>>>>>>> It is still present in 6.2-rc6 release candidate kernel (just ran kselftest).
>>>>>>>
>>>>>>> This is a vanilla Torvalds tree kernel with MGLRU and KMEMLEAK (obviously)
>>>>>>> enabled.
>>>>>>>
>>>>>>> If you think this bug is significant, I can attempt the bug bisect in the
>>>>>>> environment that triggered it (Lenovo LENOVO_MT_10TX_BU_Lenovo_FM_V530S-07ICB)
>>>>>>> with BIOS M22KT49A from 11/10/2022 and AlmaLinux 8.7.
>>>>>>>
>>>>>>> Here is the /sys/kernel/debug/kmemleak output:
>>>>>>>
>>>>>>> unreferenced object 0xffff9e67ad71f160 (size 32):
>>>>>>> comm "gpio-sim.sh", pid 208926, jiffies 4372229685 (age 2101.564s)
>>>>>>> hex dump (first 32 bytes):
>>>>>>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>>>>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>>>>> backtrace:
>>>>>>> [<0000000098bf3d1b>] slab_post_alloc_hook+0x91/0x320
>>>>>>> [<00000000da3205c5>] __kmem_cache_alloc_node+0x1bf/0x2b0
>>>>>>> [<00000000aa51a58a>] __kmalloc_node_track_caller+0x55/0x140
>>>>>>> [<00000000bd682ecc>] kvasprintf+0x6b/0xd0
>>>>>>> [<00000000a3431d55>] kasprintf+0x4e/0x70
>>>>>>> [<00000000f52d2629>] gpio_sim_device_config_live_store+0x401/0x59d [gpio_sim]
>>>>>>> [<00000000673fc6df>] configfs_write_iter+0xcc/0x130
>>>>>>> [<000000001d5d0829>] vfs_write+0x2b4/0x3d0
>>>>>>> [<00000000d2336251>] ksys_write+0x61/0xe0
>>>>>>> [<00000000f7015bb1>] __x64_sys_write+0x1a/0x20
>>>>>>> [<000000008ac743d2>] do_syscall_64+0x58/0x80
>>>>>>> [<000000004d7b7d50>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>>
>>>> Sorry, but unfortunately this patch didn't fix the memleak. Please see the result:
>>>
>>> Thank you for give a try!
>>
>> No sweat. It was worth the effort.
>>
>>> Yeah, that's why I put that I'm skeptical, because while patch is correct per
>>> se it wouldn't prevent the initial leakage (it seems it happens due to other
>>> circumstances).
>>
>> I must admit that is looks like hieroglyphs to me. My learning curve had not
>> yet reached that point of debugging memory leaks in the kernel drivers ...
>>
>>>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo clear > /sys/kernel/debug/kmemleak
>>>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# time ./gpio-sim.sh
>>>> trap: SIGTERM: bad trap
>>>> 1. chip_name and dev_name attributes
>>>> 1.1. Chip name is communicated to user
>>>> 1.2. chip_name returns 'none' if the chip is still pending
>>>> 1.3. Device name is communicated to user
>>>> 2. Creating and configuring simulated chips
>>>> 2.1. Default number of lines is 1
>>>> 2.2. Number of lines can be specified
>>>> 2.3. Label can be set
>>>> 2.4. Label can be left empty
>>>> 2.5. Line names can be configured
>>>> 2.6. Line config can remain unused if offset is greater than number of lines
>>>> 2.7. Line configfs directory names are sanitized
>>>> 2.8. Multiple chips can be created
>>>> 2.9. Can't modify settings when chip is live
>>>> 2.10. Can't create line items when chip is live
>>>> 2.11. Probe errors are propagated to user-space
>>>> 2.12. Cannot enable a chip without any GPIO banks
>>>> 2.13. Duplicate chip labels are not allowed
>>>> 2.14. Lines can be hogged
>>>> 3. Controlling simulated chips
>>>> 3.1. Pull can be set over sysfs
>>>> 3.2. Pull can be read from sysfs
>>>> 3.3. Incorrect input in sysfs is rejected
>>>> 3.4. Can't write to value
>>>> 4. Simulated GPIO chips are functional
>>>> 4.1. Values can be read from sysfs
>>>> 4.2. Bias settings work correctly
>>>> GPIO gpio-sim test PASS
>>>>
>>>> real 0m1.120s
>>>> user 0m0.283s
>>>> sys 0m0.842s
>>>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# echo scan > /sys/kernel/debug/kmemleak
>>>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# cat /sys/kernel/debug/kmemleak
>>>> unreferenced object 0xffff91f019219660 (size 32):
>>>> comm "gpio-sim.sh", pid 11223, jiffies 4295028142 (age 87.304s)
>>>> hex dump (first 32 bytes):
>>>> 67 70 69 6f 2d 73 69 6d 2e 30 2d 6e 6f 64 65 30 gpio-sim.0-node0
>>>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
>>>> backtrace:
>>>> [<ffffffff86fcd408>] __kmem_cache_alloc_node+0x1d8/0x330
>>>> [<ffffffff86f4c271>] __kmalloc_node_track_caller+0x51/0xd0
>>>> [<ffffffff872fb359>] kvasprintf+0x69/0xd0
>>>> [<ffffffff872fb4ce>] kasprintf+0x4e/0x70
>>>> [<ffffffffc1864553>] gpio_sim_device_config_live_store+0x483/0x6b5 [gpio_sim]
>>>> [<ffffffff870dde9c>] configfs_write_iter+0xcc/0x130
>>>> [<ffffffff87016869>] vfs_write+0x1f9/0x3b0
>>>> [<ffffffff87016dab>] ksys_write+0x6b/0xf0
>>>> [<ffffffff87016e59>] __x64_sys_write+0x19/0x20
>>>> [<ffffffff87b91188>] do_syscall_64+0x58/0x80
>>>> [<ffffffff87c000aa>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
>>>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio# uname -rms
>>>> Linux 6.2.0-rc8-lru-km-andy-00015-gf6feea56f66d-dirty x86_64
>>>> root@marvin-IdeaPad-3-15ITL6:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio#
>>>>
>>>> I can verify that I build the right patch:
>>>>
>>>> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$ git diff
>>>> diff --git a/drivers/gpio/gpio-sim.c b/drivers/gpio/gpio-sim.c
>>>> index 60514bc5454f..7f79e49b23d7 100644
>>>> --- a/drivers/gpio/gpio-sim.c
>>>> +++ b/drivers/gpio/gpio-sim.c
>>>> @@ -954,9 +954,9 @@ static void gpio_sim_device_deactivate_unlocked(struct gpio_sim_device *dev)
>>>>
>>>> swnode = dev_fwnode(&dev->pdev->dev);
>>>> platform_device_unregister(dev->pdev);
>>>> + gpio_sim_remove_hogs(dev);
>>>> gpio_sim_remove_swnode_recursive(swnode);
>>>> dev->pdev = NULL;
>>>> - gpio_sim_remove_hogs(dev);
>>>> }
>>>>
>>>> static ssize_t
>>>> marvin@marvin-IdeaPad-3-15ITL6:~/linux/kernel/linux_torvalds$
>>>>
>>>> Alternatively, I could try to bisect if you think it's prudent to try that.
>>>> But first I need a stroll after this kernel build :-)
>>>>
>>>> Do you think that knowing when the bug was introduced might help find the culprit?
> Thanks for the report Mirsad, just sent out a fix.
Hi, Bart,
It is really nothing. The reproducer shows that the leak is apparently gone.
As Mr. McKenney once said, a bunch of monkeys with keyboard could
have done it in a considerable number of trials and errors ;-)
But here I have something that could potentially leak as well. I could not devise a
reproducer due to the leak being lightly triggered only in extreme memory contention.
See it for yourself:
drivers/gpio/gpio-sim.c:
301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
302 {
303 struct device_attribute *val_dev_attr, *pull_dev_attr;
304 struct gpio_sim_attribute *val_attr, *pull_attr;
305 unsigned int num_lines = chip->gc.ngpio;
306 struct device *dev = chip->gc.parent;
307 struct attribute_group *attr_group;
308 struct attribute **attrs;
309 int i, ret;
310
311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
312 num_lines + 1, GFP_KERNEL);
313 if (!chip->attr_groups)
314 return -ENOMEM;
315
316 for (i = 0; i < num_lines; i++) {
317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
319 GFP_KERNEL);
320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
322 if (!attr_group || !attrs || !val_attr || !pull_attr)
323 return -ENOMEM;
324
325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
326 "sim_gpio%u", i);
327 if (!attr_group->name)
328 return -ENOMEM;
Apparently, if the memory allocation only partially succeeds, in the theoretical case
that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
free the partially succeeded allocs, would it?
To explain it better, I tried a version that is not yet full doing "all or nothing"
memory allocation for the gpio-sim driver, because I am not that familiar with the
driver internals.
I give this just as an illustration to what I mean to say (one line of code speaks
like thousand words):
301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
302 {
303 struct device_attribute *val_dev_attr, *pull_dev_attr;
304 struct gpio_sim_attribute *val_attr, *pull_attr;
305 unsigned int num_lines = chip->gc.ngpio;
306 struct device *dev = chip->gc.parent;
307 struct attribute_group *attr_group;
308 struct attribute **attrs;
309 int i, ret;
310
311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
312 num_lines + 1, GFP_KERNEL);
313 if (!chip->attr_groups)
314 return -ENOMEM;
315
316 for (i = 0; i < num_lines; i++) {
317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
319 GFP_KERNEL);
320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
322 if (!attr_group || !attrs || !val_attr || !pull_attr) {
323 if (attr_group)
324 devm_kfree(dev, attr_group);
325 if (attrs)
326 devm_kfree(dev, attrs);
327 if (val_attr)
328 devm_kfree(dev, val_attr);
329 if (pull_attr)
330 devm_kfree(dev, pull_attr);
331 return -ENOMEM;
332 }
333
334 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
335 "sim_gpio%u", i);
336 if (!attr_group->name) {
337 devm_kfree(dev, attr_group);
338 devm_kfree(dev, attrs);
339 devm_kfree(dev, val_attr);
340 devm_kfree(dev, pull_attr);
341 return -ENOMEM;
342 }
343
344 val_attr->offset = pull_attr->offset = i;
This is, of course, far from perfect, and it seems that the driver will still leak
memory in case of partially successful allocation because of the
for (i = 0; i < num_lines; i++) { ... } loop. :-/
I hope this makes any sense to you.
Thanks,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-20 13:10 ` Mirsad Todorovac
@ 2023-02-20 13:43 ` Andy Shevchenko
2023-02-21 13:52 ` Mirsad Goran Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Andy Shevchenko @ 2023-02-20 13:43 UTC (permalink / raw)
To: Mirsad Todorovac
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
> On 2/16/23 15:16, Bartosz Golaszewski wrote:
...
> As Mr. McKenney once said, a bunch of monkeys with keyboard could
> have done it in a considerable number of trials and errors ;-)
>
> But here I have something that could potentially leak as well. I could not devise a
> reproducer due to the leak being lightly triggered only in extreme memory contention.
>
> See it for yourself:
>
> drivers/gpio/gpio-sim.c:
> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
> 302 {
> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
> 305 unsigned int num_lines = chip->gc.ngpio;
> 306 struct device *dev = chip->gc.parent;
> 307 struct attribute_group *attr_group;
> 308 struct attribute **attrs;
> 309 int i, ret;
> 310
> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
> 312 num_lines + 1, GFP_KERNEL);
> 313 if (!chip->attr_groups)
> 314 return -ENOMEM;
> 315
> 316 for (i = 0; i < num_lines; i++) {
> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
> 319 GFP_KERNEL);
> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
> 323 return -ENOMEM;
> 324
> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
> 326 "sim_gpio%u", i);
> 327 if (!attr_group->name)
> 328 return -ENOMEM;
>
> Apparently, if the memory allocation only partially succeeds, in the theoretical case
> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
> free the partially succeeded allocs, would it?
>
> To explain it better, I tried a version that is not yet full doing "all or nothing"
> memory allocation for the gpio-sim driver, because I am not that familiar with the
> driver internals.
devm_*() mean that the resource allocation is made in a managed manner, so when
it's done, it will be freed automatically.
The question is: is the lifetime of the attr_groups should be lesser or the
same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
Or maybe the chip->gc.parent should be changed to something else (actual GPIO
device, but then it's unclear how to provide the attributes in non-racy way.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-20 13:43 ` Andy Shevchenko
@ 2023-02-21 13:52 ` Mirsad Goran Todorovac
2023-02-21 14:20 ` Mirsad Goran Todorovac
` (2 more replies)
0 siblings, 3 replies; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-21 13:52 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
> On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
>> On 2/16/23 15:16, Bartosz Golaszewski wrote:
>
> ...
>
>> As Mr. McKenney once said, a bunch of monkeys with keyboard could
>> have done it in a considerable number of trials and errors ;-)
>>
>> But here I have something that could potentially leak as well. I could not devise a
>> reproducer due to the leak being lightly triggered only in extreme memory contention.
>>
>> See it for yourself:
>>
>> drivers/gpio/gpio-sim.c:
>> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
>> 302 {
>> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
>> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
>> 305 unsigned int num_lines = chip->gc.ngpio;
>> 306 struct device *dev = chip->gc.parent;
>> 307 struct attribute_group *attr_group;
>> 308 struct attribute **attrs;
>> 309 int i, ret;
>> 310
>> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
>> 312 num_lines + 1, GFP_KERNEL);
>> 313 if (!chip->attr_groups)
>> 314 return -ENOMEM;
>> 315
>> 316 for (i = 0; i < num_lines; i++) {
>> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
>> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
>> 319 GFP_KERNEL);
>> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
>> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
>> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
>> 323 return -ENOMEM;
>> 324
>> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
>> 326 "sim_gpio%u", i);
>> 327 if (!attr_group->name)
>> 328 return -ENOMEM;
>>
>> Apparently, if the memory allocation only partially succeeds, in the theoretical case
>> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
>> free the partially succeeded allocs, would it?
>>
>> To explain it better, I tried a version that is not yet full doing "all or nothing"
>> memory allocation for the gpio-sim driver, because I am not that familiar with the
>> driver internals.
>
> devm_*() mean that the resource allocation is made in a managed manner, so when
> it's done, it will be freed automatically.
Didn't see that one coming ... :-/ "buzzing though the bush ..."
> The question is: is the lifetime of the attr_groups should be lesser or the
> same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
Bona fide said, I hope that automatic deallocation does things in the right order.
I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
a per driver list. But I am not sure how chip->gc was allocated?
Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
as a part of
struct gpio_sim_chip *chip;
struct gpio_chip *gc;
gc = &chip->gc;
and gc->parent is set to
gc->parent = dev;
in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
If I understood well, automatic deallocation on unloading the driver goes
in the reverse order, so lifetime of chip appears to be longer than attr_groups,
but I am really not that good at this ...
> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
> device, but then it's unclear how to provide the attributes in non-racy way
Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
device expert nor would be appropriate to try the craft not earned ;-)
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-21 13:52 ` Mirsad Goran Todorovac
@ 2023-02-21 14:20 ` Mirsad Goran Todorovac
2023-02-21 14:32 ` Mirsad Goran Todorovac
2023-02-21 15:39 ` Andy Shevchenko
2 siblings, 0 replies; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-21 14:20 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 21.2.2023. 14:52, Mirsad Goran Todorovac wrote:
> On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
>> On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
>>> On 2/16/23 15:16, Bartosz Golaszewski wrote:
>>
>> ...
>>
>>> As Mr. McKenney once said, a bunch of monkeys with keyboard could
>>> have done it in a considerable number of trials and errors ;-)
>>>
>>> But here I have something that could potentially leak as well. I could not devise a
>>> reproducer due to the leak being lightly triggered only in extreme memory contention.
>>>
>>> See it for yourself:
>>>
>>> drivers/gpio/gpio-sim.c:
>>> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
>>> 302 {
>>> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
>>> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
>>> 305 unsigned int num_lines = chip->gc.ngpio;
>>> 306 struct device *dev = chip->gc.parent;
>>> 307 struct attribute_group *attr_group;
>>> 308 struct attribute **attrs;
>>> 309 int i, ret;
>>> 310
>>> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
>>> 312 num_lines + 1, GFP_KERNEL);
>>> 313 if (!chip->attr_groups)
>>> 314 return -ENOMEM;
>>> 315
>>> 316 for (i = 0; i < num_lines; i++) {
>>> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
>>> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
>>> 319 GFP_KERNEL);
>>> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
>>> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
>>> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
>>> 323 return -ENOMEM;
>>> 324
>>> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
>>> 326 "sim_gpio%u", i);
>>> 327 if (!attr_group->name)
>>> 328 return -ENOMEM;
>>>
>>> Apparently, if the memory allocation only partially succeeds, in the theoretical case
>>> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
>>> free the partially succeeded allocs, would it?
>>>
>>> To explain it better, I tried a version that is not yet full doing "all or nothing"
>>> memory allocation for the gpio-sim driver, because I am not that familiar with the
>>> driver internals.
>>
>> devm_*() mean that the resource allocation is made in a managed manner, so when
>> it's done, it will be freed automatically.
>
> Didn't see that one coming ... :-/ "buzzing though the bush ..."
>
>> The question is: is the lifetime of the attr_groups should be lesser or the
>> same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
>
> Bona fide said, I hope that automatic deallocation does things in the right order.
> I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
> a per driver list. But I am not sure how chip->gc was allocated?
>
> Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(), as a part of
> struct gpio_sim_chip *chip;
> struct gpio_chip *gc;
>
> gc = &chip->gc;
>
> and gc->parent is set to
>
> gc->parent = dev;
>
> in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
P.S.
The exact line is:
chip = devm_kzalloc <https://elixir.bootlin.com/linux/latest/C/ident/devm_kzalloc>(dev, sizeof(*chip), GFP_KERNEL <https://elixir.bootlin.com/linux/latest/C/ident/GFP_KERNEL>); so I guess it is reasonable to assume that chip will also
be deallocated after attr_groups. chip->gc.parent appears to be a mere pointer to dev parameter in static int gpio_sim_add_bank <https://elixir.bootlin.com/linux/latest/C/ident/gpio_sim_add_bank>(struct fwnode_handle <https://elixir.bootlin.com/linux/latest/C/ident/fwnode_handle> *swnode <https://elixir.bootlin.com/linux/latest/C/ident/swnode>, struct device <https://elixir.bootlin.com/linux/latest/C/ident/device> *dev) This is OTOH called from: static int gpio_sim_probe <https://elixir.bootlin.com/linux/latest/C/ident/gpio_sim_probe>(struct platform_device <https://elixir.bootlin.com/linux/latest/C/ident/platform_device> *pdev)
{
struct device <https://elixir.bootlin.com/linux/latest/C/ident/device> *dev = &pdev->dev;
struct fwnode_handle <https://elixir.bootlin.com/linux/latest/C/ident/fwnode_handle> *swnode <https://elixir.bootlin.com/linux/latest/C/ident/swnode>;
int ret;
device_for_each_child_node <https://elixir.bootlin.com/linux/latest/C/ident/device_for_each_child_node>(dev, swnode <https://elixir.bootlin.com/linux/latest/C/ident/swnode>) {
ret = gpio_sim_add_bank <https://elixir.bootlin.com/linux/latest/C/ident/gpio_sim_add_bank>(swnode
<https://elixir.bootlin.com/linux/latest/C/ident/swnode>, dev); Which means dev passed to chip->gc.parent is initialised with &pdev->dev from pdev parm of gpio_sim_probe(). This is OTOH
referenced from the very:
static struct platform_driver <https://elixir.bootlin.com/linux/latest/C/ident/platform_driver> gpio_sim_driver
<https://elixir.bootlin.com/linux/latest/C/ident/gpio_sim_driver> = { .driver = { .name = "gpio-sim", .of_match_table
<https://elixir.bootlin.com/linux/latest/C/ident/of_match_table> = gpio_sim_of_match
<https://elixir.bootlin.com/linux/latest/C/ident/gpio_sim_of_match>, }, .probe
<https://elixir.bootlin.com/linux/latest/C/ident/probe> = gpio_sim_probe
<https://elixir.bootlin.com/linux/latest/C/ident/gpio_sim_probe>, }; Hope this helps. There's more to this than meets the eye, but
this is really an idiot's attempt to analyse a Linux kernel driver. :-)
> If I understood well, automatic deallocation on unloading the driver goes
> in the reverse order, so lifetime of chip appears to be longer than attr_groups,
> but I am really not that good at this ...
>
>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>> device, but then it's unclear how to provide the attributes in non-racy way
> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>
> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> device expert nor would be appropriate to try the craft not earned ;-)
Regards,
Mirsad
--
Mirsad Todorovac
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb
Republic of Croatia, the European Union
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-21 13:52 ` Mirsad Goran Todorovac
2023-02-21 14:20 ` Mirsad Goran Todorovac
@ 2023-02-21 14:32 ` Mirsad Goran Todorovac
2023-02-21 15:39 ` Andy Shevchenko
2 siblings, 0 replies; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-21 14:32 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 21.2.2023. 14:52, Mirsad Goran Todorovac wrote:
> On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
>> On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
>>> On 2/16/23 15:16, Bartosz Golaszewski wrote:
>>
>> ...
>>
>>> As Mr. McKenney once said, a bunch of monkeys with keyboard could
>>> have done it in a considerable number of trials and errors ;-)
>>>
>>> But here I have something that could potentially leak as well. I could not devise a
>>> reproducer due to the leak being lightly triggered only in extreme memory contention.
>>>
>>> See it for yourself:
>>>
>>> drivers/gpio/gpio-sim.c:
>>> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
>>> 302 {
>>> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
>>> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
>>> 305 unsigned int num_lines = chip->gc.ngpio;
>>> 306 struct device *dev = chip->gc.parent;
>>> 307 struct attribute_group *attr_group;
>>> 308 struct attribute **attrs;
>>> 309 int i, ret;
>>> 310
>>> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
>>> 312 num_lines + 1, GFP_KERNEL);
>>> 313 if (!chip->attr_groups)
>>> 314 return -ENOMEM;
>>> 315
>>> 316 for (i = 0; i < num_lines; i++) {
>>> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
>>> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
>>> 319 GFP_KERNEL);
>>> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
>>> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
>>> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
>>> 323 return -ENOMEM;
>>> 324
>>> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
>>> 326 "sim_gpio%u", i);
>>> 327 if (!attr_group->name)
>>> 328 return -ENOMEM;
>>>
>>> Apparently, if the memory allocation only partially succeeds, in the theoretical case
>>> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
>>> free the partially succeeded allocs, would it?
>>>
>>> To explain it better, I tried a version that is not yet full doing "all or nothing"
>>> memory allocation for the gpio-sim driver, because I am not that familiar with the
>>> driver internals.
>>
>> devm_*() mean that the resource allocation is made in a managed manner, so when
>> it's done, it will be freed automatically.
>
> Didn't see that one coming ... :-/ "buzzing though the bush ..."
>
>> The question is: is the lifetime of the attr_groups should be lesser or the
>> same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
>
> Bona fide said, I hope that automatic deallocation does things in the right order.
> I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
> a per driver list. But I am not sure how chip->gc was allocated?
>
> Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
> as a part of
>
> struct gpio_sim_chip *chip;
> struct gpio_chip *gc;
>
> gc = &chip->gc;
>
> and gc->parent is set to
>
> gc->parent = dev;
>
> in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
P.S.
(I am sorry, but my Thunderbird made unreadable mess from the previous reply.)
The exact line is:
chip = devm_kzalloc (dev, sizeof(*chip), GFP_KERNEL);
so I guess it is reasonable to assume that chip will also be deallocated after attr_groups.
chip->gc.parent appears to be a mere pointer to dev parameter in
static int gpio_sim_add_bank(struct fwnode_handle *swnode, struct device *dev)
This is OTOH called from:
static int gpio_sim_probe (struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct fwnode_handle *swnode;
int ret;
device_for_each_child_node(dev, swnode) {
ret = gpio_sim_add_bank(swnode, dev);
Which means dev passed to chip->gc.parent is initialised with &pdev->dev from
pdev parm of gpio_sim_probe().
This is OTOH referenced from the very:
static struct platform_driver gpio_sim_driver = {
.driver = {
.name = "gpio-sim",
.of_match_table = gpio_sim_of_match,
},
.probe = gpio_sim_probe
};
Hope this helps.
There's more to this than meets the eye, but this is really an idiot's attempt to
analyse a Linux kernel driver. 😄
> If I understood well, automatic deallocation on unloading the driver goes
> in the reverse order, so lifetime of chip appears to be longer than attr_groups,
> but I am really not that good at this ...
>
>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>> device, but then it's unclear how to provide the attributes in non-racy way
> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>
> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> device expert nor would be appropriate to try the craft not earned ;-)
>
> Regards,
> Mirsad
>
--
Mirsad Todorovac
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb
Republic of Croatia, the European Union
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-21 13:52 ` Mirsad Goran Todorovac
2023-02-21 14:20 ` Mirsad Goran Todorovac
2023-02-21 14:32 ` Mirsad Goran Todorovac
@ 2023-02-21 15:39 ` Andy Shevchenko
2023-02-22 10:53 ` Bartosz Golaszewski
2023-02-24 15:12 ` Mirsad Todorovac
2 siblings, 2 replies; 23+ messages in thread
From: Andy Shevchenko @ 2023-02-21 15:39 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
> On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
> > On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
> > > On 2/16/23 15:16, Bartosz Golaszewski wrote:
...
> > > As Mr. McKenney once said, a bunch of monkeys with keyboard could
> > > have done it in a considerable number of trials and errors ;-)
> > >
> > > But here I have something that could potentially leak as well. I could not devise a
> > > reproducer due to the leak being lightly triggered only in extreme memory contention.
> > >
> > > See it for yourself:
> > >
> > > drivers/gpio/gpio-sim.c:
> > > 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
> > > 302 {
> > > 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
> > > 304 struct gpio_sim_attribute *val_attr, *pull_attr;
> > > 305 unsigned int num_lines = chip->gc.ngpio;
> > > 306 struct device *dev = chip->gc.parent;
> > > 307 struct attribute_group *attr_group;
> > > 308 struct attribute **attrs;
> > > 309 int i, ret;
> > > 310
> > > 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
> > > 312 num_lines + 1, GFP_KERNEL);
> > > 313 if (!chip->attr_groups)
> > > 314 return -ENOMEM;
> > > 315
> > > 316 for (i = 0; i < num_lines; i++) {
> > > 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
> > > 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
> > > 319 GFP_KERNEL);
> > > 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
> > > 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
> > > 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
> > > 323 return -ENOMEM;
> > > 324
> > > 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
> > > 326 "sim_gpio%u", i);
> > > 327 if (!attr_group->name)
> > > 328 return -ENOMEM;
> > >
> > > Apparently, if the memory allocation only partially succeeds, in the theoretical case
> > > that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
> > > free the partially succeeded allocs, would it?
> > >
> > > To explain it better, I tried a version that is not yet full doing "all or nothing"
> > > memory allocation for the gpio-sim driver, because I am not that familiar with the
> > > driver internals.
> >
> > devm_*() mean that the resource allocation is made in a managed manner, so when
> > it's done, it will be freed automatically.
>
> Didn't see that one coming ... :-/ "buzzing though the bush ..."
>
> > The question is: is the lifetime of the attr_groups should be lesser or the
> > same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
>
> Bona fide said, I hope that automatic deallocation does things in the right order.
> I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
> a per driver list. But I am not sure how chip->gc was allocated?
>
> Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
> as a part of
>
> struct gpio_sim_chip *chip;
> struct gpio_chip *gc;
>
> gc = &chip->gc;
>
> and gc->parent is set to
>
> gc->parent = dev;
>
> in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
>
> If I understood well, automatic deallocation on unloading the driver goes
> in the reverse order, so lifetime of chip appears to be longer than attr_groups,
> but I am really not that good at this ...
So, the device is instantiated by platform_device_register_full().
It should gone with the platform_device_unregister().
In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
asynchronously.
So, there are following questions:
- is the put_device() is actually called?
- is the above mentioned option is set to Y?
- if it's in Y, does kmemleak take it into account?
- if no, do you get anything new in `dmesg` when enable it?
> > Or maybe the chip->gc.parent should be changed to something else (actual GPIO
> > device, but then it's unclear how to provide the attributes in non-racy way
> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>
> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> device expert nor would be appropriate to try the craft not earned ;-)
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-21 15:39 ` Andy Shevchenko
@ 2023-02-22 10:53 ` Bartosz Golaszewski
2023-02-22 21:27 ` Mirsad Goran Todorovac
2023-02-24 15:12 ` Mirsad Todorovac
1 sibling, 1 reply; 23+ messages in thread
From: Bartosz Golaszewski @ 2023-02-22 10:53 UTC (permalink / raw)
To: Mirsad Goran Todorovac
Cc: linux-gpio, Linus Walleij, linux-kernel, Thorsten Leemhuis,
Andy Shevchenko
On Tue, Feb 21, 2023 at 4:41 PM Andy Shevchenko
<andriy.shevchenko@intel.com> wrote:
>
> On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
> > On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
> > > On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
> > > > On 2/16/23 15:16, Bartosz Golaszewski wrote:
>
> ...
>
> > > > As Mr. McKenney once said, a bunch of monkeys with keyboard could
> > > > have done it in a considerable number of trials and errors ;-)
> > > >
> > > > But here I have something that could potentially leak as well. I could not devise a
> > > > reproducer due to the leak being lightly triggered only in extreme memory contention.
> > > >
> > > > See it for yourself:
> > > >
> > > > drivers/gpio/gpio-sim.c:
> > > > 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
> > > > 302 {
> > > > 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
> > > > 304 struct gpio_sim_attribute *val_attr, *pull_attr;
> > > > 305 unsigned int num_lines = chip->gc.ngpio;
> > > > 306 struct device *dev = chip->gc.parent;
> > > > 307 struct attribute_group *attr_group;
> > > > 308 struct attribute **attrs;
> > > > 309 int i, ret;
> > > > 310
> > > > 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
> > > > 312 num_lines + 1, GFP_KERNEL);
> > > > 313 if (!chip->attr_groups)
> > > > 314 return -ENOMEM;
> > > > 315
> > > > 316 for (i = 0; i < num_lines; i++) {
> > > > 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
> > > > 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
> > > > 319 GFP_KERNEL);
> > > > 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
> > > > 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
> > > > 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
> > > > 323 return -ENOMEM;
> > > > 324
> > > > 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
> > > > 326 "sim_gpio%u", i);
> > > > 327 if (!attr_group->name)
> > > > 328 return -ENOMEM;
> > > >
> > > > Apparently, if the memory allocation only partially succeeds, in the theoretical case
> > > > that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
> > > > free the partially succeeded allocs, would it?
> > > >
> > > > To explain it better, I tried a version that is not yet full doing "all or nothing"
> > > > memory allocation for the gpio-sim driver, because I am not that familiar with the
> > > > driver internals.
> > >
> > > devm_*() mean that the resource allocation is made in a managed manner, so when
> > > it's done, it will be freed automatically.
> >
> > Didn't see that one coming ... :-/ "buzzing though the bush ..."
> >
> > > The question is: is the lifetime of the attr_groups should be lesser or the
> > > same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
> >
> > Bona fide said, I hope that automatic deallocation does things in the right order.
> > I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
> > a per driver list. But I am not sure how chip->gc was allocated?
> >
> > Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
> > as a part of
> >
> > struct gpio_sim_chip *chip;
> > struct gpio_chip *gc;
> >
> > gc = &chip->gc;
> >
> > and gc->parent is set to
> >
> > gc->parent = dev;
> >
> > in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
> >
> > If I understood well, automatic deallocation on unloading the driver goes
> > in the reverse order, so lifetime of chip appears to be longer than attr_groups,
> > but I am really not that good at this ...
>
> So, the device is instantiated by platform_device_register_full().
>
> It should gone with the platform_device_unregister().
>
> In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
> asynchronously.
>
> So, there are following questions:
> - is the put_device() is actually called?
> - is the above mentioned option is set to Y?
> - if it's in Y, does kmemleak take it into account?
> - if no, do you get anything new in `dmesg` when enable it?
>
> > > Or maybe the chip->gc.parent should be changed to something else (actual GPIO
> > > device, but then it's unclear how to provide the attributes in non-racy way
> > Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
> >
> > I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> > device expert nor would be appropriate to try the craft not earned ;-)
>
> --
> With Best Regards,
> Andy Shevchenko
>
>
Mirsad,
I think you fear that the memory allocated for sysfs attributes could
be accessed after the driver is detached from the simulated GPIO
device? This is not possible as sysfs handles that gracefully (by
removing all sysfs attributes with driver_sysfs_remove()) before
freeing devres resources. You can test that yourself by instantiating
a gpio-sim device, opening and holding a file descriptor to one of the
sysfs attributes, disabling the device and then trying to read from
said fd - it will return -ENODEV.
Let me know if you actually mean something else?
Bart
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-22 10:53 ` Bartosz Golaszewski
@ 2023-02-22 21:27 ` Mirsad Goran Todorovac
0 siblings, 0 replies; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-02-22 21:27 UTC (permalink / raw)
To: Bartosz Golaszewski
Cc: linux-gpio, Linus Walleij, linux-kernel, Thorsten Leemhuis,
Andy Shevchenko
On 22. 02. 2023. 11:53, Bartosz Golaszewski wrote:
> On Tue, Feb 21, 2023 at 4:41 PM Andy Shevchenko
> <andriy.shevchenko@intel.com> wrote:
>>
>> On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
>>> On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
>>>> On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
>>>>> On 2/16/23 15:16, Bartosz Golaszewski wrote:
>>
>> ...
>>
>>>>> As Mr. McKenney once said, a bunch of monkeys with keyboard could
>>>>> have done it in a considerable number of trials and errors ;-)
>>>>>
>>>>> But here I have something that could potentially leak as well. I could not devise a
>>>>> reproducer due to the leak being lightly triggered only in extreme memory contention.
>>>>>
>>>>> See it for yourself:
>>>>>
>>>>> drivers/gpio/gpio-sim.c:
>>>>> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
>>>>> 302 {
>>>>> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
>>>>> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
>>>>> 305 unsigned int num_lines = chip->gc.ngpio;
>>>>> 306 struct device *dev = chip->gc.parent;
>>>>> 307 struct attribute_group *attr_group;
>>>>> 308 struct attribute **attrs;
>>>>> 309 int i, ret;
>>>>> 310
>>>>> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
>>>>> 312 num_lines + 1, GFP_KERNEL);
>>>>> 313 if (!chip->attr_groups)
>>>>> 314 return -ENOMEM;
>>>>> 315
>>>>> 316 for (i = 0; i < num_lines; i++) {
>>>>> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
>>>>> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
>>>>> 319 GFP_KERNEL);
>>>>> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
>>>>> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
>>>>> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
>>>>> 323 return -ENOMEM;
>>>>> 324
>>>>> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
>>>>> 326 "sim_gpio%u", i);
>>>>> 327 if (!attr_group->name)
>>>>> 328 return -ENOMEM;
>>>>>
>>>>> Apparently, if the memory allocation only partially succeeds, in the theoretical case
>>>>> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
>>>>> free the partially succeeded allocs, would it?
>>>>>
>>>>> To explain it better, I tried a version that is not yet full doing "all or nothing"
>>>>> memory allocation for the gpio-sim driver, because I am not that familiar with the
>>>>> driver internals.
>>>>
>>>> devm_*() mean that the resource allocation is made in a managed manner, so when
>>>> it's done, it will be freed automatically.
>>>
>>> Didn't see that one coming ... :-/ "buzzing though the bush ..."
>>>
>>>> The question is: is the lifetime of the attr_groups should be lesser or the
>>>> same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
>>>
>>> Bona fide said, I hope that automatic deallocation does things in the right order.
>>> I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
>>> a per driver list. But I am not sure how chip->gc was allocated?
>>>
>>> Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
>>> as a part of
>>>
>>> struct gpio_sim_chip *chip;
>>> struct gpio_chip *gc;
>>>
>>> gc = &chip->gc;
>>>
>>> and gc->parent is set to
>>>
>>> gc->parent = dev;
>>>
>>> in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
>>>
>>> If I understood well, automatic deallocation on unloading the driver goes
>>> in the reverse order, so lifetime of chip appears to be longer than attr_groups,
>>> but I am really not that good at this ...
>>
>> So, the device is instantiated by platform_device_register_full().
>>
>> It should gone with the platform_device_unregister().
>>
>> In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
>> asynchronously.
>>
>> So, there are following questions:
>> - is the put_device() is actually called?
>> - is the above mentioned option is set to Y?
>> - if it's in Y, does kmemleak take it into account?
>> - if no, do you get anything new in `dmesg` when enable it?
>>
>>>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>>>> device, but then it's unclear how to provide the attributes in non-racy way
>>> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>>>
>>> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
>>> device expert nor would be appropriate to try the craft not earned ;-)
>>
>> --
>> With Best Regards,
>> Andy Shevchenko
> Mirsad,
>
> I think you fear that the memory allocated for sysfs attributes could
> be accessed after the driver is detached from the simulated GPIO
> device? This is not possible as sysfs handles that gracefully (by
> removing all sysfs attributes with driver_sysfs_remove()) before
> freeing devres resources. You can test that yourself by instantiating
> a gpio-sim device, opening and holding a file descriptor to one of the
> sysfs attributes, disabling the device and then trying to read from
> said fd - it will return -ENODEV.
>
> Let me know if you actually mean something else?
Hi Bartosz,
I have to repeat that I am not an expert in the Linux kernel
drivers, but have only found the leak using the DEBUG_KMEMLEAK
kernel tweak.
After (in gpio/gpio-sim.sh):
SYSFS_PATH="/sys/devices/platform/$DEVNAME/$CHIPNAME/sim_gpio0/value"
echo "4.1b Disabling gpio-sim device $DEVNAME $CHIPNAME $SYSFS_PATH"
tail -f "$SYSFS_PATH" &
sleep 3
echo 0 > /sys/kernel/config/gpio-sim/chip/live
I get "No such file or directory" when trying to read from sysfs parameters:
Failed to open /dev/none: No such file or directory
Actually, this apparently comes from the tail command unexpectedly using
inotify()
to monitor changes on the .../sim_gpio0/value, and reopening the file.
This sounds like a good idea for a gpio-sim test, as it might be impractical
to change the gpio-sim.sh source for a newbie kernel tester and they may obviously
lack your insight.
However, at my level I am content that the leak is patched at its core
source. It is human to make such omission, and may I remind you that one time
a satellite was lost due to an error traced down to a single wrong character
or digit.
Finding a bug only reminds me of my own bugs in much less complicated
situations with fewer gotchas ... I wonder if such cases could be discovered
by static analysis, but from the given loop, I simply do not see how a robot
could have guessed the intent of the test ...
Let's stay positive, a leak is fixed, and this was a great experience with
a great team.
Further hardening of the driver might be a good thing, but I suppose the way is
in what Andy proposed, but I am still doing a homework on that one.
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-21 15:39 ` Andy Shevchenko
2023-02-22 10:53 ` Bartosz Golaszewski
@ 2023-02-24 15:12 ` Mirsad Todorovac
2023-02-24 17:40 ` Andy Shevchenko
1 sibling, 1 reply; 23+ messages in thread
From: Mirsad Todorovac @ 2023-02-24 15:12 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 2/21/23 16:39, Andy Shevchenko wrote:
> On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
>> On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
>>> On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
>>>> On 2/16/23 15:16, Bartosz Golaszewski wrote:
>
> ...
>
>>>> As Mr. McKenney once said, a bunch of monkeys with keyboard could
>>>> have done it in a considerable number of trials and errors ;-)
>>>>
>>>> But here I have something that could potentially leak as well. I could not devise a
>>>> reproducer due to the leak being lightly triggered only in extreme memory contention.
>>>>
>>>> See it for yourself:
>>>>
>>>> drivers/gpio/gpio-sim.c:
>>>> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
>>>> 302 {
>>>> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
>>>> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
>>>> 305 unsigned int num_lines = chip->gc.ngpio;
>>>> 306 struct device *dev = chip->gc.parent;
>>>> 307 struct attribute_group *attr_group;
>>>> 308 struct attribute **attrs;
>>>> 309 int i, ret;
>>>> 310
>>>> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
>>>> 312 num_lines + 1, GFP_KERNEL);
>>>> 313 if (!chip->attr_groups)
>>>> 314 return -ENOMEM;
>>>> 315
>>>> 316 for (i = 0; i < num_lines; i++) {
>>>> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
>>>> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
>>>> 319 GFP_KERNEL);
>>>> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
>>>> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
>>>> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
>>>> 323 return -ENOMEM;
>>>> 324
>>>> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
>>>> 326 "sim_gpio%u", i);
>>>> 327 if (!attr_group->name)
>>>> 328 return -ENOMEM;
>>>>
>>>> Apparently, if the memory allocation only partially succeeds, in the theoretical case
>>>> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
>>>> free the partially succeeded allocs, would it?
>>>>
>>>> To explain it better, I tried a version that is not yet full doing "all or nothing"
>>>> memory allocation for the gpio-sim driver, because I am not that familiar with the
>>>> driver internals.
>>>
>>> devm_*() mean that the resource allocation is made in a managed manner, so when
>>> it's done, it will be freed automatically.
>>
>> Didn't see that one coming ... :-/ "buzzing though the bush ..."
>>
>>> The question is: is the lifetime of the attr_groups should be lesser or the
>>> same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
>>
>> Bona fide said, I hope that automatic deallocation does things in the right order.
>> I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
>> a per driver list. But I am not sure how chip->gc was allocated?
>>
>> Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
>> as a part of
>>
>> struct gpio_sim_chip *chip;
>> struct gpio_chip *gc;
>>
>> gc = &chip->gc;
>>
>> and gc->parent is set to
>>
>> gc->parent = dev;
>>
>> in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
>>
>> If I understood well, automatic deallocation on unloading the driver goes
>> in the reverse order, so lifetime of chip appears to be longer than attr_groups,
>> but I am really not that good at this ...
>
> So, the device is instantiated by platform_device_register_full().
>
> It should gone with the platform_device_unregister().
>
> In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
> asynchronously.
>
> So, there are following questions:
> - is the put_device() is actually called?
> - is the above mentioned option is set to Y?
> - if it's in Y, does kmemleak take it into account?
> - if no, do you get anything new in `dmesg` when enable it?
Hi, Andy,
Having set CONFIG_DEBUG_KOBJECT=y.
CONFIG_DEBUG_KOBJECT_RELEASE=y and CONFIG_DEBUG_OBJECTS_TIMERS=y disappear after "make olddefconfig"
So, I cannot tell about whether release() was called asynchronously, all I get is (after driver unload):
[ 810.989742] kobject: 'gpio-sim' (00000000251afa19): kobject_cleanup, parent 00000000447da7a7
[ 810.990216] kobject: 'gpio-sim' (00000000251afa19): auto cleanup kobject_del
[ 810.990674] kobject: 'gpio-sim' (00000000251afa19): auto cleanup 'remove' event
[ 810.991175] kobject: 'gpio-sim' (00000000251afa19): kobject_uevent_env
[ 810.991674] kobject: 'gpio-sim' (00000000251afa19): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
[ 810.992154] kobject: 'gpio-sim' (00000000251afa19): calling ktype release
[ 810.992644] kobject: 'gpio-sim': free name
I am still trying to convince "make olddefconfig" to accept the above values he did not like :-/
Regards,
Mirsad
>>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>>> device, but then it's unclear how to provide the attributes in non-racy way
>> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>>
>> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
>> device expert nor would be appropriate to try the craft not earned ;-)
>
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-24 15:12 ` Mirsad Todorovac
@ 2023-02-24 17:40 ` Andy Shevchenko
2023-02-27 18:38 ` Mirsad Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Andy Shevchenko @ 2023-02-24 17:40 UTC (permalink / raw)
To: Mirsad Todorovac
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On Fri, Feb 24, 2023 at 04:12:31PM +0100, Mirsad Todorovac wrote:
> On 2/21/23 16:39, Andy Shevchenko wrote:
> > On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
> > > On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
> > > > On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
> > > > > On 2/16/23 15:16, Bartosz Golaszewski wrote:
...
> > > > > As Mr. McKenney once said, a bunch of monkeys with keyboard could
> > > > > have done it in a considerable number of trials and errors ;-)
> > > > >
> > > > > But here I have something that could potentially leak as well. I could not devise a
> > > > > reproducer due to the leak being lightly triggered only in extreme memory contention.
> > > > >
> > > > > See it for yourself:
> > > > >
> > > > > drivers/gpio/gpio-sim.c:
> > > > > 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
> > > > > 302 {
> > > > > 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
> > > > > 304 struct gpio_sim_attribute *val_attr, *pull_attr;
> > > > > 305 unsigned int num_lines = chip->gc.ngpio;
> > > > > 306 struct device *dev = chip->gc.parent;
> > > > > 307 struct attribute_group *attr_group;
> > > > > 308 struct attribute **attrs;
> > > > > 309 int i, ret;
> > > > > 310
> > > > > 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
> > > > > 312 num_lines + 1, GFP_KERNEL);
> > > > > 313 if (!chip->attr_groups)
> > > > > 314 return -ENOMEM;
> > > > > 315
> > > > > 316 for (i = 0; i < num_lines; i++) {
> > > > > 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
> > > > > 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
> > > > > 319 GFP_KERNEL);
> > > > > 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
> > > > > 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
> > > > > 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
> > > > > 323 return -ENOMEM;
> > > > > 324
> > > > > 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
> > > > > 326 "sim_gpio%u", i);
> > > > > 327 if (!attr_group->name)
> > > > > 328 return -ENOMEM;
> > > > >
> > > > > Apparently, if the memory allocation only partially succeeds, in the theoretical case
> > > > > that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
> > > > > free the partially succeeded allocs, would it?
> > > > >
> > > > > To explain it better, I tried a version that is not yet full doing "all or nothing"
> > > > > memory allocation for the gpio-sim driver, because I am not that familiar with the
> > > > > driver internals.
> > > >
> > > > devm_*() mean that the resource allocation is made in a managed manner, so when
> > > > it's done, it will be freed automatically.
> > >
> > > Didn't see that one coming ... :-/ "buzzing though the bush ..."
> > >
> > > > The question is: is the lifetime of the attr_groups should be lesser or the
> > > > same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
> > >
> > > Bona fide said, I hope that automatic deallocation does things in the right order.
> > > I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
> > > a per driver list. But I am not sure how chip->gc was allocated?
> > >
> > > Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
> > > as a part of
> > >
> > > struct gpio_sim_chip *chip;
> > > struct gpio_chip *gc;
> > >
> > > gc = &chip->gc;
> > >
> > > and gc->parent is set to
> > >
> > > gc->parent = dev;
> > >
> > > in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
> > >
> > > If I understood well, automatic deallocation on unloading the driver goes
> > > in the reverse order, so lifetime of chip appears to be longer than attr_groups,
> > > but I am really not that good at this ...
> >
> > So, the device is instantiated by platform_device_register_full().
> >
> > It should gone with the platform_device_unregister().
> >
> > In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
> > asynchronously.
> >
> > So, there are following questions:
> > - is the put_device() is actually called?
> > - is the above mentioned option is set to Y?
> > - if it's in Y, does kmemleak take it into account?
> > - if no, do you get anything new in `dmesg` when enable it?
>
> Hi, Andy,
>
> Having set CONFIG_DEBUG_KOBJECT=y.
> CONFIG_DEBUG_KOBJECT_RELEASE=y and CONFIG_DEBUG_OBJECTS_TIMERS=y disappear after "make olddefconfig"
>
> So, I cannot tell about whether release() was called asynchronously, all I get is (after driver unload):
>
> [ 810.989742] kobject: 'gpio-sim' (00000000251afa19): kobject_cleanup, parent 00000000447da7a7
> [ 810.990216] kobject: 'gpio-sim' (00000000251afa19): auto cleanup kobject_del
> [ 810.990674] kobject: 'gpio-sim' (00000000251afa19): auto cleanup 'remove' event
> [ 810.991175] kobject: 'gpio-sim' (00000000251afa19): kobject_uevent_env
> [ 810.991674] kobject: 'gpio-sim' (00000000251afa19): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> [ 810.992154] kobject: 'gpio-sim' (00000000251afa19): calling ktype release
> [ 810.992644] kobject: 'gpio-sim': free name
>
> I am still trying to convince "make olddefconfig" to accept the above values he did not like :-/
Btw, since devm_*() is in use, you may enable trace events (I have added them a
few releases back) and see how objects allocated with devm are getting
released.
Before that there is a possibility to enable debug for devm.
CONFIG_DEBUG_DEVRES for that (you may see how it's used in the
drivers/base/devres.c).
> > > > Or maybe the chip->gc.parent should be changed to something else (actual GPIO
> > > > device, but then it's unclear how to provide the attributes in non-racy way
> > > Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
> > >
> > > I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> > > device expert nor would be appropriate to try the craft not earned ;-)
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-24 17:40 ` Andy Shevchenko
@ 2023-02-27 18:38 ` Mirsad Todorovac
2023-02-27 23:13 ` Andy Shevchenko
0 siblings, 1 reply; 23+ messages in thread
From: Mirsad Todorovac @ 2023-02-27 18:38 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 2/24/23 18:40, Andy Shevchenko wrote:
> On Fri, Feb 24, 2023 at 04:12:31PM +0100, Mirsad Todorovac wrote:
>> On 2/21/23 16:39, Andy Shevchenko wrote:
>>> On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
>>>> On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
>>>>> On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
>>>>>> On 2/16/23 15:16, Bartosz Golaszewski wrote:
>
> ...
>
>>>>>> As Mr. McKenney once said, a bunch of monkeys with keyboard could
>>>>>> have done it in a considerable number of trials and errors ;-)
>>>>>>
>>>>>> But here I have something that could potentially leak as well. I could not devise a
>>>>>> reproducer due to the leak being lightly triggered only in extreme memory contention.
>>>>>>
>>>>>> See it for yourself:
>>>>>>
>>>>>> drivers/gpio/gpio-sim.c:
>>>>>> 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
>>>>>> 302 {
>>>>>> 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
>>>>>> 304 struct gpio_sim_attribute *val_attr, *pull_attr;
>>>>>> 305 unsigned int num_lines = chip->gc.ngpio;
>>>>>> 306 struct device *dev = chip->gc.parent;
>>>>>> 307 struct attribute_group *attr_group;
>>>>>> 308 struct attribute **attrs;
>>>>>> 309 int i, ret;
>>>>>> 310
>>>>>> 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
>>>>>> 312 num_lines + 1, GFP_KERNEL);
>>>>>> 313 if (!chip->attr_groups)
>>>>>> 314 return -ENOMEM;
>>>>>> 315
>>>>>> 316 for (i = 0; i < num_lines; i++) {
>>>>>> 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
>>>>>> 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
>>>>>> 319 GFP_KERNEL);
>>>>>> 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
>>>>>> 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
>>>>>> 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
>>>>>> 323 return -ENOMEM;
>>>>>> 324
>>>>>> 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
>>>>>> 326 "sim_gpio%u", i);
>>>>>> 327 if (!attr_group->name)
>>>>>> 328 return -ENOMEM;
>>>>>>
>>>>>> Apparently, if the memory allocation only partially succeeds, in the theoretical case
>>>>>> that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
>>>>>> free the partially succeeded allocs, would it?
>>>>>>
>>>>>> To explain it better, I tried a version that is not yet full doing "all or nothing"
>>>>>> memory allocation for the gpio-sim driver, because I am not that familiar with the
>>>>>> driver internals.
>>>>>
>>>>> devm_*() mean that the resource allocation is made in a managed manner, so when
>>>>> it's done, it will be freed automatically.
>>>>
>>>> Didn't see that one coming ... :-/ "buzzing though the bush ..."
>>>>
>>>>> The question is: is the lifetime of the attr_groups should be lesser or the
>>>>> same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
>>>>
>>>> Bona fide said, I hope that automatic deallocation does things in the right order.
>>>> I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
>>>> a per driver list. But I am not sure how chip->gc was allocated?
>>>>
>>>> Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
>>>> as a part of
>>>>
>>>> struct gpio_sim_chip *chip;
>>>> struct gpio_chip *gc;
>>>>
>>>> gc = &chip->gc;
>>>>
>>>> and gc->parent is set to
>>>>
>>>> gc->parent = dev;
>>>>
>>>> in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
>>>>
>>>> If I understood well, automatic deallocation on unloading the driver goes
>>>> in the reverse order, so lifetime of chip appears to be longer than attr_groups,
>>>> but I am really not that good at this ...
>>>
>>> So, the device is instantiated by platform_device_register_full().
>>>
>>> It should gone with the platform_device_unregister().
>>>
>>> In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
>>> asynchronously.
>>>
>>> So, there are following questions:
>>> - is the put_device() is actually called?
>>> - is the above mentioned option is set to Y?
>>> - if it's in Y, does kmemleak take it into account?
>>> - if no, do you get anything new in `dmesg` when enable it?
>>
>> Hi, Andy,
>>
>> Having set CONFIG_DEBUG_KOBJECT=y.
>> CONFIG_DEBUG_KOBJECT_RELEASE=y and CONFIG_DEBUG_OBJECTS_TIMERS=y disappear after "make olddefconfig"
>>
>> So, I cannot tell about whether release() was called asynchronously, all I get is (after driver unload):
>>
>> [ 810.989742] kobject: 'gpio-sim' (00000000251afa19): kobject_cleanup, parent 00000000447da7a7
>> [ 810.990216] kobject: 'gpio-sim' (00000000251afa19): auto cleanup kobject_del
>> [ 810.990674] kobject: 'gpio-sim' (00000000251afa19): auto cleanup 'remove' event
>> [ 810.991175] kobject: 'gpio-sim' (00000000251afa19): kobject_uevent_env
>> [ 810.991674] kobject: 'gpio-sim' (00000000251afa19): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
>> [ 810.992154] kobject: 'gpio-sim' (00000000251afa19): calling ktype release
>> [ 810.992644] kobject: 'gpio-sim': free name
>>
>> I am still trying to convince "make olddefconfig" to accept the above values he did not like :-/
>
> Btw, since devm_*() is in use, you may enable trace events (I have added them a
> few releases back) and see how objects allocated with devm are getting
> released.
>
> Before that there is a possibility to enable debug for devm.
> CONFIG_DEBUG_DEVRES for that (you may see how it's used in the
> drivers/base/devres.c).
The loga are extensive if you are willing to skim over them, but I believe the interesting
part is this:
[ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
[ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
[ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
[ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
[ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
[ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
[ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
[ 4833.908315] kobject: 'gpio-sim': free name
[ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
[ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
[ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
[ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
[ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
[ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
[ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
[ 4835.952448] kobject: 'gpio_sim': free name
Or, with CONFIG_DEBUG_DEVRES=y, it looks like this:
[ 180.472131] kobject: 'gpio-sim' (00000000e72df678): kobject_release, parent 0000000053a50b05 (delayed 250)
[ 180.472270] kobject: 'drivers' (00000000a3c60031): kobject_release, parent 00000000363d6076 (delayed 250)
[ 180.472274] kobject: 'holders' (000000006bd5c478): kobject_release, parent 00000000363d6076 (delayed 250)
[ 180.472281] kobject: 'notes' (00000000c85ae91b): kobject_release, parent 00000000363d6076 (delayed 1000)
[ 181.495687] kobject: 'holders' (000000006bd5c478): kobject_cleanup, parent 00000000363d6076
[ 181.495693] kobject: 'holders' (000000006bd5c478): auto cleanup kobject_del
[ 181.495696] kobject: 'holders' (000000006bd5c478): calling ktype release
[ 181.495697] kobject: (000000006bd5c478): dynamic_kobj_release
[ 181.495703] kobject: 'holders': free name
[ 181.495717] kobject: 'drivers' (00000000a3c60031): kobject_cleanup, parent 00000000363d6076
[ 181.495719] kobject: 'drivers' (00000000a3c60031): auto cleanup kobject_del
[ 181.495723] kobject: 'drivers' (00000000a3c60031): calling ktype release
[ 181.495724] kobject: (00000000a3c60031): dynamic_kobj_release
[ 181.495727] kobject: 'drivers': free name
[ 181.495728] kobject: 'gpio-sim' (00000000e72df678): kobject_cleanup, parent 0000000053a50b05
[ 181.495730] kobject: 'gpio-sim' (00000000e72df678): auto cleanup kobject_del
[ 181.495731] kobject: 'gpio-sim' (00000000e72df678): auto cleanup 'remove' event
[ 181.495733] kobject: 'gpio-sim' (00000000e72df678): kobject_uevent_env
[ 181.495742] kobject: 'gpio-sim' (00000000e72df678): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
[ 181.495764] kobject: 'gpio-sim' (00000000e72df678): calling ktype release
[ 181.495768] kobject: 'gpio-sim': free name
[ 184.663810] kobject: 'notes' (00000000c85ae91b): kobject_cleanup, parent 00000000363d6076
[ 184.663816] kobject: 'notes' (00000000c85ae91b): auto cleanup kobject_del
[ 184.663824] kobject: 'notes' (00000000c85ae91b): calling ktype release
[ 184.663838] kobject: (00000000c85ae91b): dynamic_kobj_release
[ 184.663841] kobject: 'notes': free name
[ 184.663843] kobject: 'gpio_sim' (00000000363d6076): kobject_release, parent 00000000dfc73d50 (delayed 500)
[ 186.679811] kobject: 'gpio_sim' (00000000363d6076): kobject_cleanup, parent 00000000dfc73d50
[ 186.679817] kobject: 'gpio_sim' (00000000363d6076): auto cleanup kobject_del
[ 186.679819] kobject: 'gpio_sim' (00000000363d6076): auto cleanup 'remove' event
[ 186.679820] kobject: 'gpio_sim' (00000000363d6076): kobject_uevent_env
[ 186.679832] kobject: 'gpio_sim' (00000000363d6076): fill_kobj_path: path = '/module/gpio_sim'
[ 186.679855] kobject: 'gpio_sim' (00000000363d6076): calling ktype release
[ 186.679859] kobject: 'gpio_sim': free name
Here are the /var/log/messages and `dmesg | grep gpio` output:
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000065b01c31): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip512' (000000008b9638a5): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000e7ddc5ce): kobject_release, parent 000000007063b679 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000052bfa782): kobject_release, parent 0000000017e526f4 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000007e927b76): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip513' (00000000307a0461): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000007025cedf): kobject_release, parent 00000000c6873be9 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000002030b0fc): kobject_release, parent 00000000cd2bbef4 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000008afdcce8): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip514' (000000008e860d8b): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000dc5ac60d): kobject_release, parent 0000000034808fa5 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000401b009a): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000b0dfd44e): kobject_release, parent 000000001a955210 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000a1399cd0): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip515' (000000002b851bb5): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000276535c4): kobject_release, parent 00000000f89849bd (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000b204b566): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000581391d9): kobject_release, parent 000000002a3e415e (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000babdf5db): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip531' (00000000ad4538e8): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (0000000045377336): kobject_release, parent 00000000e06680f5 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000554e0e6d): kobject_release, parent 000000003fe8f71b (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000008b98f863): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip532' (00000000ac07803f): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (0000000007a45939): kobject_release, parent 000000004ef2dca6 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (0000000034300a6b): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000089fb2874): kobject_release, parent 00000000f81b3244 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000dedc22f2): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip533' (000000003541481f): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000002d7008b9): kobject_release, parent 00000000a22546a6 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000bbace83b): kobject_release, parent 0000000050fa69b3 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000e467cb12): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip549' (0000000014379c6d): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000186d3bd6): kobject_release, parent 000000006b8d9c99 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000bb358104): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000ab70f947): kobject_release, parent 000000000876b849 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000016f05b5a): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip551' (00000000a55e630b): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000948e7c59): kobject_release, parent 00000000cb0b3463 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000461517ae): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000b41576d9): kobject_release, parent 0000000060e34f22 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000176df109): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip552' (000000001750e50f): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000008026d597): kobject_release, parent 00000000bdaa0a47 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.1' (0000000018f833bf): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000005a55ef16): kobject_release, parent 000000008dc012bc (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000886e1a77): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip553' (0000000010cfe833): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000d8b3d64b): kobject_release, parent 00000000e3becc55 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000002e06f814): kobject_release, parent 00000000a555c1b2 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000002835f4a4): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip554' (00000000e0038afb): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000004e7dad05): kobject_release, parent 00000000b39a3c5f (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000bfef6047): kobject_release, parent 00000000b8559267 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000d962e90a): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip555' (00000000f5bbd198): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000af8b42a8): kobject_release, parent 0000000051b24fd2 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000008ef7f514): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000041578d2d): kobject_release, parent 00000000b738a3e4 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: gpio-sim: probe of gpio-sim.0 failed with error -34
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000005418b2af): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000001f644365): kobject_release, parent 00000000515b4b1b (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: gpio-560 (?): hogged as input
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000006cce49d7): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip556' (00000000d2b04f32): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000843ea14e): kobject_release, parent 00000000dc27f1aa (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000030fbb9d7): kobject_release, parent 000000001576402e (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000007164bff6): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip564' (00000000ee87da7b): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000009eaeda39): kobject_release, parent 0000000092a079b6 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000006053a4bf): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000032fe601a): kobject_release, parent 0000000063e66fd0 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000012ac3b4a): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip572' (00000000178509ff): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (0000000018c19582): kobject_release, parent 00000000b0535d8d (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000c56fc456): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000095747aa3): kobject_release, parent 00000000c314f13a (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000000333f422): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip580' (00000000f6371a1e): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000006dc3da55): kobject_release, parent 000000002be607c1 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000000471ebbe): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000008cc134db): kobject_release, parent 000000008af54826 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000005a238f5): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip588' (0000000075b76b85): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000003f03127c): kobject_release, parent 000000006ee06561 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (0000000048f1330d): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000023dec04e): kobject_release, parent 000000005fd3b6d2 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000064e73453): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip589' (000000007e1cf13a): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000cdec14e3): kobject_release, parent 00000000a8ca1f8c (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000f462f78a): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000b5aa4b07): kobject_release, parent 00000000ea466ae7 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000001bd064f6): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip597' (00000000a32f70da): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000aab83a1c): kobject_release, parent 000000008385ae47 (delayed 1000)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000c6c3323c): kobject_release, parent 00000000c2405972 (delayed 250)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node2' (0000000017e526f4): kobject_release, parent 00000000d00083c8 (delayed 500)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node3' (00000000cd2bbef4): kobject_release, parent 00000000d00083c8 (delayed 250)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'gpiochip3' (00000000f89849bd): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node8' (0000000050fa69b3): kobject_release, parent 00000000d00083c8 (delayed 250)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'gpiochip11' (00000000b39a3c5f): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'gpiochip12' (0000000051b24fd2): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node15' (00000000515b4b1b): kobject_release, parent 00000000d00083c8 (delayed 750)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node17' (0000000063e66fd0): kobject_release, parent 00000000d00083c8 (delayed 500)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node20' (000000005fd3b6d2): kobject_release, parent 00000000d00083c8 (delayed 1000)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node19' (000000008af54826): kobject_release, parent 00000000d00083c8 (delayed 750)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node21' (00000000ea466ae7): kobject_release, parent 00000000d00083c8 (delayed 500)
Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node22' (00000000c2405972): kobject_release, parent 00000000d00083c8 (delayed 750)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip2' (0000000034808fa5): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip4' (00000000e06680f5): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'node7' (00000000f81b3244): kobject_release, parent 00000000d00083c8 (delayed 500)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip5' (000000004ef2dca6): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip7' (000000006b8d9c99): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip9' (00000000bdaa0a47): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'node10' (0000000060e34f22): kobject_release, parent 00000000d00083c8 (delayed 250)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip8' (00000000cb0b3463): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'node13' (00000000b8559267): kobject_release, parent 00000000d00083c8 (delayed 1000)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip14' (0000000092a079b6): kobject_release, parent 0000000000000000 (delayed 1000)
Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip15' (00000000b0535d8d): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'gpiochip1' (00000000c6873be9): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node4' (000000001a955210): kobject_release, parent 00000000d00083c8 (delayed 250)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node12' (00000000a555c1b2): kobject_release, parent 00000000d00083c8 (delayed 1000)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node6' (000000003fe8f71b): kobject_release, parent 00000000d00083c8 (delayed 750)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node9' (000000000876b849): kobject_release, parent 00000000d00083c8 (delayed 750)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'gpiochip17' (000000006ee06561): kobject_release, parent 0000000000000000 (delayed 750)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node16' (000000001576402e): kobject_release, parent 00000000d00083c8 (delayed 1000)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'gpiochip13' (00000000dc27f1aa): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node14' (00000000b738a3e4): kobject_release, parent 00000000d00083c8 (delayed 1000)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip0' (000000007063b679): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'node5' (000000002a3e415e): kobject_release, parent 00000000d00083c8 (delayed 1000)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip10' (00000000e3becc55): kobject_release, parent 0000000000000000 (delayed 500)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'node11' (000000008dc012bc): kobject_release, parent 00000000d00083c8 (delayed 500)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip6' (00000000a22546a6): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'node18' (00000000c314f13a): kobject_release, parent 00000000d00083c8 (delayed 250)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip16' (000000002be607c1): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip19' (000000008385ae47): kobject_release, parent 0000000000000000 (delayed 250)
Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'drivers' (00000000bca28211): kobject_release, parent 0000000096ea0bb1 (delayed 500)
Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'holders' (00000000ef926144): kobject_release, parent 0000000096ea0bb1 (delayed 1000)
Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'notes' (000000000c2f27e4): kobject_release, parent 0000000096ea0bb1 (delayed 500)
Feb 27 19:00:47 pc-mtodorov kernel: kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
dmesg is more extensive on the events:
[ 4819.240904] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_add_internal: parent: 'module', set: 'module'
[ 4819.240925] kobject: 'holders' (00000000ef926144): kobject_add_internal: parent: 'gpio_sim', set: '<NULL>'
[ 4819.241067] kobject: 'notes' (000000000c2f27e4): kobject_add_internal: parent: 'gpio_sim', set: '<NULL>'
[ 4819.241106] kobject: 'gpio-sim' (000000005b8d0726): kobject_add_internal: parent: 'drivers', set: 'drivers'
[ 4819.241134] kobject: 'drivers' (00000000bca28211): kobject_add_internal: parent: 'gpio_sim', set: '<NULL>'
[ 4819.241148] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
[ 4819.241155] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
[ 4819.241210] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
[ 4819.241215] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
[ 4819.257117] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.257172] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
[ 4819.257177] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.257297] kobject: 'gpiochip0' (000000007063b679): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.257484] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
[ 4819.257495] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
[ 4819.257533] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
[ 4819.257545] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
[ 4819.257571] kobject: 'gpio' (0000000065b01c31): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.257577] kobject: 'gpiochip512' (000000008b9638a5): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.257629] kobject: 'gpiochip512' (000000008b9638a5): kobject_uevent_env
[ 4819.257634] kobject: 'gpiochip512' (000000008b9638a5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip512'
[ 4819.257687] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
[ 4819.257693] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.275191] kobject: 'gpiochip512' (000000008b9638a5): kobject_uevent_env
[ 4819.275202] kobject: 'gpiochip512' (000000008b9638a5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip512'
[ 4819.275224] kobject: 'gpio' (0000000065b01c31): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.275227] kobject: 'gpiochip512' (000000008b9638a5): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.275329] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
[ 4819.275337] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
[ 4819.275358] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
[ 4819.275363] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
[ 4819.275406] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
[ 4819.275415] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.275432] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
[ 4819.275437] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.275447] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.303376] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.303430] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
[ 4819.303435] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.303541] kobject: 'gpiochip1' (00000000c6873be9): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.303679] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
[ 4819.303689] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
[ 4819.303724] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
[ 4819.303729] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
[ 4819.303752] kobject: 'gpio' (000000007e927b76): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.303756] kobject: 'gpiochip513' (00000000307a0461): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.303800] kobject: 'gpiochip513' (00000000307a0461): kobject_uevent_env
[ 4819.303805] kobject: 'gpiochip513' (00000000307a0461): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip513'
[ 4819.303852] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
[ 4819.303858] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.321116] kobject: 'gpiochip513' (00000000307a0461): kobject_uevent_env
[ 4819.321128] kobject: 'gpiochip513' (00000000307a0461): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip513'
[ 4819.321155] kobject: 'gpio' (000000007e927b76): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.321160] kobject: 'gpiochip513' (00000000307a0461): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.321268] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
[ 4819.321278] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
[ 4819.321299] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
[ 4819.321304] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
[ 4819.321335] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
[ 4819.321343] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.321362] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
[ 4819.321368] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.321378] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.327855] kobject: 'gpio-sim.0' (00000000401b009a): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.327906] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
[ 4819.327911] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.328025] kobject: 'gpiochip2' (0000000034808fa5): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.328130] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
[ 4819.328137] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
[ 4819.328221] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
[ 4819.328228] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
[ 4819.328250] kobject: 'gpio' (000000008afdcce8): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.328255] kobject: 'gpiochip514' (000000008e860d8b): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.328303] kobject: 'gpiochip514' (000000008e860d8b): kobject_uevent_env
[ 4819.328309] kobject: 'gpiochip514' (000000008e860d8b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip514'
[ 4819.328354] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
[ 4819.328360] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.347557] kobject: 'gpiochip514' (000000008e860d8b): kobject_uevent_env
[ 4819.347567] kobject: 'gpiochip514' (000000008e860d8b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip514'
[ 4819.347589] kobject: 'gpio' (000000008afdcce8): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.347593] kobject: 'gpiochip514' (000000008e860d8b): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.347703] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
[ 4819.347711] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
[ 4819.347731] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
[ 4819.347736] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
[ 4819.347774] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
[ 4819.347782] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.347800] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
[ 4819.347806] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.347816] kobject: 'gpio-sim.0' (00000000401b009a): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.355214] kobject: 'gpio-sim.0' (00000000b204b566): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.355265] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
[ 4819.355270] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.355364] kobject: 'gpiochip3' (00000000f89849bd): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.355495] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
[ 4819.355505] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
[ 4819.355537] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
[ 4819.355543] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
[ 4819.355564] kobject: 'gpio' (00000000a1399cd0): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.355568] kobject: 'gpiochip515' (000000002b851bb5): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.355608] kobject: 'gpiochip515' (000000002b851bb5): kobject_uevent_env
[ 4819.355614] kobject: 'gpiochip515' (000000002b851bb5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip515'
[ 4819.355954] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
[ 4819.355961] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.375898] kobject: 'gpiochip515' (000000002b851bb5): kobject_uevent_env
[ 4819.375908] kobject: 'gpiochip515' (000000002b851bb5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip515'
[ 4819.375930] kobject: 'gpio' (00000000a1399cd0): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.375932] kobject: 'gpiochip515' (000000002b851bb5): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.375998] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
[ 4819.376004] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
[ 4819.376018] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
[ 4819.376022] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
[ 4819.376048] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
[ 4819.376055] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.376069] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
[ 4819.376074] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.376083] kobject: 'gpio-sim.0' (00000000b204b566): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.382519] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.382577] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
[ 4819.382584] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.382684] kobject: 'gpiochip4' (00000000e06680f5): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.382826] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
[ 4819.382838] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
[ 4819.382874] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
[ 4819.382880] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
[ 4819.382904] kobject: 'gpio' (00000000babdf5db): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.382909] kobject: 'gpiochip531' (00000000ad4538e8): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.382952] kobject: 'gpiochip531' (00000000ad4538e8): kobject_uevent_env
[ 4819.382958] kobject: 'gpiochip531' (00000000ad4538e8): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip531'
[ 4819.383005] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
[ 4819.383010] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.403312] kobject: 'gpiochip531' (00000000ad4538e8): kobject_uevent_env
[ 4819.403323] kobject: 'gpiochip531' (00000000ad4538e8): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip531'
[ 4819.403343] kobject: 'gpio' (00000000babdf5db): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.403346] kobject: 'gpiochip531' (00000000ad4538e8): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.403445] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
[ 4819.403454] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
[ 4819.403473] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
[ 4819.403477] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
[ 4819.403514] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
[ 4819.403521] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.403537] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
[ 4819.403543] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.403553] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.409746] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.409799] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
[ 4819.409804] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.409899] kobject: 'gpiochip5' (000000004ef2dca6): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.410038] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
[ 4819.410048] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
[ 4819.410081] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
[ 4819.410087] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
[ 4819.410110] kobject: 'gpio' (000000008b98f863): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.410114] kobject: 'gpiochip532' (00000000ac07803f): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.410154] kobject: 'gpiochip532' (00000000ac07803f): kobject_uevent_env
[ 4819.410159] kobject: 'gpiochip532' (00000000ac07803f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip532'
[ 4819.410202] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
[ 4819.410207] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.427858] kobject: 'gpiochip532' (00000000ac07803f): kobject_uevent_env
[ 4819.427870] kobject: 'gpiochip532' (00000000ac07803f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip532'
[ 4819.427891] kobject: 'gpio' (000000008b98f863): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.427895] kobject: 'gpiochip532' (00000000ac07803f): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.427970] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
[ 4819.427976] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
[ 4819.427994] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
[ 4819.427999] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
[ 4819.428044] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
[ 4819.428053] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.428071] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
[ 4819.428077] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.428088] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.440540] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.440596] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
[ 4819.440601] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.440694] kobject: 'gpiochip6' (00000000a22546a6): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.440824] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
[ 4819.440834] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
[ 4819.440869] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
[ 4819.440875] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
[ 4819.440898] kobject: 'gpio' (00000000dedc22f2): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.440903] kobject: 'gpiochip533' (000000003541481f): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.440943] kobject: 'gpiochip533' (000000003541481f): kobject_uevent_env
[ 4819.440948] kobject: 'gpiochip533' (000000003541481f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip533'
[ 4819.441305] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
[ 4819.441311] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.470191] kobject: 'gpiochip533' (000000003541481f): kobject_uevent_env
[ 4819.470202] kobject: 'gpiochip533' (000000003541481f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip533'
[ 4819.470226] kobject: 'gpio' (00000000dedc22f2): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.470230] kobject: 'gpiochip533' (000000003541481f): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.470337] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
[ 4819.470345] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
[ 4819.470364] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
[ 4819.470369] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
[ 4819.470401] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
[ 4819.470408] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.470425] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
[ 4819.470431] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.470441] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.479607] kobject: 'gpio-sim.0' (00000000bb358104): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.479661] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
[ 4819.479667] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.479757] kobject: 'gpiochip7' (000000006b8d9c99): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.479878] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
[ 4819.479885] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
[ 4819.479914] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
[ 4819.479919] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
[ 4819.479939] kobject: 'gpio' (00000000e467cb12): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.479943] kobject: 'gpiochip549' (0000000014379c6d): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.479990] kobject: 'gpiochip549' (0000000014379c6d): kobject_uevent_env
[ 4819.479995] kobject: 'gpiochip549' (0000000014379c6d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip549'
[ 4819.480055] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
[ 4819.480060] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.506928] kobject: 'gpiochip549' (0000000014379c6d): kobject_uevent_env
[ 4819.506940] kobject: 'gpiochip549' (0000000014379c6d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip549'
[ 4819.506961] kobject: 'gpio' (00000000e467cb12): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.506965] kobject: 'gpiochip549' (0000000014379c6d): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.507073] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
[ 4819.507083] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
[ 4819.507104] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
[ 4819.507109] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
[ 4819.507137] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
[ 4819.507145] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.507162] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
[ 4819.507167] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.507178] kobject: 'gpio-sim.0' (00000000bb358104): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.538632] kobject: 'gpio-sim.0' (00000000461517ae): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.538690] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
[ 4819.538695] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.538804] kobject: 'gpiochip8' (00000000cb0b3463): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.538943] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
[ 4819.538953] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
[ 4819.538990] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
[ 4819.538996] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
[ 4819.539020] kobject: 'gpio' (0000000016f05b5a): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.539025] kobject: 'gpiochip551' (00000000a55e630b): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.539066] kobject: 'gpiochip551' (00000000a55e630b): kobject_uevent_env
[ 4819.539072] kobject: 'gpiochip551' (00000000a55e630b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip551'
[ 4819.539118] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
[ 4819.539124] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.545644] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.545706] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
[ 4819.545712] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
[ 4819.545808] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_add_internal: parent: 'gpio-sim.1', set: 'devices'
[ 4819.545937] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
[ 4819.545948] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
[ 4819.545981] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
[ 4819.545986] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
[ 4819.546010] kobject: 'gpio' (00000000176df109): kobject_add_internal: parent: 'gpio-sim.1', set: '(null)'
[ 4819.546014] kobject: 'gpiochip552' (000000001750e50f): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.546056] kobject: 'gpiochip552' (000000001750e50f): kobject_uevent_env
[ 4819.546061] kobject: 'gpiochip552' (000000001750e50f): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpio/gpiochip552'
[ 4819.546105] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
[ 4819.546112] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
[ 4819.553126] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.553181] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
[ 4819.553188] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
[ 4819.553297] kobject: 'gpiochip10' (00000000e3becc55): kobject_add_internal: parent: 'gpio-sim.2', set: 'devices'
[ 4819.553398] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
[ 4819.553404] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
[ 4819.553432] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
[ 4819.553437] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
[ 4819.553458] kobject: 'gpio' (00000000886e1a77): kobject_add_internal: parent: 'gpio-sim.2', set: '(null)'
[ 4819.553463] kobject: 'gpiochip553' (0000000010cfe833): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.553504] kobject: 'gpiochip553' (0000000010cfe833): kobject_uevent_env
[ 4819.553512] kobject: 'gpiochip553' (0000000010cfe833): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpio/gpiochip553'
[ 4819.553561] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
[ 4819.553569] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
[ 4819.568521] kobject: 'gpiochip551' (00000000a55e630b): kobject_uevent_env
[ 4819.568536] kobject: 'gpiochip551' (00000000a55e630b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip551'
[ 4819.568570] kobject: 'gpio' (0000000016f05b5a): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.568574] kobject: 'gpiochip551' (00000000a55e630b): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.568864] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
[ 4819.568871] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
[ 4819.568890] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
[ 4819.568895] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
[ 4819.568932] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
[ 4819.568939] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.568954] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
[ 4819.568960] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.568970] kobject: 'gpio-sim.0' (00000000461517ae): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.583881] kobject: 'gpiochip552' (000000001750e50f): kobject_uevent_env
[ 4819.583892] kobject: 'gpiochip552' (000000001750e50f): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpio/gpiochip552'
[ 4819.583914] kobject: 'gpio' (00000000176df109): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.583918] kobject: 'gpiochip552' (000000001750e50f): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.583988] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
[ 4819.583994] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
[ 4819.584007] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
[ 4819.584013] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
[ 4819.584048] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
[ 4819.584056] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
[ 4819.584072] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
[ 4819.584077] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
[ 4819.584087] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.599281] kobject: 'gpiochip553' (0000000010cfe833): kobject_uevent_env
[ 4819.599292] kobject: 'gpiochip553' (0000000010cfe833): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpio/gpiochip553'
[ 4819.599316] kobject: 'gpio' (00000000886e1a77): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.599319] kobject: 'gpiochip553' (0000000010cfe833): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.599418] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
[ 4819.599426] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
[ 4819.599441] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
[ 4819.599446] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
[ 4819.599479] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
[ 4819.599488] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
[ 4819.599504] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
[ 4819.599509] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
[ 4819.599521] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.606440] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.606495] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
[ 4819.606501] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.606609] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.606742] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
[ 4819.606753] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
[ 4819.606789] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
[ 4819.606795] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
[ 4819.606817] kobject: 'gpio' (000000002835f4a4): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.606821] kobject: 'gpiochip554' (00000000e0038afb): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.606864] kobject: 'gpiochip554' (00000000e0038afb): kobject_uevent_env
[ 4819.606869] kobject: 'gpiochip554' (00000000e0038afb): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip554'
[ 4819.606914] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
[ 4819.606919] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.622045] kobject: 'gpiochip554' (00000000e0038afb): kobject_uevent_env
[ 4819.622055] kobject: 'gpiochip554' (00000000e0038afb): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip554'
[ 4819.622076] kobject: 'gpio' (000000002835f4a4): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.622079] kobject: 'gpiochip554' (00000000e0038afb): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.622179] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
[ 4819.622187] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
[ 4819.622205] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
[ 4819.622210] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
[ 4819.622243] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
[ 4819.622252] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.622268] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
[ 4819.622274] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.622283] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.628405] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.628464] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
[ 4819.628471] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.628588] kobject: 'gpiochip12' (0000000051b24fd2): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.628727] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
[ 4819.628737] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
[ 4819.628772] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
[ 4819.628777] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
[ 4819.628799] kobject: 'gpio' (00000000d962e90a): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.628804] kobject: 'gpiochip555' (00000000f5bbd198): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.628845] kobject: 'gpiochip555' (00000000f5bbd198): kobject_uevent_env
[ 4819.628850] kobject: 'gpiochip555' (00000000f5bbd198): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip555'
[ 4819.628893] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
[ 4819.628899] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.646727] kobject: 'gpiochip555' (00000000f5bbd198): kobject_uevent_env
[ 4819.646740] kobject: 'gpiochip555' (00000000f5bbd198): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip555'
[ 4819.646764] kobject: 'gpio' (00000000d962e90a): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.646768] kobject: 'gpiochip555' (00000000f5bbd198): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.646880] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
[ 4819.646889] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
[ 4819.646912] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
[ 4819.646918] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
[ 4819.646962] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
[ 4819.646971] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.646991] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
[ 4819.646997] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.647009] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.654427] kobject: 'gpio-sim.0' (000000005418b2af): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.654482] kobject: 'gpio-sim.0' (000000005418b2af): kobject_uevent_env
[ 4819.654488] kobject: 'gpio-sim.0' (000000005418b2af): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.654536] gpio-sim: probe of gpio-sim.0 failed with error -34
[ 4819.654556] kobject: 'gpio-sim.0' (000000005418b2af): kobject_uevent_env
[ 4819.654562] kobject: 'gpio-sim.0' (000000005418b2af): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.654576] kobject: 'gpio-sim.0' (000000005418b2af): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.717310] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.717365] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
[ 4819.717371] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.717473] gpio-560 (?): hogged as input
[ 4819.717479] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.717621] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
[ 4819.717630] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
[ 4819.717669] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
[ 4819.717676] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
[ 4819.717698] kobject: 'gpio' (000000006cce49d7): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.717703] kobject: 'gpiochip556' (00000000d2b04f32): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.717753] kobject: 'gpiochip556' (00000000d2b04f32): kobject_uevent_env
[ 4819.717760] kobject: 'gpiochip556' (00000000d2b04f32): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip556'
[ 4819.717948] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
[ 4819.717954] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.740722] kobject: 'gpiochip556' (00000000d2b04f32): kobject_uevent_env
[ 4819.740733] kobject: 'gpiochip556' (00000000d2b04f32): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip556'
[ 4819.740753] kobject: 'gpio' (000000006cce49d7): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.740757] kobject: 'gpiochip556' (00000000d2b04f32): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.740863] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
[ 4819.740871] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
[ 4819.740890] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
[ 4819.740896] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
[ 4819.740928] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
[ 4819.740936] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.740951] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
[ 4819.740958] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.740970] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.747033] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.747082] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
[ 4819.747087] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.747182] kobject: 'gpiochip14' (0000000092a079b6): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.747311] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
[ 4819.747321] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
[ 4819.747354] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
[ 4819.747359] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
[ 4819.747381] kobject: 'gpio' (000000007164bff6): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.747386] kobject: 'gpiochip564' (00000000ee87da7b): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.747426] kobject: 'gpiochip564' (00000000ee87da7b): kobject_uevent_env
[ 4819.747431] kobject: 'gpiochip564' (00000000ee87da7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip564'
[ 4819.747622] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
[ 4819.747629] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.782537] kobject: 'gpiochip564' (00000000ee87da7b): kobject_uevent_env
[ 4819.782547] kobject: 'gpiochip564' (00000000ee87da7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip564'
[ 4819.782567] kobject: 'gpio' (000000007164bff6): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.782571] kobject: 'gpiochip564' (00000000ee87da7b): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.782653] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
[ 4819.782662] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
[ 4819.782680] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
[ 4819.782685] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
[ 4819.782719] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
[ 4819.782729] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.782745] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
[ 4819.782750] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.782760] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.789540] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.789591] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
[ 4819.789597] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.789698] kobject: 'gpiochip15' (00000000b0535d8d): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.789829] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
[ 4819.789839] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
[ 4819.789873] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
[ 4819.789880] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
[ 4819.789902] kobject: 'gpio' (0000000012ac3b4a): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.789907] kobject: 'gpiochip572' (00000000178509ff): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.789957] kobject: 'gpiochip572' (00000000178509ff): kobject_uevent_env
[ 4819.789963] kobject: 'gpiochip572' (00000000178509ff): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip572'
[ 4819.790172] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
[ 4819.790179] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.820474] kobject: 'gpiochip572' (00000000178509ff): kobject_uevent_env
[ 4819.820489] kobject: 'gpiochip572' (00000000178509ff): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip572'
[ 4819.820515] kobject: 'gpio' (0000000012ac3b4a): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.820519] kobject: 'gpiochip572' (00000000178509ff): kobject_release, parent 0000000000000000 (delayed 750)
[ 4819.820629] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
[ 4819.820640] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
[ 4819.820664] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
[ 4819.820669] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
[ 4819.820715] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
[ 4819.820724] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.820745] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
[ 4819.820751] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.820761] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.827333] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.827384] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
[ 4819.827390] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.827489] kobject: 'gpiochip16' (000000002be607c1): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.827596] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
[ 4819.827606] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
[ 4819.827647] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
[ 4819.827652] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
[ 4819.827675] kobject: 'gpio' (000000000333f422): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.827679] kobject: 'gpiochip580' (00000000f6371a1e): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.827719] kobject: 'gpiochip580' (00000000f6371a1e): kobject_uevent_env
[ 4819.827724] kobject: 'gpiochip580' (00000000f6371a1e): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip580'
[ 4819.827920] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
[ 4819.827927] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.849122] kobject: 'gpiochip580' (00000000f6371a1e): kobject_uevent_env
[ 4819.849136] kobject: 'gpiochip580' (00000000f6371a1e): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip580'
[ 4819.849160] kobject: 'gpio' (000000000333f422): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.849163] kobject: 'gpiochip580' (00000000f6371a1e): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.849279] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
[ 4819.849293] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
[ 4819.849317] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
[ 4819.849323] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
[ 4819.849381] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
[ 4819.849391] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.849421] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
[ 4819.849428] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.849442] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.856384] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.856438] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
[ 4819.856445] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.856563] kobject: 'gpiochip17' (000000006ee06561): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.856712] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
[ 4819.856724] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
[ 4819.856786] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
[ 4819.856794] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
[ 4819.856821] kobject: 'gpio' (0000000005a238f5): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.856825] kobject: 'gpiochip588' (0000000075b76b85): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.856868] kobject: 'gpiochip588' (0000000075b76b85): kobject_uevent_env
[ 4819.856874] kobject: 'gpiochip588' (0000000075b76b85): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip588'
[ 4819.856922] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
[ 4819.856928] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.878720] kobject: 'gpiochip588' (0000000075b76b85): kobject_uevent_env
[ 4819.878731] kobject: 'gpiochip588' (0000000075b76b85): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip588'
[ 4819.878750] kobject: 'gpio' (0000000005a238f5): kobject_release, parent 0000000000000000 (delayed 500)
[ 4819.878755] kobject: 'gpiochip588' (0000000075b76b85): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4819.878864] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
[ 4819.878872] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
[ 4819.878890] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
[ 4819.878896] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
[ 4819.878928] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
[ 4819.878936] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.878950] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
[ 4819.878954] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.878964] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_release, parent 0000000000000000 (delayed 250)
[ 4819.885904] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4819.885967] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
[ 4819.885973] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4819.886080] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4819.886214] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
[ 4819.886224] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
[ 4819.886257] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
[ 4819.886264] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
[ 4819.886289] kobject: 'gpio' (0000000064e73453): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4819.886293] kobject: 'gpiochip589' (000000007e1cf13a): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4819.886340] kobject: 'gpiochip589' (000000007e1cf13a): kobject_uevent_env
[ 4819.886345] kobject: 'gpiochip589' (000000007e1cf13a): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip589'
[ 4819.886554] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
[ 4819.886560] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.013863] kobject: 'gpiochip589' (000000007e1cf13a): kobject_uevent_env
[ 4820.013872] kobject: 'gpiochip589' (000000007e1cf13a): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip589'
[ 4820.013894] kobject: 'gpio' (0000000064e73453): kobject_release, parent 0000000000000000 (delayed 750)
[ 4820.013897] kobject: 'gpiochip589' (000000007e1cf13a): kobject_release, parent 0000000000000000 (delayed 500)
[ 4820.013994] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
[ 4820.014004] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
[ 4820.014022] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
[ 4820.014027] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
[ 4820.014059] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
[ 4820.014065] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.014082] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
[ 4820.014087] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.014098] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_release, parent 0000000000000000 (delayed 500)
[ 4820.020299] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_add_internal: parent: 'platform', set: 'devices'
[ 4820.020347] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
[ 4820.020352] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.020447] kobject: 'gpiochip19' (000000008385ae47): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
[ 4820.020581] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
[ 4820.020590] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
[ 4820.020622] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
[ 4820.020627] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
[ 4820.020651] kobject: 'gpio' (000000001bd064f6): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
[ 4820.020655] kobject: 'gpiochip597' (00000000a32f70da): kobject_add_internal: parent: 'gpio', set: 'devices'
[ 4820.020701] kobject: 'gpiochip597' (00000000a32f70da): kobject_uevent_env
[ 4820.020707] kobject: 'gpiochip597' (00000000a32f70da): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip597'
[ 4820.020890] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
[ 4820.020896] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.048138] kobject: 'gpiochip597' (00000000a32f70da): kobject_uevent_env
[ 4820.048148] kobject: 'gpiochip597' (00000000a32f70da): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip597'
[ 4820.048175] kobject: 'gpio' (000000001bd064f6): kobject_release, parent 0000000000000000 (delayed 750)
[ 4820.048179] kobject: 'gpiochip597' (00000000a32f70da): kobject_release, parent 0000000000000000 (delayed 500)
[ 4820.048273] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
[ 4820.048282] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
[ 4820.048307] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
[ 4820.048313] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
[ 4820.048356] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
[ 4820.048364] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.048385] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
[ 4820.048391] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
[ 4820.048407] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_release, parent 0000000000000000 (delayed 250)
[ 4820.304376] kobject: 'gpiochip512' (000000008b9638a5): kobject_cleanup, parent 0000000000000000
[ 4820.304378] kobject: 'gpiochip512' (000000008b9638a5): calling ktype release
[ 4820.304384] kobject: 'gpiochip512': free name
[ 4820.304387] kobject: 'gpio' (0000000065b01c31): kobject_cleanup, parent 0000000000000000
[ 4820.304388] kobject: 'gpio' (0000000065b01c31): calling ktype release
[ 4820.304391] kobject: 'gpio': free name
[ 4820.400182] kobject: 'gpiochip3' (00000000f89849bd): kobject_release, parent 0000000000000000 (delayed 500)
[ 4820.436325] kobject: 'gpio' (00000000babdf5db): kobject_cleanup, parent 0000000000000000
[ 4820.436331] kobject: 'gpio' (00000000babdf5db): calling ktype release
[ 4820.436335] kobject: 'gpio': free name
[ 4820.436344] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_cleanup, parent 0000000000000000
[ 4820.436348] kobject: 'gpio-sim.0' (0000000034300a6b): calling ktype release
[ 4820.436355] kobject: 'gpio-sim.0': free name
[ 4820.436357] kobject: 'gpio' (000000008b98f863): kobject_cleanup, parent 0000000000000000
[ 4820.436359] kobject: 'gpio' (000000008b98f863): calling ktype release
[ 4820.436363] kobject: 'gpio': free name
[ 4820.496304] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_cleanup, parent 0000000000000000
[ 4820.496306] kobject: 'gpio-sim.0' (00000000cdc367f1): calling ktype release
[ 4820.496312] kobject: 'gpio-sim.0': free name
[ 4820.528235] kobject: 'gpiochip549' (0000000014379c6d): kobject_cleanup, parent 0000000000000000
[ 4820.528242] kobject: 'gpiochip549' (0000000014379c6d): calling ktype release
[ 4820.528248] kobject: 'gpiochip549': free name
[ 4820.596327] kobject: 'gpiochip551' (00000000a55e630b): kobject_cleanup, parent 0000000000000000
[ 4820.596333] kobject: 'gpiochip551' (00000000a55e630b): calling ktype release
[ 4820.596341] kobject: 'gpiochip551': free name
[ 4820.624451] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4820.624455] kobject: 'gpio' (000000002835f4a4): kobject_cleanup, parent 0000000000000000
[ 4820.624456] kobject: 'gpio' (000000002835f4a4): calling ktype release
[ 4820.624461] kobject: 'gpio': free name
[ 4820.656309] kobject: 'gpiochip12' (0000000051b24fd2): kobject_release, parent 0000000000000000 (delayed 250)
[ 4820.656313] kobject: 'gpiochip555' (00000000f5bbd198): kobject_cleanup, parent 0000000000000000
[ 4820.656315] kobject: 'gpiochip555' (00000000f5bbd198): calling ktype release
[ 4820.656321] kobject: 'gpiochip555': free name
[ 4820.656324] kobject: 'gpio' (00000000d962e90a): kobject_cleanup, parent 0000000000000000
[ 4820.656326] kobject: 'gpio' (00000000d962e90a): calling ktype release
[ 4820.656329] kobject: 'gpio': free name
[ 4820.752233] kobject: 'gpiochip556' (00000000d2b04f32): kobject_cleanup, parent 0000000000000000
[ 4820.752240] kobject: 'gpiochip556' (00000000d2b04f32): calling ktype release
[ 4820.752246] kobject: 'gpiochip556': free name
[ 4820.788358] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_cleanup, parent 0000000000000000
[ 4820.788360] kobject: 'gpio-sim.0' (000000006053a4bf): calling ktype release
[ 4820.788365] kobject: 'gpio-sim.0': free name
[ 4820.788367] kobject: 'gpio' (000000007164bff6): kobject_cleanup, parent 0000000000000000
[ 4820.788369] kobject: 'gpio' (000000007164bff6): calling ktype release
[ 4820.788373] kobject: 'gpio': free name
[ 4820.848443] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_cleanup, parent 0000000000000000
[ 4820.848449] kobject: 'gpio-sim.0' (00000000c56fc456): calling ktype release
[ 4820.848457] kobject: 'gpio-sim.0': free name
[ 4820.852177] kobject: 'gpio' (0000000012ac3b4a): kobject_cleanup, parent 0000000000000000
[ 4820.852183] kobject: 'gpio' (0000000012ac3b4a): calling ktype release
[ 4820.852187] kobject: 'gpio': free name
[ 4820.880305] kobject: 'gpio' (000000000333f422): kobject_cleanup, parent 0000000000000000
[ 4820.880326] kobject: 'gpio' (000000000333f422): calling ktype release
[ 4820.880332] kobject: 'gpio': free name
[ 4820.884286] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_cleanup, parent 0000000000000000
[ 4820.884288] kobject: 'gpio-sim.0' (0000000048f1330d): calling ktype release
[ 4820.884293] kobject: 'gpio-sim.0': free name
[ 4820.884296] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_cleanup, parent 0000000000000000
[ 4820.884298] kobject: 'gpio-sim.0' (000000000471ebbe): calling ktype release
[ 4820.884304] kobject: 'gpio-sim.0': free name
[ 4821.076363] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_cleanup, parent 0000000000000000
[ 4821.076365] kobject: 'gpio-sim.0' (00000000b1d6276b): calling ktype release
[ 4821.076370] kobject: 'gpio-sim.0': free name
[ 4821.328246] kobject: 'gpio' (000000007e927b76): kobject_cleanup, parent 0000000000000000
[ 4821.328252] kobject: 'gpio' (000000007e927b76): calling ktype release
[ 4821.328258] kobject: 'gpio': free name
[ 4821.360465] kobject: 'gpiochip2' (0000000034808fa5): kobject_release, parent 0000000000000000 (delayed 250)
[ 4821.360468] kobject: 'gpio' (000000008afdcce8): kobject_cleanup, parent 0000000000000000
[ 4821.360470] kobject: 'gpio' (000000008afdcce8): calling ktype release
[ 4821.360475] kobject: 'gpio': free name
[ 4821.392179] kobject: 'gpio-sim.0' (00000000b204b566): kobject_cleanup, parent 0000000000000000
[ 4821.392186] kobject: 'gpio-sim.0' (00000000b204b566): calling ktype release
[ 4821.392193] kobject: 'gpio-sim.0': free name
[ 4821.392196] kobject: 'gpiochip515' (000000002b851bb5): kobject_cleanup, parent 0000000000000000
[ 4821.392221] kobject: 'gpiochip515' (000000002b851bb5): calling ktype release
[ 4821.392226] kobject: 'gpiochip515': free name
[ 4821.428255] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_cleanup, parent 0000000000000000
[ 4821.428262] kobject: 'gpio-sim.0' (00000000b9ef5a28): calling ktype release
[ 4821.428283] kobject: 'gpio-sim.0': free name
[ 4821.428291] kobject: 'gpiochip4' (00000000e06680f5): kobject_release, parent 0000000000000000 (delayed 250)
[ 4821.460377] kobject: 'gpiochip5' (000000004ef2dca6): kobject_release, parent 0000000000000000 (delayed 750)
[ 4821.460380] kobject: 'gpiochip532' (00000000ac07803f): kobject_cleanup, parent 0000000000000000
[ 4821.460382] kobject: 'gpiochip532' (00000000ac07803f): calling ktype release
[ 4821.460386] kobject: 'gpiochip532': free name
[ 4821.488407] kobject: 'gpio' (00000000dedc22f2): kobject_cleanup, parent 0000000000000000
[ 4821.488426] kobject: 'gpio' (00000000dedc22f2): calling ktype release
[ 4821.488432] kobject: 'gpio': free name
[ 4821.524217] kobject: 'gpiochip7' (000000006b8d9c99): kobject_release, parent 0000000000000000 (delayed 250)
[ 4821.588303] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4821.588372] kobject: 'gpiochip8' (00000000cb0b3463): kobject_release, parent 0000000000000000 (delayed 250)
[ 4821.588375] kobject: 'gpio' (0000000016f05b5a): kobject_cleanup, parent 0000000000000000
[ 4821.588377] kobject: 'gpio' (0000000016f05b5a): calling ktype release
[ 4821.588380] kobject: 'gpio': free name
[ 4821.616206] kobject: 'gpiochip553' (0000000010cfe833): kobject_cleanup, parent 0000000000000000
[ 4821.616213] kobject: 'gpiochip553' (0000000010cfe833): calling ktype release
[ 4821.616224] kobject: 'gpiochip553': free name
[ 4821.616228] kobject: 'gpio' (00000000886e1a77): kobject_cleanup, parent 0000000000000000
[ 4821.616230] kobject: 'gpio' (00000000886e1a77): calling ktype release
[ 4821.616234] kobject: 'gpio': free name
[ 4821.652317] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_cleanup, parent 0000000000000000
[ 4821.652323] kobject: 'gpio-sim.0' (000000008ef7f514): calling ktype release
[ 4821.652348] kobject: 'gpio-sim.0': free name
[ 4821.684240] kobject: 'gpiochip12' (0000000051b24fd2): kobject_cleanup, parent 0000000000000000
[ 4821.684246] kobject: 'gpiochip12' (0000000051b24fd2): calling ktype release
[ 4821.684275] kobject: 'gpiochip12': free name
[ 4821.748311] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_cleanup, parent 0000000000000000
[ 4821.748317] kobject: 'gpio-sim.0' (00000000cd797c7b): calling ktype release
[ 4821.748325] kobject: 'gpio-sim.0': free name
[ 4821.748328] kobject: 'gpio' (000000006cce49d7): kobject_cleanup, parent 0000000000000000
[ 4821.748330] kobject: 'gpio' (000000006cce49d7): calling ktype release
[ 4821.748333] kobject: 'gpio': free name
[ 4821.812244] kobject: 'gpiochip14' (0000000092a079b6): kobject_release, parent 0000000000000000 (delayed 1000)
[ 4821.840208] kobject: 'gpiochip15' (00000000b0535d8d): kobject_release, parent 0000000000000000 (delayed 750)
[ 4821.872262] kobject: 'gpiochip580' (00000000f6371a1e): kobject_cleanup, parent 0000000000000000
[ 4821.872269] kobject: 'gpiochip580' (00000000f6371a1e): calling ktype release
[ 4821.872277] kobject: 'gpiochip580': free name
[ 4821.904224] kobject: 'gpio' (0000000005a238f5): kobject_cleanup, parent 0000000000000000
[ 4821.904230] kobject: 'gpio' (0000000005a238f5): calling ktype release
[ 4821.904235] kobject: 'gpio': free name
[ 4822.036185] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_cleanup, parent 0000000000000000
[ 4822.036192] kobject: 'gpio-sim.0' (00000000f462f78a): calling ktype release
[ 4822.036198] kobject: 'gpio-sim.0': free name
[ 4822.036202] kobject: 'gpiochip589' (000000007e1cf13a): kobject_cleanup, parent 0000000000000000
[ 4822.036204] kobject: 'gpiochip589' (000000007e1cf13a): calling ktype release
[ 4822.036206] kobject: 'gpiochip589': free name
[ 4822.068252] kobject: 'gpiochip597' (00000000a32f70da): kobject_cleanup, parent 0000000000000000
[ 4822.068258] kobject: 'gpiochip597' (00000000a32f70da): calling ktype release
[ 4822.068265] kobject: 'gpiochip597': free name
[ 4822.384245] kobject: 'gpiochip1' (00000000c6873be9): kobject_release, parent 0000000000000000 (delayed 500)
[ 4822.384248] kobject: 'gpiochip513' (00000000307a0461): kobject_cleanup, parent 0000000000000000
[ 4822.384250] kobject: 'gpiochip513' (00000000307a0461): calling ktype release
[ 4822.384271] kobject: 'gpiochip513': free name
[ 4822.384286] kobject: 'gpio-sim.0' (00000000401b009a): kobject_cleanup, parent 0000000000000000
[ 4822.384288] kobject: 'gpio-sim.0' (00000000401b009a): calling ktype release
[ 4822.384294] kobject: 'gpio-sim.0': free name
[ 4822.384297] kobject: 'gpiochip514' (000000008e860d8b): kobject_cleanup, parent 0000000000000000
[ 4822.384298] kobject: 'gpiochip514' (000000008e860d8b): calling ktype release
[ 4822.384302] kobject: 'gpiochip514': free name
[ 4822.384304] kobject: 'gpiochip2' (0000000034808fa5): kobject_cleanup, parent 0000000000000000
[ 4822.384305] kobject: 'gpiochip2' (0000000034808fa5): calling ktype release
[ 4822.384312] kobject: 'gpiochip2': free name
[ 4822.416325] kobject: 'gpiochip3' (00000000f89849bd): kobject_cleanup, parent 0000000000000000
[ 4822.416331] kobject: 'gpiochip3' (00000000f89849bd): calling ktype release
[ 4822.416344] kobject: 'gpiochip3': free name
[ 4822.452250] kobject: 'gpiochip4' (00000000e06680f5): kobject_cleanup, parent 0000000000000000
[ 4822.452256] kobject: 'gpiochip4' (00000000e06680f5): calling ktype release
[ 4822.452265] kobject: 'gpiochip4': free name
[ 4822.548316] kobject: 'gpiochip7' (000000006b8d9c99): kobject_cleanup, parent 0000000000000000
[ 4822.548322] kobject: 'gpiochip7' (000000006b8d9c99): calling ktype release
[ 4822.548350] kobject: 'gpiochip7': free name
[ 4822.608239] kobject: 'gpiochip8' (00000000cb0b3463): kobject_cleanup, parent 0000000000000000
[ 4822.608260] kobject: 'gpiochip8' (00000000cb0b3463): calling ktype release
[ 4822.608275] kobject: 'gpiochip8': free name
[ 4822.640315] kobject: 'gpiochip554' (00000000e0038afb): kobject_cleanup, parent 0000000000000000
[ 4822.640339] kobject: 'gpiochip554' (00000000e0038afb): calling ktype release
[ 4822.640343] kobject: 'gpio' (00000000e467cb12): kobject_cleanup, parent 0000000000000000
[ 4822.640346] kobject: 'gpiochip554': free name
[ 4822.640348] kobject: 'gpio' (00000000e467cb12): calling ktype release
[ 4822.640349] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_cleanup, parent 0000000000000000
[ 4822.640351] kobject: 'gpio-sim.1' (0000000018f833bf): calling ktype release
[ 4822.640354] kobject: 'gpio': free name
[ 4822.640360] kobject: 'gpio-sim.1': free name
[ 4822.640408] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_cleanup, parent 0000000000000000
[ 4822.640410] kobject: 'gpio-sim.2' (00000000bbfd5238): calling ktype release
[ 4822.640416] kobject: 'gpio-sim.2': free name
[ 4822.644186] kobject: 'gpio-sim.0' (00000000461517ae): kobject_cleanup, parent 0000000000000000
[ 4822.644192] kobject: 'gpio-sim.0' (00000000461517ae): calling ktype release
[ 4822.644198] kobject: 'gpio-sim.0': free name
[ 4822.644266] kobject: 'gpio-sim.0' (00000000bb358104): kobject_cleanup, parent 0000000000000000
[ 4822.644268] kobject: 'gpio-sim.0' (00000000bb358104): calling ktype release
[ 4822.644272] kobject: 'gpio-sim.0': free name
[ 4822.896320] kobject: 'gpiochip572' (00000000178509ff): kobject_cleanup, parent 0000000000000000
[ 4822.896326] kobject: 'gpiochip572' (00000000178509ff): calling ktype release
[ 4822.896334] kobject: 'gpiochip572': free name
[ 4822.896337] kobject: 'gpio-sim.0' (000000005418b2af): kobject_cleanup, parent 0000000000000000
[ 4822.896339] kobject: 'gpio-sim.0' (000000005418b2af): calling ktype release
[ 4822.896343] kobject: 'gpio-sim.0': free name
[ 4822.900449] kobject: 'gpiochip564' (00000000ee87da7b): kobject_cleanup, parent 0000000000000000
[ 4822.900455] kobject: 'gpiochip564' (00000000ee87da7b): calling ktype release
[ 4822.900461] kobject: 'gpiochip17' (000000006ee06561): kobject_release, parent 0000000000000000 (delayed 750)
[ 4822.900463] kobject: 'gpiochip564': free name
[ 4822.900524] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_release, parent 0000000000000000 (delayed 250)
[ 4823.152251] kobject: 'gpio' (0000000064e73453): kobject_cleanup, parent 0000000000000000
[ 4823.152257] kobject: 'gpio' (0000000064e73453): calling ktype release
[ 4823.152260] kobject: 'gpio': free name
[ 4823.156258] kobject: 'gpio' (000000001bd064f6): kobject_cleanup, parent 0000000000000000
[ 4823.156277] kobject: 'gpio' (000000001bd064f6): calling ktype release
[ 4823.156307] kobject: 'gpio': free name
[ 4823.408426] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_cleanup, parent 0000000000000000
[ 4823.408433] kobject: 'gpio-sim.0' (000000007cf5eda9): calling ktype release
[ 4823.408433] kobject: 'gpiochip531' (00000000ad4538e8): kobject_cleanup, parent 0000000000000000
[ 4823.408436] kobject: 'gpiochip531' (00000000ad4538e8): calling ktype release
[ 4823.408444] kobject: 'gpio-sim.0': free name
[ 4823.408452] kobject: 'gpiochip531': free name
[ 4823.408456] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_cleanup, parent 0000000000000000
[ 4823.408459] kobject: 'gpio-sim.0' (00000000ca4e28ab): calling ktype release
[ 4823.408467] kobject: 'gpio-sim.0': free name
[ 4823.408476] kobject: 'gpiochip0' (000000007063b679): kobject_release, parent 0000000000000000 (delayed 250)
[ 4823.408523] kobject: 'gpio' (00000000a1399cd0): kobject_cleanup, parent 0000000000000000
[ 4823.408525] kobject: 'gpio' (00000000a1399cd0): calling ktype release
[ 4823.408533] kobject: 'gpio': free name
[ 4823.664421] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_cleanup, parent 0000000000000000
[ 4823.664426] kobject: 'gpio-sim.0' (00000000e8c2899c): calling ktype release
[ 4823.664450] kobject: 'gpio-sim.0': free name
[ 4823.664516] kobject: 'gpiochip10' (00000000e3becc55): kobject_release, parent 0000000000000000 (delayed 500)
[ 4823.664524] kobject: 'gpiochip552' (000000001750e50f): kobject_cleanup, parent 0000000000000000
[ 4823.664526] kobject: 'gpiochip552' (000000001750e50f): calling ktype release
[ 4823.664530] kobject: 'gpiochip552': free name
[ 4823.664533] kobject: 'gpio' (00000000176df109): kobject_cleanup, parent 0000000000000000
[ 4823.664535] kobject: 'gpio' (00000000176df109): calling ktype release
[ 4823.664538] kobject: 'gpio': free name
[ 4823.664556] kobject: 'gpiochip6' (00000000a22546a6): kobject_release, parent 0000000000000000 (delayed 250)
[ 4823.664558] kobject: 'gpiochip533' (000000003541481f): kobject_cleanup, parent 0000000000000000
[ 4823.664560] kobject: 'gpiochip533' (000000003541481f): calling ktype release
[ 4823.664564] kobject: 'gpiochip533': free name
[ 4823.924314] kobject: 'gpiochip588' (0000000075b76b85): kobject_cleanup, parent 0000000000000000
[ 4823.924334] kobject: 'gpiochip588' (0000000075b76b85): calling ktype release
[ 4823.924340] kobject: 'gpiochip588': free name
[ 4823.924344] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_cleanup, parent 0000000000000000
[ 4823.924346] kobject: 'gpiochip13' (00000000dc27f1aa): calling ktype release
[ 4823.924354] kobject: 'gpiochip13': free name
[ 4823.924371] kobject: 'gpiochip16' (000000002be607c1): kobject_release, parent 0000000000000000 (delayed 250)
[ 4824.176336] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_release, parent 0000000000000000 (delayed 250)
[ 4824.176381] kobject: 'gpiochip19' (000000008385ae47): kobject_release, parent 0000000000000000 (delayed 250)
[ 4824.400258] kobject: 'gpiochip1' (00000000c6873be9): kobject_cleanup, parent 0000000000000000
[ 4824.400264] kobject: 'gpiochip1' (00000000c6873be9): calling ktype release
[ 4824.400276] kobject: 'gpiochip1': free name
[ 4824.432259] kobject: 'gpiochip0' (000000007063b679): kobject_cleanup, parent 0000000000000000
[ 4824.432266] kobject: 'gpiochip0' (000000007063b679): calling ktype release
[ 4824.432278] kobject: 'gpiochip0': free name
[ 4824.688241] kobject: 'gpiochip5' (000000004ef2dca6): kobject_cleanup, parent 0000000000000000
[ 4824.688248] kobject: 'gpiochip5' (000000004ef2dca6): calling ktype release
[ 4824.688261] kobject: 'gpiochip5': free name
[ 4824.692227] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_cleanup, parent 0000000000000000
[ 4824.692246] kobject: 'gpiochip11' (00000000b39a3c5f): calling ktype release
[ 4824.692278] kobject: 'gpiochip11': free name
[ 4824.692290] kobject: 'gpiochip6' (00000000a22546a6): kobject_cleanup, parent 0000000000000000
[ 4824.692317] kobject: 'gpiochip6' (00000000a22546a6): calling ktype release
[ 4824.692338] kobject: 'gpiochip6': free name
[ 4824.948242] kobject: 'gpiochip15' (00000000b0535d8d): kobject_cleanup, parent 0000000000000000
[ 4824.948247] kobject: 'gpiochip15' (00000000b0535d8d): calling ktype release
[ 4824.948251] kobject: 'gpiochip16' (000000002be607c1): kobject_cleanup, parent 0000000000000000
[ 4824.948255] kobject: 'gpiochip16' (000000002be607c1): calling ktype release
[ 4824.948259] kobject: 'gpiochip15': free name
[ 4824.948272] kobject: 'gpiochip16': free name
[ 4825.200324] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_cleanup, parent 0000000000000000
[ 4825.200330] kobject: 'gpiochip18' (00000000a8ca1f8c): calling ktype release
[ 4825.200343] kobject: 'gpiochip18': free name
[ 4825.204318] kobject: 'gpiochip19' (000000008385ae47): kobject_cleanup, parent 0000000000000000
[ 4825.204325] kobject: 'gpiochip19' (000000008385ae47): calling ktype release
[ 4825.204340] kobject: 'gpiochip19': free name
[ 4825.680450] kobject: 'gpiochip10' (00000000e3becc55): kobject_cleanup, parent 0000000000000000
[ 4825.680470] kobject: 'gpiochip10' (00000000e3becc55): calling ktype release
[ 4825.680488] kobject: 'gpiochip10': free name
[ 4825.716394] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_cleanup, parent 0000000000000000
[ 4825.716396] kobject: 'gpiochip9' (00000000bdaa0a47): calling ktype release
[ 4825.716405] kobject: 'gpiochip9': free name
[ 4825.968324] kobject: 'gpiochip17' (000000006ee06561): kobject_cleanup, parent 0000000000000000
[ 4825.968331] kobject: 'gpiochip17' (000000006ee06561): calling ktype release
[ 4825.968360] kobject: 'gpiochip17': free name
[ 4825.968363] kobject: 'gpiochip14' (0000000092a079b6): kobject_cleanup, parent 0000000000000000
[ 4825.968367] kobject: 'gpiochip14' (0000000092a079b6): calling ktype release
[ 4825.968377] kobject: 'gpiochip14': free name
[ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
[ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
[ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
[ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
[ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
[ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
[ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
[ 4833.908315] kobject: 'gpio-sim': free name
[ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
[ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
[ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
[ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
[ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
[ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
[ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
[ 4835.952448] kobject: 'gpio_sim': free name
>>>>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>>>>> device, but then it's unclear how to provide the attributes in non-racy way
>>>> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>>>>
>>>> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
>>>> device expert nor would be appropriate to try the craft not earned ;-)
With all of these additional debugging, cat /sys/kernel/debug/kmemleak
showed nothing new.
I believe this is reasonably safe.
However, I was unsuccessful in seeing gpio trace, even with
echo 1 > /sys/kernel/tracing/events/gpio/enable ... :-/
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-27 18:38 ` Mirsad Todorovac
@ 2023-02-27 23:13 ` Andy Shevchenko
2023-03-08 13:11 ` Mirsad Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Andy Shevchenko @ 2023-02-27 23:13 UTC (permalink / raw)
To: Mirsad Todorovac
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On Mon, Feb 27, 2023 at 07:38:01PM +0100, Mirsad Todorovac wrote:
> On 2/24/23 18:40, Andy Shevchenko wrote:
> > On Fri, Feb 24, 2023 at 04:12:31PM +0100, Mirsad Todorovac wrote:
> > > On 2/21/23 16:39, Andy Shevchenko wrote:
> > > > On Tue, Feb 21, 2023 at 02:52:38PM +0100, Mirsad Goran Todorovac wrote:
> > > > > On 20. 02. 2023. 14:43, Andy Shevchenko wrote:
> > > > > > On Mon, Feb 20, 2023 at 02:10:00PM +0100, Mirsad Todorovac wrote:
> > > > > > > On 2/16/23 15:16, Bartosz Golaszewski wrote:
...
> > > > > > > As Mr. McKenney once said, a bunch of monkeys with keyboard could
> > > > > > > have done it in a considerable number of trials and errors ;-)
> > > > > > >
> > > > > > > But here I have something that could potentially leak as well. I could not devise a
> > > > > > > reproducer due to the leak being lightly triggered only in extreme memory contention.
> > > > > > >
> > > > > > > See it for yourself:
> > > > > > >
> > > > > > > drivers/gpio/gpio-sim.c:
> > > > > > > 301 static int gpio_sim_setup_sysfs(struct gpio_sim_chip *chip)
> > > > > > > 302 {
> > > > > > > 303 struct device_attribute *val_dev_attr, *pull_dev_attr;
> > > > > > > 304 struct gpio_sim_attribute *val_attr, *pull_attr;
> > > > > > > 305 unsigned int num_lines = chip->gc.ngpio;
> > > > > > > 306 struct device *dev = chip->gc.parent;
> > > > > > > 307 struct attribute_group *attr_group;
> > > > > > > 308 struct attribute **attrs;
> > > > > > > 309 int i, ret;
> > > > > > > 310
> > > > > > > 311 chip->attr_groups = devm_kcalloc(dev, sizeof(*chip->attr_groups),
> > > > > > > 312 num_lines + 1, GFP_KERNEL);
> > > > > > > 313 if (!chip->attr_groups)
> > > > > > > 314 return -ENOMEM;
> > > > > > > 315
> > > > > > > 316 for (i = 0; i < num_lines; i++) {
> > > > > > > 317 attr_group = devm_kzalloc(dev, sizeof(*attr_group), GFP_KERNEL);
> > > > > > > 318 attrs = devm_kcalloc(dev, GPIO_SIM_NUM_ATTRS, sizeof(*attrs),
> > > > > > > 319 GFP_KERNEL);
> > > > > > > 320 val_attr = devm_kzalloc(dev, sizeof(*val_attr), GFP_KERNEL);
> > > > > > > 321 pull_attr = devm_kzalloc(dev, sizeof(*pull_attr), GFP_KERNEL);
> > > > > > > 322 if (!attr_group || !attrs || !val_attr || !pull_attr)
> > > > > > > 323 return -ENOMEM;
> > > > > > > 324
> > > > > > > 325 attr_group->name = devm_kasprintf(dev, GFP_KERNEL,
> > > > > > > 326 "sim_gpio%u", i);
> > > > > > > 327 if (!attr_group->name)
> > > > > > > 328 return -ENOMEM;
> > > > > > >
> > > > > > > Apparently, if the memory allocation only partially succeeds, in the theoretical case
> > > > > > > that the system is close to its kernel memory exhaustion, `return -ENOMEM` would not
> > > > > > > free the partially succeeded allocs, would it?
> > > > > > >
> > > > > > > To explain it better, I tried a version that is not yet full doing "all or nothing"
> > > > > > > memory allocation for the gpio-sim driver, because I am not that familiar with the
> > > > > > > driver internals.
> > > > > >
> > > > > > devm_*() mean that the resource allocation is made in a managed manner, so when
> > > > > > it's done, it will be freed automatically.
> > > > >
> > > > > Didn't see that one coming ... :-/ "buzzing though the bush ..."
> > > > >
> > > > > > The question is: is the lifetime of the attr_groups should be lesser or the
> > > > > > same as chip->gc.parent? Maybe it's incorrect to call devm_*() in the first place?
> > > > >
> > > > > Bona fide said, I hope that automatic deallocation does things in the right order.
> > > > > I've realised that devm_kzalloc() calls devm_kmalloc() that registers allocations on
> > > > > a per driver list. But I am not sure how chip->gc was allocated?
> > > > >
> > > > > Here is said it is allocated in drivers/gpio/gpio-sim.c:386 in gpio_sim_add_bank(),
> > > > > as a part of
> > > > >
> > > > > struct gpio_sim_chip *chip;
> > > > > struct gpio_chip *gc;
> > > > >
> > > > > gc = &chip->gc;
> > > > >
> > > > > and gc->parent is set to
> > > > >
> > > > > gc->parent = dev;
> > > > >
> > > > > in line 420, which appears called before gpio_sim_setup_sysfs() and the lines above.
> > > > >
> > > > > If I understood well, automatic deallocation on unloading the driver goes
> > > > > in the reverse order, so lifetime of chip appears to be longer than attr_groups,
> > > > > but I am really not that good at this ...
> > > >
> > > > So, the device is instantiated by platform_device_register_full().
> > > >
> > > > It should gone with the platform_device_unregister().
> > > >
> > > > In case of CONFIG_DEBUG_KOBJECT_RELEASE=y the ->release() can be called
> > > > asynchronously.
> > > >
> > > > So, there are following questions:
> > > > - is the put_device() is actually called?
> > > > - is the above mentioned option is set to Y?
> > > > - if it's in Y, does kmemleak take it into account?
> > > > - if no, do you get anything new in `dmesg` when enable it?
> > >
> > > Hi, Andy,
> > >
> > > Having set CONFIG_DEBUG_KOBJECT=y.
> > > CONFIG_DEBUG_KOBJECT_RELEASE=y and CONFIG_DEBUG_OBJECTS_TIMERS=y disappear after "make olddefconfig"
> > >
> > > So, I cannot tell about whether release() was called asynchronously, all I get is (after driver unload):
> > >
> > > [ 810.989742] kobject: 'gpio-sim' (00000000251afa19): kobject_cleanup, parent 00000000447da7a7
> > > [ 810.990216] kobject: 'gpio-sim' (00000000251afa19): auto cleanup kobject_del
> > > [ 810.990674] kobject: 'gpio-sim' (00000000251afa19): auto cleanup 'remove' event
> > > [ 810.991175] kobject: 'gpio-sim' (00000000251afa19): kobject_uevent_env
> > > [ 810.991674] kobject: 'gpio-sim' (00000000251afa19): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> > > [ 810.992154] kobject: 'gpio-sim' (00000000251afa19): calling ktype release
> > > [ 810.992644] kobject: 'gpio-sim': free name
> > >
> > > I am still trying to convince "make olddefconfig" to accept the above values he did not like :-/
> >
> > Btw, since devm_*() is in use, you may enable trace events (I have added them a
> > few releases back) and see how objects allocated with devm are getting
> > released.
> >
> > Before that there is a possibility to enable debug for devm.
> > CONFIG_DEBUG_DEVRES for that (you may see how it's used in the
> > drivers/base/devres.c).
>
> The loga are extensive if you are willing to skim over them, but I believe the interesting
> part is this:
I'm not sure I understand where the problem is.
> [ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
> [ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
> [ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
> [ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
> [ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
> [ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> [ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
> [ 4833.908315] kobject: 'gpio-sim': free name
> [ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
> [ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
> [ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
> [ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
> [ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
> [ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
> [ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
> [ 4835.952448] kobject: 'gpio_sim': free name
>
> Or, with CONFIG_DEBUG_DEVRES=y, it looks like this:
I don't see that been enabled (it requires to pass a command line option to the kernel).
> [ 180.472131] kobject: 'gpio-sim' (00000000e72df678): kobject_release, parent 0000000053a50b05 (delayed 250)
> [ 180.472270] kobject: 'drivers' (00000000a3c60031): kobject_release, parent 00000000363d6076 (delayed 250)
> [ 180.472274] kobject: 'holders' (000000006bd5c478): kobject_release, parent 00000000363d6076 (delayed 250)
> [ 180.472281] kobject: 'notes' (00000000c85ae91b): kobject_release, parent 00000000363d6076 (delayed 1000)
> [ 181.495687] kobject: 'holders' (000000006bd5c478): kobject_cleanup, parent 00000000363d6076
> [ 181.495693] kobject: 'holders' (000000006bd5c478): auto cleanup kobject_del
> [ 181.495696] kobject: 'holders' (000000006bd5c478): calling ktype release
> [ 181.495697] kobject: (000000006bd5c478): dynamic_kobj_release
> [ 181.495703] kobject: 'holders': free name
> [ 181.495717] kobject: 'drivers' (00000000a3c60031): kobject_cleanup, parent 00000000363d6076
> [ 181.495719] kobject: 'drivers' (00000000a3c60031): auto cleanup kobject_del
> [ 181.495723] kobject: 'drivers' (00000000a3c60031): calling ktype release
> [ 181.495724] kobject: (00000000a3c60031): dynamic_kobj_release
> [ 181.495727] kobject: 'drivers': free name
> [ 181.495728] kobject: 'gpio-sim' (00000000e72df678): kobject_cleanup, parent 0000000053a50b05
> [ 181.495730] kobject: 'gpio-sim' (00000000e72df678): auto cleanup kobject_del
> [ 181.495731] kobject: 'gpio-sim' (00000000e72df678): auto cleanup 'remove' event
> [ 181.495733] kobject: 'gpio-sim' (00000000e72df678): kobject_uevent_env
> [ 181.495742] kobject: 'gpio-sim' (00000000e72df678): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> [ 181.495764] kobject: 'gpio-sim' (00000000e72df678): calling ktype release
> [ 181.495768] kobject: 'gpio-sim': free name
> [ 184.663810] kobject: 'notes' (00000000c85ae91b): kobject_cleanup, parent 00000000363d6076
> [ 184.663816] kobject: 'notes' (00000000c85ae91b): auto cleanup kobject_del
> [ 184.663824] kobject: 'notes' (00000000c85ae91b): calling ktype release
> [ 184.663838] kobject: (00000000c85ae91b): dynamic_kobj_release
> [ 184.663841] kobject: 'notes': free name
> [ 184.663843] kobject: 'gpio_sim' (00000000363d6076): kobject_release, parent 00000000dfc73d50 (delayed 500)
> [ 186.679811] kobject: 'gpio_sim' (00000000363d6076): kobject_cleanup, parent 00000000dfc73d50
> [ 186.679817] kobject: 'gpio_sim' (00000000363d6076): auto cleanup kobject_del
> [ 186.679819] kobject: 'gpio_sim' (00000000363d6076): auto cleanup 'remove' event
> [ 186.679820] kobject: 'gpio_sim' (00000000363d6076): kobject_uevent_env
> [ 186.679832] kobject: 'gpio_sim' (00000000363d6076): fill_kobj_path: path = '/module/gpio_sim'
> [ 186.679855] kobject: 'gpio_sim' (00000000363d6076): calling ktype release
> [ 186.679859] kobject: 'gpio_sim': free name
>
> Here are the /var/log/messages and `dmesg | grep gpio` output:
>
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000065b01c31): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip512' (000000008b9638a5): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000e7ddc5ce): kobject_release, parent 000000007063b679 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000052bfa782): kobject_release, parent 0000000017e526f4 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000007e927b76): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip513' (00000000307a0461): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000007025cedf): kobject_release, parent 00000000c6873be9 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000002030b0fc): kobject_release, parent 00000000cd2bbef4 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000008afdcce8): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip514' (000000008e860d8b): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000dc5ac60d): kobject_release, parent 0000000034808fa5 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000401b009a): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000b0dfd44e): kobject_release, parent 000000001a955210 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000a1399cd0): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip515' (000000002b851bb5): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000276535c4): kobject_release, parent 00000000f89849bd (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000b204b566): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000581391d9): kobject_release, parent 000000002a3e415e (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000babdf5db): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip531' (00000000ad4538e8): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (0000000045377336): kobject_release, parent 00000000e06680f5 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000554e0e6d): kobject_release, parent 000000003fe8f71b (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000008b98f863): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip532' (00000000ac07803f): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (0000000007a45939): kobject_release, parent 000000004ef2dca6 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (0000000034300a6b): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000089fb2874): kobject_release, parent 00000000f81b3244 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000dedc22f2): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip533' (000000003541481f): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000002d7008b9): kobject_release, parent 00000000a22546a6 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000bbace83b): kobject_release, parent 0000000050fa69b3 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000e467cb12): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip549' (0000000014379c6d): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000186d3bd6): kobject_release, parent 000000006b8d9c99 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000bb358104): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000ab70f947): kobject_release, parent 000000000876b849 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000016f05b5a): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip551' (00000000a55e630b): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000948e7c59): kobject_release, parent 00000000cb0b3463 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000461517ae): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000b41576d9): kobject_release, parent 0000000060e34f22 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000176df109): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip552' (000000001750e50f): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000008026d597): kobject_release, parent 00000000bdaa0a47 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.1' (0000000018f833bf): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000005a55ef16): kobject_release, parent 000000008dc012bc (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000886e1a77): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip553' (0000000010cfe833): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000d8b3d64b): kobject_release, parent 00000000e3becc55 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000002e06f814): kobject_release, parent 00000000a555c1b2 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000002835f4a4): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip554' (00000000e0038afb): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000004e7dad05): kobject_release, parent 00000000b39a3c5f (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000bfef6047): kobject_release, parent 00000000b8559267 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (00000000d962e90a): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip555' (00000000f5bbd198): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000af8b42a8): kobject_release, parent 0000000051b24fd2 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000008ef7f514): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000041578d2d): kobject_release, parent 00000000b738a3e4 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: gpio-sim: probe of gpio-sim.0 failed with error -34
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000005418b2af): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000001f644365): kobject_release, parent 00000000515b4b1b (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: gpio-560 (?): hogged as input
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000006cce49d7): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip556' (00000000d2b04f32): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000843ea14e): kobject_release, parent 00000000dc27f1aa (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000030fbb9d7): kobject_release, parent 000000001576402e (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000007164bff6): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip564' (00000000ee87da7b): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000009eaeda39): kobject_release, parent 0000000092a079b6 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000006053a4bf): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000032fe601a): kobject_release, parent 0000000063e66fd0 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000012ac3b4a): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip572' (00000000178509ff): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (0000000018c19582): kobject_release, parent 00000000b0535d8d (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000c56fc456): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000095747aa3): kobject_release, parent 00000000c314f13a (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000000333f422): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip580' (00000000f6371a1e): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000006dc3da55): kobject_release, parent 000000002be607c1 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (000000000471ebbe): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (000000008cc134db): kobject_release, parent 000000008af54826 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000005a238f5): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip588' (0000000075b76b85): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (000000003f03127c): kobject_release, parent 000000006ee06561 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (0000000048f1330d): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (0000000023dec04e): kobject_release, parent 000000005fd3b6d2 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (0000000064e73453): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip589' (000000007e1cf13a): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000cdec14e3): kobject_release, parent 00000000a8ca1f8c (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000f462f78a): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000b5aa4b07): kobject_release, parent 00000000ea466ae7 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio' (000000001bd064f6): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpiochip597' (00000000a32f70da): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: '(null)' (00000000aab83a1c): kobject_release, parent 000000008385ae47 (delayed 1000)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:32 pc-mtodorov kernel: kobject: 'node0' (00000000c6c3323c): kobject_release, parent 00000000c2405972 (delayed 250)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node2' (0000000017e526f4): kobject_release, parent 00000000d00083c8 (delayed 500)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node3' (00000000cd2bbef4): kobject_release, parent 00000000d00083c8 (delayed 250)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'gpiochip3' (00000000f89849bd): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node8' (0000000050fa69b3): kobject_release, parent 00000000d00083c8 (delayed 250)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'gpiochip11' (00000000b39a3c5f): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'gpiochip12' (0000000051b24fd2): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node15' (00000000515b4b1b): kobject_release, parent 00000000d00083c8 (delayed 750)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node17' (0000000063e66fd0): kobject_release, parent 00000000d00083c8 (delayed 500)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node20' (000000005fd3b6d2): kobject_release, parent 00000000d00083c8 (delayed 1000)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node19' (000000008af54826): kobject_release, parent 00000000d00083c8 (delayed 750)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node21' (00000000ea466ae7): kobject_release, parent 00000000d00083c8 (delayed 500)
> Feb 27 19:00:33 pc-mtodorov kernel: kobject: 'node22' (00000000c2405972): kobject_release, parent 00000000d00083c8 (delayed 750)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip2' (0000000034808fa5): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip4' (00000000e06680f5): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'node7' (00000000f81b3244): kobject_release, parent 00000000d00083c8 (delayed 500)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip5' (000000004ef2dca6): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip7' (000000006b8d9c99): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip9' (00000000bdaa0a47): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'node10' (0000000060e34f22): kobject_release, parent 00000000d00083c8 (delayed 250)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip8' (00000000cb0b3463): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'node13' (00000000b8559267): kobject_release, parent 00000000d00083c8 (delayed 1000)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip14' (0000000092a079b6): kobject_release, parent 0000000000000000 (delayed 1000)
> Feb 27 19:00:34 pc-mtodorov kernel: kobject: 'gpiochip15' (00000000b0535d8d): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'gpiochip1' (00000000c6873be9): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node4' (000000001a955210): kobject_release, parent 00000000d00083c8 (delayed 250)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node12' (00000000a555c1b2): kobject_release, parent 00000000d00083c8 (delayed 1000)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node6' (000000003fe8f71b): kobject_release, parent 00000000d00083c8 (delayed 750)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node9' (000000000876b849): kobject_release, parent 00000000d00083c8 (delayed 750)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'gpiochip17' (000000006ee06561): kobject_release, parent 0000000000000000 (delayed 750)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node16' (000000001576402e): kobject_release, parent 00000000d00083c8 (delayed 1000)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'gpiochip13' (00000000dc27f1aa): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:35 pc-mtodorov kernel: kobject: 'node14' (00000000b738a3e4): kobject_release, parent 00000000d00083c8 (delayed 1000)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip0' (000000007063b679): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'node5' (000000002a3e415e): kobject_release, parent 00000000d00083c8 (delayed 1000)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip10' (00000000e3becc55): kobject_release, parent 0000000000000000 (delayed 500)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'node11' (000000008dc012bc): kobject_release, parent 00000000d00083c8 (delayed 500)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip6' (00000000a22546a6): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'node18' (00000000c314f13a): kobject_release, parent 00000000d00083c8 (delayed 250)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip16' (000000002be607c1): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:36 pc-mtodorov kernel: kobject: 'gpiochip19' (000000008385ae47): kobject_release, parent 0000000000000000 (delayed 250)
> Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
> Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'drivers' (00000000bca28211): kobject_release, parent 0000000096ea0bb1 (delayed 500)
> Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'holders' (00000000ef926144): kobject_release, parent 0000000096ea0bb1 (delayed 1000)
> Feb 27 19:00:43 pc-mtodorov kernel: kobject: 'notes' (000000000c2f27e4): kobject_release, parent 0000000096ea0bb1 (delayed 500)
> Feb 27 19:00:47 pc-mtodorov kernel: kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
>
> dmesg is more extensive on the events:
>
> [ 4819.240904] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_add_internal: parent: 'module', set: 'module'
> [ 4819.240925] kobject: 'holders' (00000000ef926144): kobject_add_internal: parent: 'gpio_sim', set: '<NULL>'
> [ 4819.241067] kobject: 'notes' (000000000c2f27e4): kobject_add_internal: parent: 'gpio_sim', set: '<NULL>'
> [ 4819.241106] kobject: 'gpio-sim' (000000005b8d0726): kobject_add_internal: parent: 'drivers', set: 'drivers'
> [ 4819.241134] kobject: 'drivers' (00000000bca28211): kobject_add_internal: parent: 'gpio_sim', set: '<NULL>'
> [ 4819.241148] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
> [ 4819.241155] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> [ 4819.241210] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
> [ 4819.241215] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
> [ 4819.257117] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.257172] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
> [ 4819.257177] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.257297] kobject: 'gpiochip0' (000000007063b679): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.257484] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
> [ 4819.257495] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
> [ 4819.257533] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
> [ 4819.257545] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
> [ 4819.257571] kobject: 'gpio' (0000000065b01c31): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.257577] kobject: 'gpiochip512' (000000008b9638a5): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.257629] kobject: 'gpiochip512' (000000008b9638a5): kobject_uevent_env
> [ 4819.257634] kobject: 'gpiochip512' (000000008b9638a5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip512'
> [ 4819.257687] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
> [ 4819.257693] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.275191] kobject: 'gpiochip512' (000000008b9638a5): kobject_uevent_env
> [ 4819.275202] kobject: 'gpiochip512' (000000008b9638a5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip512'
> [ 4819.275224] kobject: 'gpio' (0000000065b01c31): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.275227] kobject: 'gpiochip512' (000000008b9638a5): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.275329] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
> [ 4819.275337] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
> [ 4819.275358] kobject: 'gpiochip0' (000000007063b679): kobject_uevent_env
> [ 4819.275363] kobject: 'gpiochip0' (000000007063b679): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip0'
> [ 4819.275406] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
> [ 4819.275415] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.275432] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_uevent_env
> [ 4819.275437] kobject: 'gpio-sim.0' (00000000ca4e28ab): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.275447] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.303376] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.303430] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
> [ 4819.303435] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.303541] kobject: 'gpiochip1' (00000000c6873be9): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.303679] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
> [ 4819.303689] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
> [ 4819.303724] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
> [ 4819.303729] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
> [ 4819.303752] kobject: 'gpio' (000000007e927b76): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.303756] kobject: 'gpiochip513' (00000000307a0461): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.303800] kobject: 'gpiochip513' (00000000307a0461): kobject_uevent_env
> [ 4819.303805] kobject: 'gpiochip513' (00000000307a0461): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip513'
> [ 4819.303852] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
> [ 4819.303858] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.321116] kobject: 'gpiochip513' (00000000307a0461): kobject_uevent_env
> [ 4819.321128] kobject: 'gpiochip513' (00000000307a0461): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip513'
> [ 4819.321155] kobject: 'gpio' (000000007e927b76): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.321160] kobject: 'gpiochip513' (00000000307a0461): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.321268] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
> [ 4819.321278] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
> [ 4819.321299] kobject: 'gpiochip1' (00000000c6873be9): kobject_uevent_env
> [ 4819.321304] kobject: 'gpiochip1' (00000000c6873be9): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip1'
> [ 4819.321335] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
> [ 4819.321343] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.321362] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_uevent_env
> [ 4819.321368] kobject: 'gpio-sim.0' (000000007cf5eda9): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.321378] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.327855] kobject: 'gpio-sim.0' (00000000401b009a): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.327906] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
> [ 4819.327911] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.328025] kobject: 'gpiochip2' (0000000034808fa5): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.328130] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
> [ 4819.328137] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
> [ 4819.328221] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
> [ 4819.328228] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
> [ 4819.328250] kobject: 'gpio' (000000008afdcce8): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.328255] kobject: 'gpiochip514' (000000008e860d8b): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.328303] kobject: 'gpiochip514' (000000008e860d8b): kobject_uevent_env
> [ 4819.328309] kobject: 'gpiochip514' (000000008e860d8b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip514'
> [ 4819.328354] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
> [ 4819.328360] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.347557] kobject: 'gpiochip514' (000000008e860d8b): kobject_uevent_env
> [ 4819.347567] kobject: 'gpiochip514' (000000008e860d8b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip514'
> [ 4819.347589] kobject: 'gpio' (000000008afdcce8): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.347593] kobject: 'gpiochip514' (000000008e860d8b): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.347703] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
> [ 4819.347711] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
> [ 4819.347731] kobject: 'gpiochip2' (0000000034808fa5): kobject_uevent_env
> [ 4819.347736] kobject: 'gpiochip2' (0000000034808fa5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip2'
> [ 4819.347774] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
> [ 4819.347782] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.347800] kobject: 'gpio-sim.0' (00000000401b009a): kobject_uevent_env
> [ 4819.347806] kobject: 'gpio-sim.0' (00000000401b009a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.347816] kobject: 'gpio-sim.0' (00000000401b009a): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.355214] kobject: 'gpio-sim.0' (00000000b204b566): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.355265] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
> [ 4819.355270] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.355364] kobject: 'gpiochip3' (00000000f89849bd): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.355495] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
> [ 4819.355505] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
> [ 4819.355537] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
> [ 4819.355543] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
> [ 4819.355564] kobject: 'gpio' (00000000a1399cd0): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.355568] kobject: 'gpiochip515' (000000002b851bb5): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.355608] kobject: 'gpiochip515' (000000002b851bb5): kobject_uevent_env
> [ 4819.355614] kobject: 'gpiochip515' (000000002b851bb5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip515'
> [ 4819.355954] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
> [ 4819.355961] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.375898] kobject: 'gpiochip515' (000000002b851bb5): kobject_uevent_env
> [ 4819.375908] kobject: 'gpiochip515' (000000002b851bb5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip515'
> [ 4819.375930] kobject: 'gpio' (00000000a1399cd0): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.375932] kobject: 'gpiochip515' (000000002b851bb5): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.375998] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
> [ 4819.376004] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
> [ 4819.376018] kobject: 'gpiochip3' (00000000f89849bd): kobject_uevent_env
> [ 4819.376022] kobject: 'gpiochip3' (00000000f89849bd): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip3'
> [ 4819.376048] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
> [ 4819.376055] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.376069] kobject: 'gpio-sim.0' (00000000b204b566): kobject_uevent_env
> [ 4819.376074] kobject: 'gpio-sim.0' (00000000b204b566): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.376083] kobject: 'gpio-sim.0' (00000000b204b566): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.382519] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.382577] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
> [ 4819.382584] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.382684] kobject: 'gpiochip4' (00000000e06680f5): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.382826] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
> [ 4819.382838] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
> [ 4819.382874] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
> [ 4819.382880] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
> [ 4819.382904] kobject: 'gpio' (00000000babdf5db): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.382909] kobject: 'gpiochip531' (00000000ad4538e8): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.382952] kobject: 'gpiochip531' (00000000ad4538e8): kobject_uevent_env
> [ 4819.382958] kobject: 'gpiochip531' (00000000ad4538e8): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip531'
> [ 4819.383005] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
> [ 4819.383010] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.403312] kobject: 'gpiochip531' (00000000ad4538e8): kobject_uevent_env
> [ 4819.403323] kobject: 'gpiochip531' (00000000ad4538e8): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip531'
> [ 4819.403343] kobject: 'gpio' (00000000babdf5db): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.403346] kobject: 'gpiochip531' (00000000ad4538e8): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.403445] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
> [ 4819.403454] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
> [ 4819.403473] kobject: 'gpiochip4' (00000000e06680f5): kobject_uevent_env
> [ 4819.403477] kobject: 'gpiochip4' (00000000e06680f5): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip4'
> [ 4819.403514] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
> [ 4819.403521] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.403537] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_uevent_env
> [ 4819.403543] kobject: 'gpio-sim.0' (00000000b9ef5a28): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.403553] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.409746] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.409799] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
> [ 4819.409804] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.409899] kobject: 'gpiochip5' (000000004ef2dca6): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.410038] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
> [ 4819.410048] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
> [ 4819.410081] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
> [ 4819.410087] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
> [ 4819.410110] kobject: 'gpio' (000000008b98f863): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.410114] kobject: 'gpiochip532' (00000000ac07803f): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.410154] kobject: 'gpiochip532' (00000000ac07803f): kobject_uevent_env
> [ 4819.410159] kobject: 'gpiochip532' (00000000ac07803f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip532'
> [ 4819.410202] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
> [ 4819.410207] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.427858] kobject: 'gpiochip532' (00000000ac07803f): kobject_uevent_env
> [ 4819.427870] kobject: 'gpiochip532' (00000000ac07803f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip532'
> [ 4819.427891] kobject: 'gpio' (000000008b98f863): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.427895] kobject: 'gpiochip532' (00000000ac07803f): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.427970] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
> [ 4819.427976] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
> [ 4819.427994] kobject: 'gpiochip5' (000000004ef2dca6): kobject_uevent_env
> [ 4819.427999] kobject: 'gpiochip5' (000000004ef2dca6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip5'
> [ 4819.428044] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
> [ 4819.428053] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.428071] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_uevent_env
> [ 4819.428077] kobject: 'gpio-sim.0' (0000000034300a6b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.428088] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.440540] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.440596] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
> [ 4819.440601] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.440694] kobject: 'gpiochip6' (00000000a22546a6): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.440824] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
> [ 4819.440834] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
> [ 4819.440869] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
> [ 4819.440875] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
> [ 4819.440898] kobject: 'gpio' (00000000dedc22f2): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.440903] kobject: 'gpiochip533' (000000003541481f): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.440943] kobject: 'gpiochip533' (000000003541481f): kobject_uevent_env
> [ 4819.440948] kobject: 'gpiochip533' (000000003541481f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip533'
> [ 4819.441305] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
> [ 4819.441311] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.470191] kobject: 'gpiochip533' (000000003541481f): kobject_uevent_env
> [ 4819.470202] kobject: 'gpiochip533' (000000003541481f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip533'
> [ 4819.470226] kobject: 'gpio' (00000000dedc22f2): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.470230] kobject: 'gpiochip533' (000000003541481f): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.470337] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
> [ 4819.470345] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
> [ 4819.470364] kobject: 'gpiochip6' (00000000a22546a6): kobject_uevent_env
> [ 4819.470369] kobject: 'gpiochip6' (00000000a22546a6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip6'
> [ 4819.470401] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
> [ 4819.470408] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.470425] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_uevent_env
> [ 4819.470431] kobject: 'gpio-sim.0' (00000000cdc367f1): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.470441] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.479607] kobject: 'gpio-sim.0' (00000000bb358104): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.479661] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
> [ 4819.479667] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.479757] kobject: 'gpiochip7' (000000006b8d9c99): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.479878] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
> [ 4819.479885] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
> [ 4819.479914] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
> [ 4819.479919] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
> [ 4819.479939] kobject: 'gpio' (00000000e467cb12): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.479943] kobject: 'gpiochip549' (0000000014379c6d): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.479990] kobject: 'gpiochip549' (0000000014379c6d): kobject_uevent_env
> [ 4819.479995] kobject: 'gpiochip549' (0000000014379c6d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip549'
> [ 4819.480055] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
> [ 4819.480060] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.506928] kobject: 'gpiochip549' (0000000014379c6d): kobject_uevent_env
> [ 4819.506940] kobject: 'gpiochip549' (0000000014379c6d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip549'
> [ 4819.506961] kobject: 'gpio' (00000000e467cb12): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.506965] kobject: 'gpiochip549' (0000000014379c6d): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.507073] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
> [ 4819.507083] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
> [ 4819.507104] kobject: 'gpiochip7' (000000006b8d9c99): kobject_uevent_env
> [ 4819.507109] kobject: 'gpiochip7' (000000006b8d9c99): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip7'
> [ 4819.507137] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
> [ 4819.507145] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.507162] kobject: 'gpio-sim.0' (00000000bb358104): kobject_uevent_env
> [ 4819.507167] kobject: 'gpio-sim.0' (00000000bb358104): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.507178] kobject: 'gpio-sim.0' (00000000bb358104): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.538632] kobject: 'gpio-sim.0' (00000000461517ae): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.538690] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
> [ 4819.538695] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.538804] kobject: 'gpiochip8' (00000000cb0b3463): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.538943] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
> [ 4819.538953] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
> [ 4819.538990] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
> [ 4819.538996] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
> [ 4819.539020] kobject: 'gpio' (0000000016f05b5a): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.539025] kobject: 'gpiochip551' (00000000a55e630b): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.539066] kobject: 'gpiochip551' (00000000a55e630b): kobject_uevent_env
> [ 4819.539072] kobject: 'gpiochip551' (00000000a55e630b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip551'
> [ 4819.539118] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
> [ 4819.539124] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.545644] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.545706] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
> [ 4819.545712] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
> [ 4819.545808] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_add_internal: parent: 'gpio-sim.1', set: 'devices'
> [ 4819.545937] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
> [ 4819.545948] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
> [ 4819.545981] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
> [ 4819.545986] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
> [ 4819.546010] kobject: 'gpio' (00000000176df109): kobject_add_internal: parent: 'gpio-sim.1', set: '(null)'
> [ 4819.546014] kobject: 'gpiochip552' (000000001750e50f): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.546056] kobject: 'gpiochip552' (000000001750e50f): kobject_uevent_env
> [ 4819.546061] kobject: 'gpiochip552' (000000001750e50f): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpio/gpiochip552'
> [ 4819.546105] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
> [ 4819.546112] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
> [ 4819.553126] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.553181] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
> [ 4819.553188] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
> [ 4819.553297] kobject: 'gpiochip10' (00000000e3becc55): kobject_add_internal: parent: 'gpio-sim.2', set: 'devices'
> [ 4819.553398] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
> [ 4819.553404] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
> [ 4819.553432] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
> [ 4819.553437] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
> [ 4819.553458] kobject: 'gpio' (00000000886e1a77): kobject_add_internal: parent: 'gpio-sim.2', set: '(null)'
> [ 4819.553463] kobject: 'gpiochip553' (0000000010cfe833): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.553504] kobject: 'gpiochip553' (0000000010cfe833): kobject_uevent_env
> [ 4819.553512] kobject: 'gpiochip553' (0000000010cfe833): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpio/gpiochip553'
> [ 4819.553561] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
> [ 4819.553569] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
> [ 4819.568521] kobject: 'gpiochip551' (00000000a55e630b): kobject_uevent_env
> [ 4819.568536] kobject: 'gpiochip551' (00000000a55e630b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip551'
> [ 4819.568570] kobject: 'gpio' (0000000016f05b5a): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.568574] kobject: 'gpiochip551' (00000000a55e630b): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.568864] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
> [ 4819.568871] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
> [ 4819.568890] kobject: 'gpiochip8' (00000000cb0b3463): kobject_uevent_env
> [ 4819.568895] kobject: 'gpiochip8' (00000000cb0b3463): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip8'
> [ 4819.568932] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
> [ 4819.568939] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.568954] kobject: 'gpio-sim.0' (00000000461517ae): kobject_uevent_env
> [ 4819.568960] kobject: 'gpio-sim.0' (00000000461517ae): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.568970] kobject: 'gpio-sim.0' (00000000461517ae): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.583881] kobject: 'gpiochip552' (000000001750e50f): kobject_uevent_env
> [ 4819.583892] kobject: 'gpiochip552' (000000001750e50f): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpio/gpiochip552'
> [ 4819.583914] kobject: 'gpio' (00000000176df109): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.583918] kobject: 'gpiochip552' (000000001750e50f): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.583988] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
> [ 4819.583994] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
> [ 4819.584007] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_uevent_env
> [ 4819.584013] kobject: 'gpiochip9' (00000000bdaa0a47): fill_kobj_path: path = '/devices/platform/gpio-sim.1/gpiochip9'
> [ 4819.584048] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
> [ 4819.584056] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
> [ 4819.584072] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_uevent_env
> [ 4819.584077] kobject: 'gpio-sim.1' (0000000018f833bf): fill_kobj_path: path = '/devices/platform/gpio-sim.1'
> [ 4819.584087] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.599281] kobject: 'gpiochip553' (0000000010cfe833): kobject_uevent_env
> [ 4819.599292] kobject: 'gpiochip553' (0000000010cfe833): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpio/gpiochip553'
> [ 4819.599316] kobject: 'gpio' (00000000886e1a77): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.599319] kobject: 'gpiochip553' (0000000010cfe833): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.599418] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
> [ 4819.599426] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
> [ 4819.599441] kobject: 'gpiochip10' (00000000e3becc55): kobject_uevent_env
> [ 4819.599446] kobject: 'gpiochip10' (00000000e3becc55): fill_kobj_path: path = '/devices/platform/gpio-sim.2/gpiochip10'
> [ 4819.599479] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
> [ 4819.599488] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
> [ 4819.599504] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_uevent_env
> [ 4819.599509] kobject: 'gpio-sim.2' (00000000bbfd5238): fill_kobj_path: path = '/devices/platform/gpio-sim.2'
> [ 4819.599521] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.606440] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.606495] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
> [ 4819.606501] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.606609] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.606742] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
> [ 4819.606753] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
> [ 4819.606789] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
> [ 4819.606795] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
> [ 4819.606817] kobject: 'gpio' (000000002835f4a4): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.606821] kobject: 'gpiochip554' (00000000e0038afb): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.606864] kobject: 'gpiochip554' (00000000e0038afb): kobject_uevent_env
> [ 4819.606869] kobject: 'gpiochip554' (00000000e0038afb): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip554'
> [ 4819.606914] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
> [ 4819.606919] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.622045] kobject: 'gpiochip554' (00000000e0038afb): kobject_uevent_env
> [ 4819.622055] kobject: 'gpiochip554' (00000000e0038afb): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip554'
> [ 4819.622076] kobject: 'gpio' (000000002835f4a4): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.622079] kobject: 'gpiochip554' (00000000e0038afb): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.622179] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
> [ 4819.622187] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
> [ 4819.622205] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_uevent_env
> [ 4819.622210] kobject: 'gpiochip11' (00000000b39a3c5f): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip11'
> [ 4819.622243] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
> [ 4819.622252] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.622268] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_uevent_env
> [ 4819.622274] kobject: 'gpio-sim.0' (00000000e8c2899c): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.622283] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.628405] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.628464] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
> [ 4819.628471] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.628588] kobject: 'gpiochip12' (0000000051b24fd2): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.628727] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
> [ 4819.628737] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
> [ 4819.628772] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
> [ 4819.628777] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
> [ 4819.628799] kobject: 'gpio' (00000000d962e90a): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.628804] kobject: 'gpiochip555' (00000000f5bbd198): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.628845] kobject: 'gpiochip555' (00000000f5bbd198): kobject_uevent_env
> [ 4819.628850] kobject: 'gpiochip555' (00000000f5bbd198): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip555'
> [ 4819.628893] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
> [ 4819.628899] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.646727] kobject: 'gpiochip555' (00000000f5bbd198): kobject_uevent_env
> [ 4819.646740] kobject: 'gpiochip555' (00000000f5bbd198): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip555'
> [ 4819.646764] kobject: 'gpio' (00000000d962e90a): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.646768] kobject: 'gpiochip555' (00000000f5bbd198): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.646880] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
> [ 4819.646889] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
> [ 4819.646912] kobject: 'gpiochip12' (0000000051b24fd2): kobject_uevent_env
> [ 4819.646918] kobject: 'gpiochip12' (0000000051b24fd2): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip12'
> [ 4819.646962] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
> [ 4819.646971] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.646991] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_uevent_env
> [ 4819.646997] kobject: 'gpio-sim.0' (000000008ef7f514): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.647009] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.654427] kobject: 'gpio-sim.0' (000000005418b2af): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.654482] kobject: 'gpio-sim.0' (000000005418b2af): kobject_uevent_env
> [ 4819.654488] kobject: 'gpio-sim.0' (000000005418b2af): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.654536] gpio-sim: probe of gpio-sim.0 failed with error -34
> [ 4819.654556] kobject: 'gpio-sim.0' (000000005418b2af): kobject_uevent_env
> [ 4819.654562] kobject: 'gpio-sim.0' (000000005418b2af): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.654576] kobject: 'gpio-sim.0' (000000005418b2af): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.717310] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.717365] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
> [ 4819.717371] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.717473] gpio-560 (?): hogged as input
> [ 4819.717479] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.717621] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
> [ 4819.717630] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
> [ 4819.717669] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
> [ 4819.717676] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
> [ 4819.717698] kobject: 'gpio' (000000006cce49d7): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.717703] kobject: 'gpiochip556' (00000000d2b04f32): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.717753] kobject: 'gpiochip556' (00000000d2b04f32): kobject_uevent_env
> [ 4819.717760] kobject: 'gpiochip556' (00000000d2b04f32): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip556'
> [ 4819.717948] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
> [ 4819.717954] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.740722] kobject: 'gpiochip556' (00000000d2b04f32): kobject_uevent_env
> [ 4819.740733] kobject: 'gpiochip556' (00000000d2b04f32): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip556'
> [ 4819.740753] kobject: 'gpio' (000000006cce49d7): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.740757] kobject: 'gpiochip556' (00000000d2b04f32): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.740863] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
> [ 4819.740871] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
> [ 4819.740890] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_uevent_env
> [ 4819.740896] kobject: 'gpiochip13' (00000000dc27f1aa): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip13'
> [ 4819.740928] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
> [ 4819.740936] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.740951] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_uevent_env
> [ 4819.740958] kobject: 'gpio-sim.0' (00000000cd797c7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.740970] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.747033] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.747082] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
> [ 4819.747087] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.747182] kobject: 'gpiochip14' (0000000092a079b6): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.747311] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
> [ 4819.747321] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
> [ 4819.747354] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
> [ 4819.747359] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
> [ 4819.747381] kobject: 'gpio' (000000007164bff6): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.747386] kobject: 'gpiochip564' (00000000ee87da7b): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.747426] kobject: 'gpiochip564' (00000000ee87da7b): kobject_uevent_env
> [ 4819.747431] kobject: 'gpiochip564' (00000000ee87da7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip564'
> [ 4819.747622] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
> [ 4819.747629] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.782537] kobject: 'gpiochip564' (00000000ee87da7b): kobject_uevent_env
> [ 4819.782547] kobject: 'gpiochip564' (00000000ee87da7b): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip564'
> [ 4819.782567] kobject: 'gpio' (000000007164bff6): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.782571] kobject: 'gpiochip564' (00000000ee87da7b): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.782653] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
> [ 4819.782662] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
> [ 4819.782680] kobject: 'gpiochip14' (0000000092a079b6): kobject_uevent_env
> [ 4819.782685] kobject: 'gpiochip14' (0000000092a079b6): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip14'
> [ 4819.782719] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
> [ 4819.782729] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.782745] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_uevent_env
> [ 4819.782750] kobject: 'gpio-sim.0' (000000006053a4bf): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.782760] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.789540] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.789591] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
> [ 4819.789597] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.789698] kobject: 'gpiochip15' (00000000b0535d8d): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.789829] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
> [ 4819.789839] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
> [ 4819.789873] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
> [ 4819.789880] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
> [ 4819.789902] kobject: 'gpio' (0000000012ac3b4a): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.789907] kobject: 'gpiochip572' (00000000178509ff): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.789957] kobject: 'gpiochip572' (00000000178509ff): kobject_uevent_env
> [ 4819.789963] kobject: 'gpiochip572' (00000000178509ff): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip572'
> [ 4819.790172] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
> [ 4819.790179] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.820474] kobject: 'gpiochip572' (00000000178509ff): kobject_uevent_env
> [ 4819.820489] kobject: 'gpiochip572' (00000000178509ff): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip572'
> [ 4819.820515] kobject: 'gpio' (0000000012ac3b4a): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.820519] kobject: 'gpiochip572' (00000000178509ff): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4819.820629] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
> [ 4819.820640] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
> [ 4819.820664] kobject: 'gpiochip15' (00000000b0535d8d): kobject_uevent_env
> [ 4819.820669] kobject: 'gpiochip15' (00000000b0535d8d): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip15'
> [ 4819.820715] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
> [ 4819.820724] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.820745] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_uevent_env
> [ 4819.820751] kobject: 'gpio-sim.0' (00000000c56fc456): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.820761] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.827333] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.827384] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
> [ 4819.827390] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.827489] kobject: 'gpiochip16' (000000002be607c1): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.827596] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
> [ 4819.827606] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
> [ 4819.827647] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
> [ 4819.827652] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
> [ 4819.827675] kobject: 'gpio' (000000000333f422): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.827679] kobject: 'gpiochip580' (00000000f6371a1e): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.827719] kobject: 'gpiochip580' (00000000f6371a1e): kobject_uevent_env
> [ 4819.827724] kobject: 'gpiochip580' (00000000f6371a1e): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip580'
> [ 4819.827920] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
> [ 4819.827927] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.849122] kobject: 'gpiochip580' (00000000f6371a1e): kobject_uevent_env
> [ 4819.849136] kobject: 'gpiochip580' (00000000f6371a1e): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip580'
> [ 4819.849160] kobject: 'gpio' (000000000333f422): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.849163] kobject: 'gpiochip580' (00000000f6371a1e): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.849279] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
> [ 4819.849293] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
> [ 4819.849317] kobject: 'gpiochip16' (000000002be607c1): kobject_uevent_env
> [ 4819.849323] kobject: 'gpiochip16' (000000002be607c1): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip16'
> [ 4819.849381] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
> [ 4819.849391] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.849421] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_uevent_env
> [ 4819.849428] kobject: 'gpio-sim.0' (000000000471ebbe): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.849442] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.856384] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.856438] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
> [ 4819.856445] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.856563] kobject: 'gpiochip17' (000000006ee06561): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.856712] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
> [ 4819.856724] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
> [ 4819.856786] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
> [ 4819.856794] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
> [ 4819.856821] kobject: 'gpio' (0000000005a238f5): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.856825] kobject: 'gpiochip588' (0000000075b76b85): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.856868] kobject: 'gpiochip588' (0000000075b76b85): kobject_uevent_env
> [ 4819.856874] kobject: 'gpiochip588' (0000000075b76b85): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip588'
> [ 4819.856922] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
> [ 4819.856928] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.878720] kobject: 'gpiochip588' (0000000075b76b85): kobject_uevent_env
> [ 4819.878731] kobject: 'gpiochip588' (0000000075b76b85): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip588'
> [ 4819.878750] kobject: 'gpio' (0000000005a238f5): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4819.878755] kobject: 'gpiochip588' (0000000075b76b85): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4819.878864] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
> [ 4819.878872] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
> [ 4819.878890] kobject: 'gpiochip17' (000000006ee06561): kobject_uevent_env
> [ 4819.878896] kobject: 'gpiochip17' (000000006ee06561): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip17'
> [ 4819.878928] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
> [ 4819.878936] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.878950] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_uevent_env
> [ 4819.878954] kobject: 'gpio-sim.0' (0000000048f1330d): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.878964] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4819.885904] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4819.885967] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
> [ 4819.885973] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4819.886080] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4819.886214] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
> [ 4819.886224] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
> [ 4819.886257] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
> [ 4819.886264] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
> [ 4819.886289] kobject: 'gpio' (0000000064e73453): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4819.886293] kobject: 'gpiochip589' (000000007e1cf13a): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4819.886340] kobject: 'gpiochip589' (000000007e1cf13a): kobject_uevent_env
> [ 4819.886345] kobject: 'gpiochip589' (000000007e1cf13a): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip589'
> [ 4819.886554] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
> [ 4819.886560] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.013863] kobject: 'gpiochip589' (000000007e1cf13a): kobject_uevent_env
> [ 4820.013872] kobject: 'gpiochip589' (000000007e1cf13a): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip589'
> [ 4820.013894] kobject: 'gpio' (0000000064e73453): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4820.013897] kobject: 'gpiochip589' (000000007e1cf13a): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4820.013994] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
> [ 4820.014004] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
> [ 4820.014022] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_uevent_env
> [ 4820.014027] kobject: 'gpiochip18' (00000000a8ca1f8c): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip18'
> [ 4820.014059] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
> [ 4820.014065] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.014082] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_uevent_env
> [ 4820.014087] kobject: 'gpio-sim.0' (00000000f462f78a): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.014098] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4820.020299] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_add_internal: parent: 'platform', set: 'devices'
> [ 4820.020347] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
> [ 4820.020352] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.020447] kobject: 'gpiochip19' (000000008385ae47): kobject_add_internal: parent: 'gpio-sim.0', set: 'devices'
> [ 4820.020581] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
> [ 4820.020590] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
> [ 4820.020622] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
> [ 4820.020627] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
> [ 4820.020651] kobject: 'gpio' (000000001bd064f6): kobject_add_internal: parent: 'gpio-sim.0', set: '(null)'
> [ 4820.020655] kobject: 'gpiochip597' (00000000a32f70da): kobject_add_internal: parent: 'gpio', set: 'devices'
> [ 4820.020701] kobject: 'gpiochip597' (00000000a32f70da): kobject_uevent_env
> [ 4820.020707] kobject: 'gpiochip597' (00000000a32f70da): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip597'
> [ 4820.020890] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
> [ 4820.020896] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.048138] kobject: 'gpiochip597' (00000000a32f70da): kobject_uevent_env
> [ 4820.048148] kobject: 'gpiochip597' (00000000a32f70da): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpio/gpiochip597'
> [ 4820.048175] kobject: 'gpio' (000000001bd064f6): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4820.048179] kobject: 'gpiochip597' (00000000a32f70da): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4820.048273] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
> [ 4820.048282] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
> [ 4820.048307] kobject: 'gpiochip19' (000000008385ae47): kobject_uevent_env
> [ 4820.048313] kobject: 'gpiochip19' (000000008385ae47): fill_kobj_path: path = '/devices/platform/gpio-sim.0/gpiochip19'
> [ 4820.048356] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
> [ 4820.048364] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.048385] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_uevent_env
> [ 4820.048391] kobject: 'gpio-sim.0' (00000000b1d6276b): fill_kobj_path: path = '/devices/platform/gpio-sim.0'
> [ 4820.048407] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4820.304376] kobject: 'gpiochip512' (000000008b9638a5): kobject_cleanup, parent 0000000000000000
> [ 4820.304378] kobject: 'gpiochip512' (000000008b9638a5): calling ktype release
> [ 4820.304384] kobject: 'gpiochip512': free name
> [ 4820.304387] kobject: 'gpio' (0000000065b01c31): kobject_cleanup, parent 0000000000000000
> [ 4820.304388] kobject: 'gpio' (0000000065b01c31): calling ktype release
> [ 4820.304391] kobject: 'gpio': free name
> [ 4820.400182] kobject: 'gpiochip3' (00000000f89849bd): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4820.436325] kobject: 'gpio' (00000000babdf5db): kobject_cleanup, parent 0000000000000000
> [ 4820.436331] kobject: 'gpio' (00000000babdf5db): calling ktype release
> [ 4820.436335] kobject: 'gpio': free name
> [ 4820.436344] kobject: 'gpio-sim.0' (0000000034300a6b): kobject_cleanup, parent 0000000000000000
> [ 4820.436348] kobject: 'gpio-sim.0' (0000000034300a6b): calling ktype release
> [ 4820.436355] kobject: 'gpio-sim.0': free name
> [ 4820.436357] kobject: 'gpio' (000000008b98f863): kobject_cleanup, parent 0000000000000000
> [ 4820.436359] kobject: 'gpio' (000000008b98f863): calling ktype release
> [ 4820.436363] kobject: 'gpio': free name
> [ 4820.496304] kobject: 'gpio-sim.0' (00000000cdc367f1): kobject_cleanup, parent 0000000000000000
> [ 4820.496306] kobject: 'gpio-sim.0' (00000000cdc367f1): calling ktype release
> [ 4820.496312] kobject: 'gpio-sim.0': free name
> [ 4820.528235] kobject: 'gpiochip549' (0000000014379c6d): kobject_cleanup, parent 0000000000000000
> [ 4820.528242] kobject: 'gpiochip549' (0000000014379c6d): calling ktype release
> [ 4820.528248] kobject: 'gpiochip549': free name
> [ 4820.596327] kobject: 'gpiochip551' (00000000a55e630b): kobject_cleanup, parent 0000000000000000
> [ 4820.596333] kobject: 'gpiochip551' (00000000a55e630b): calling ktype release
> [ 4820.596341] kobject: 'gpiochip551': free name
> [ 4820.624451] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4820.624455] kobject: 'gpio' (000000002835f4a4): kobject_cleanup, parent 0000000000000000
> [ 4820.624456] kobject: 'gpio' (000000002835f4a4): calling ktype release
> [ 4820.624461] kobject: 'gpio': free name
> [ 4820.656309] kobject: 'gpiochip12' (0000000051b24fd2): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4820.656313] kobject: 'gpiochip555' (00000000f5bbd198): kobject_cleanup, parent 0000000000000000
> [ 4820.656315] kobject: 'gpiochip555' (00000000f5bbd198): calling ktype release
> [ 4820.656321] kobject: 'gpiochip555': free name
> [ 4820.656324] kobject: 'gpio' (00000000d962e90a): kobject_cleanup, parent 0000000000000000
> [ 4820.656326] kobject: 'gpio' (00000000d962e90a): calling ktype release
> [ 4820.656329] kobject: 'gpio': free name
> [ 4820.752233] kobject: 'gpiochip556' (00000000d2b04f32): kobject_cleanup, parent 0000000000000000
> [ 4820.752240] kobject: 'gpiochip556' (00000000d2b04f32): calling ktype release
> [ 4820.752246] kobject: 'gpiochip556': free name
> [ 4820.788358] kobject: 'gpio-sim.0' (000000006053a4bf): kobject_cleanup, parent 0000000000000000
> [ 4820.788360] kobject: 'gpio-sim.0' (000000006053a4bf): calling ktype release
> [ 4820.788365] kobject: 'gpio-sim.0': free name
> [ 4820.788367] kobject: 'gpio' (000000007164bff6): kobject_cleanup, parent 0000000000000000
> [ 4820.788369] kobject: 'gpio' (000000007164bff6): calling ktype release
> [ 4820.788373] kobject: 'gpio': free name
> [ 4820.848443] kobject: 'gpio-sim.0' (00000000c56fc456): kobject_cleanup, parent 0000000000000000
> [ 4820.848449] kobject: 'gpio-sim.0' (00000000c56fc456): calling ktype release
> [ 4820.848457] kobject: 'gpio-sim.0': free name
> [ 4820.852177] kobject: 'gpio' (0000000012ac3b4a): kobject_cleanup, parent 0000000000000000
> [ 4820.852183] kobject: 'gpio' (0000000012ac3b4a): calling ktype release
> [ 4820.852187] kobject: 'gpio': free name
> [ 4820.880305] kobject: 'gpio' (000000000333f422): kobject_cleanup, parent 0000000000000000
> [ 4820.880326] kobject: 'gpio' (000000000333f422): calling ktype release
> [ 4820.880332] kobject: 'gpio': free name
> [ 4820.884286] kobject: 'gpio-sim.0' (0000000048f1330d): kobject_cleanup, parent 0000000000000000
> [ 4820.884288] kobject: 'gpio-sim.0' (0000000048f1330d): calling ktype release
> [ 4820.884293] kobject: 'gpio-sim.0': free name
> [ 4820.884296] kobject: 'gpio-sim.0' (000000000471ebbe): kobject_cleanup, parent 0000000000000000
> [ 4820.884298] kobject: 'gpio-sim.0' (000000000471ebbe): calling ktype release
> [ 4820.884304] kobject: 'gpio-sim.0': free name
> [ 4821.076363] kobject: 'gpio-sim.0' (00000000b1d6276b): kobject_cleanup, parent 0000000000000000
> [ 4821.076365] kobject: 'gpio-sim.0' (00000000b1d6276b): calling ktype release
> [ 4821.076370] kobject: 'gpio-sim.0': free name
> [ 4821.328246] kobject: 'gpio' (000000007e927b76): kobject_cleanup, parent 0000000000000000
> [ 4821.328252] kobject: 'gpio' (000000007e927b76): calling ktype release
> [ 4821.328258] kobject: 'gpio': free name
> [ 4821.360465] kobject: 'gpiochip2' (0000000034808fa5): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4821.360468] kobject: 'gpio' (000000008afdcce8): kobject_cleanup, parent 0000000000000000
> [ 4821.360470] kobject: 'gpio' (000000008afdcce8): calling ktype release
> [ 4821.360475] kobject: 'gpio': free name
> [ 4821.392179] kobject: 'gpio-sim.0' (00000000b204b566): kobject_cleanup, parent 0000000000000000
> [ 4821.392186] kobject: 'gpio-sim.0' (00000000b204b566): calling ktype release
> [ 4821.392193] kobject: 'gpio-sim.0': free name
> [ 4821.392196] kobject: 'gpiochip515' (000000002b851bb5): kobject_cleanup, parent 0000000000000000
> [ 4821.392221] kobject: 'gpiochip515' (000000002b851bb5): calling ktype release
> [ 4821.392226] kobject: 'gpiochip515': free name
> [ 4821.428255] kobject: 'gpio-sim.0' (00000000b9ef5a28): kobject_cleanup, parent 0000000000000000
> [ 4821.428262] kobject: 'gpio-sim.0' (00000000b9ef5a28): calling ktype release
> [ 4821.428283] kobject: 'gpio-sim.0': free name
> [ 4821.428291] kobject: 'gpiochip4' (00000000e06680f5): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4821.460377] kobject: 'gpiochip5' (000000004ef2dca6): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4821.460380] kobject: 'gpiochip532' (00000000ac07803f): kobject_cleanup, parent 0000000000000000
> [ 4821.460382] kobject: 'gpiochip532' (00000000ac07803f): calling ktype release
> [ 4821.460386] kobject: 'gpiochip532': free name
> [ 4821.488407] kobject: 'gpio' (00000000dedc22f2): kobject_cleanup, parent 0000000000000000
> [ 4821.488426] kobject: 'gpio' (00000000dedc22f2): calling ktype release
> [ 4821.488432] kobject: 'gpio': free name
> [ 4821.524217] kobject: 'gpiochip7' (000000006b8d9c99): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4821.588303] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4821.588372] kobject: 'gpiochip8' (00000000cb0b3463): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4821.588375] kobject: 'gpio' (0000000016f05b5a): kobject_cleanup, parent 0000000000000000
> [ 4821.588377] kobject: 'gpio' (0000000016f05b5a): calling ktype release
> [ 4821.588380] kobject: 'gpio': free name
> [ 4821.616206] kobject: 'gpiochip553' (0000000010cfe833): kobject_cleanup, parent 0000000000000000
> [ 4821.616213] kobject: 'gpiochip553' (0000000010cfe833): calling ktype release
> [ 4821.616224] kobject: 'gpiochip553': free name
> [ 4821.616228] kobject: 'gpio' (00000000886e1a77): kobject_cleanup, parent 0000000000000000
> [ 4821.616230] kobject: 'gpio' (00000000886e1a77): calling ktype release
> [ 4821.616234] kobject: 'gpio': free name
> [ 4821.652317] kobject: 'gpio-sim.0' (000000008ef7f514): kobject_cleanup, parent 0000000000000000
> [ 4821.652323] kobject: 'gpio-sim.0' (000000008ef7f514): calling ktype release
> [ 4821.652348] kobject: 'gpio-sim.0': free name
> [ 4821.684240] kobject: 'gpiochip12' (0000000051b24fd2): kobject_cleanup, parent 0000000000000000
> [ 4821.684246] kobject: 'gpiochip12' (0000000051b24fd2): calling ktype release
> [ 4821.684275] kobject: 'gpiochip12': free name
> [ 4821.748311] kobject: 'gpio-sim.0' (00000000cd797c7b): kobject_cleanup, parent 0000000000000000
> [ 4821.748317] kobject: 'gpio-sim.0' (00000000cd797c7b): calling ktype release
> [ 4821.748325] kobject: 'gpio-sim.0': free name
> [ 4821.748328] kobject: 'gpio' (000000006cce49d7): kobject_cleanup, parent 0000000000000000
> [ 4821.748330] kobject: 'gpio' (000000006cce49d7): calling ktype release
> [ 4821.748333] kobject: 'gpio': free name
> [ 4821.812244] kobject: 'gpiochip14' (0000000092a079b6): kobject_release, parent 0000000000000000 (delayed 1000)
> [ 4821.840208] kobject: 'gpiochip15' (00000000b0535d8d): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4821.872262] kobject: 'gpiochip580' (00000000f6371a1e): kobject_cleanup, parent 0000000000000000
> [ 4821.872269] kobject: 'gpiochip580' (00000000f6371a1e): calling ktype release
> [ 4821.872277] kobject: 'gpiochip580': free name
> [ 4821.904224] kobject: 'gpio' (0000000005a238f5): kobject_cleanup, parent 0000000000000000
> [ 4821.904230] kobject: 'gpio' (0000000005a238f5): calling ktype release
> [ 4821.904235] kobject: 'gpio': free name
> [ 4822.036185] kobject: 'gpio-sim.0' (00000000f462f78a): kobject_cleanup, parent 0000000000000000
> [ 4822.036192] kobject: 'gpio-sim.0' (00000000f462f78a): calling ktype release
> [ 4822.036198] kobject: 'gpio-sim.0': free name
> [ 4822.036202] kobject: 'gpiochip589' (000000007e1cf13a): kobject_cleanup, parent 0000000000000000
> [ 4822.036204] kobject: 'gpiochip589' (000000007e1cf13a): calling ktype release
> [ 4822.036206] kobject: 'gpiochip589': free name
> [ 4822.068252] kobject: 'gpiochip597' (00000000a32f70da): kobject_cleanup, parent 0000000000000000
> [ 4822.068258] kobject: 'gpiochip597' (00000000a32f70da): calling ktype release
> [ 4822.068265] kobject: 'gpiochip597': free name
> [ 4822.384245] kobject: 'gpiochip1' (00000000c6873be9): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4822.384248] kobject: 'gpiochip513' (00000000307a0461): kobject_cleanup, parent 0000000000000000
> [ 4822.384250] kobject: 'gpiochip513' (00000000307a0461): calling ktype release
> [ 4822.384271] kobject: 'gpiochip513': free name
> [ 4822.384286] kobject: 'gpio-sim.0' (00000000401b009a): kobject_cleanup, parent 0000000000000000
> [ 4822.384288] kobject: 'gpio-sim.0' (00000000401b009a): calling ktype release
> [ 4822.384294] kobject: 'gpio-sim.0': free name
> [ 4822.384297] kobject: 'gpiochip514' (000000008e860d8b): kobject_cleanup, parent 0000000000000000
> [ 4822.384298] kobject: 'gpiochip514' (000000008e860d8b): calling ktype release
> [ 4822.384302] kobject: 'gpiochip514': free name
> [ 4822.384304] kobject: 'gpiochip2' (0000000034808fa5): kobject_cleanup, parent 0000000000000000
> [ 4822.384305] kobject: 'gpiochip2' (0000000034808fa5): calling ktype release
> [ 4822.384312] kobject: 'gpiochip2': free name
> [ 4822.416325] kobject: 'gpiochip3' (00000000f89849bd): kobject_cleanup, parent 0000000000000000
> [ 4822.416331] kobject: 'gpiochip3' (00000000f89849bd): calling ktype release
> [ 4822.416344] kobject: 'gpiochip3': free name
> [ 4822.452250] kobject: 'gpiochip4' (00000000e06680f5): kobject_cleanup, parent 0000000000000000
> [ 4822.452256] kobject: 'gpiochip4' (00000000e06680f5): calling ktype release
> [ 4822.452265] kobject: 'gpiochip4': free name
> [ 4822.548316] kobject: 'gpiochip7' (000000006b8d9c99): kobject_cleanup, parent 0000000000000000
> [ 4822.548322] kobject: 'gpiochip7' (000000006b8d9c99): calling ktype release
> [ 4822.548350] kobject: 'gpiochip7': free name
> [ 4822.608239] kobject: 'gpiochip8' (00000000cb0b3463): kobject_cleanup, parent 0000000000000000
> [ 4822.608260] kobject: 'gpiochip8' (00000000cb0b3463): calling ktype release
> [ 4822.608275] kobject: 'gpiochip8': free name
> [ 4822.640315] kobject: 'gpiochip554' (00000000e0038afb): kobject_cleanup, parent 0000000000000000
> [ 4822.640339] kobject: 'gpiochip554' (00000000e0038afb): calling ktype release
> [ 4822.640343] kobject: 'gpio' (00000000e467cb12): kobject_cleanup, parent 0000000000000000
> [ 4822.640346] kobject: 'gpiochip554': free name
> [ 4822.640348] kobject: 'gpio' (00000000e467cb12): calling ktype release
> [ 4822.640349] kobject: 'gpio-sim.1' (0000000018f833bf): kobject_cleanup, parent 0000000000000000
> [ 4822.640351] kobject: 'gpio-sim.1' (0000000018f833bf): calling ktype release
> [ 4822.640354] kobject: 'gpio': free name
> [ 4822.640360] kobject: 'gpio-sim.1': free name
> [ 4822.640408] kobject: 'gpio-sim.2' (00000000bbfd5238): kobject_cleanup, parent 0000000000000000
> [ 4822.640410] kobject: 'gpio-sim.2' (00000000bbfd5238): calling ktype release
> [ 4822.640416] kobject: 'gpio-sim.2': free name
> [ 4822.644186] kobject: 'gpio-sim.0' (00000000461517ae): kobject_cleanup, parent 0000000000000000
> [ 4822.644192] kobject: 'gpio-sim.0' (00000000461517ae): calling ktype release
> [ 4822.644198] kobject: 'gpio-sim.0': free name
> [ 4822.644266] kobject: 'gpio-sim.0' (00000000bb358104): kobject_cleanup, parent 0000000000000000
> [ 4822.644268] kobject: 'gpio-sim.0' (00000000bb358104): calling ktype release
> [ 4822.644272] kobject: 'gpio-sim.0': free name
> [ 4822.896320] kobject: 'gpiochip572' (00000000178509ff): kobject_cleanup, parent 0000000000000000
> [ 4822.896326] kobject: 'gpiochip572' (00000000178509ff): calling ktype release
> [ 4822.896334] kobject: 'gpiochip572': free name
> [ 4822.896337] kobject: 'gpio-sim.0' (000000005418b2af): kobject_cleanup, parent 0000000000000000
> [ 4822.896339] kobject: 'gpio-sim.0' (000000005418b2af): calling ktype release
> [ 4822.896343] kobject: 'gpio-sim.0': free name
> [ 4822.900449] kobject: 'gpiochip564' (00000000ee87da7b): kobject_cleanup, parent 0000000000000000
> [ 4822.900455] kobject: 'gpiochip564' (00000000ee87da7b): calling ktype release
> [ 4822.900461] kobject: 'gpiochip17' (000000006ee06561): kobject_release, parent 0000000000000000 (delayed 750)
> [ 4822.900463] kobject: 'gpiochip564': free name
> [ 4822.900524] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4823.152251] kobject: 'gpio' (0000000064e73453): kobject_cleanup, parent 0000000000000000
> [ 4823.152257] kobject: 'gpio' (0000000064e73453): calling ktype release
> [ 4823.152260] kobject: 'gpio': free name
> [ 4823.156258] kobject: 'gpio' (000000001bd064f6): kobject_cleanup, parent 0000000000000000
> [ 4823.156277] kobject: 'gpio' (000000001bd064f6): calling ktype release
> [ 4823.156307] kobject: 'gpio': free name
> [ 4823.408426] kobject: 'gpio-sim.0' (000000007cf5eda9): kobject_cleanup, parent 0000000000000000
> [ 4823.408433] kobject: 'gpio-sim.0' (000000007cf5eda9): calling ktype release
> [ 4823.408433] kobject: 'gpiochip531' (00000000ad4538e8): kobject_cleanup, parent 0000000000000000
> [ 4823.408436] kobject: 'gpiochip531' (00000000ad4538e8): calling ktype release
> [ 4823.408444] kobject: 'gpio-sim.0': free name
> [ 4823.408452] kobject: 'gpiochip531': free name
> [ 4823.408456] kobject: 'gpio-sim.0' (00000000ca4e28ab): kobject_cleanup, parent 0000000000000000
> [ 4823.408459] kobject: 'gpio-sim.0' (00000000ca4e28ab): calling ktype release
> [ 4823.408467] kobject: 'gpio-sim.0': free name
> [ 4823.408476] kobject: 'gpiochip0' (000000007063b679): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4823.408523] kobject: 'gpio' (00000000a1399cd0): kobject_cleanup, parent 0000000000000000
> [ 4823.408525] kobject: 'gpio' (00000000a1399cd0): calling ktype release
> [ 4823.408533] kobject: 'gpio': free name
> [ 4823.664421] kobject: 'gpio-sim.0' (00000000e8c2899c): kobject_cleanup, parent 0000000000000000
> [ 4823.664426] kobject: 'gpio-sim.0' (00000000e8c2899c): calling ktype release
> [ 4823.664450] kobject: 'gpio-sim.0': free name
> [ 4823.664516] kobject: 'gpiochip10' (00000000e3becc55): kobject_release, parent 0000000000000000 (delayed 500)
> [ 4823.664524] kobject: 'gpiochip552' (000000001750e50f): kobject_cleanup, parent 0000000000000000
> [ 4823.664526] kobject: 'gpiochip552' (000000001750e50f): calling ktype release
> [ 4823.664530] kobject: 'gpiochip552': free name
> [ 4823.664533] kobject: 'gpio' (00000000176df109): kobject_cleanup, parent 0000000000000000
> [ 4823.664535] kobject: 'gpio' (00000000176df109): calling ktype release
> [ 4823.664538] kobject: 'gpio': free name
> [ 4823.664556] kobject: 'gpiochip6' (00000000a22546a6): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4823.664558] kobject: 'gpiochip533' (000000003541481f): kobject_cleanup, parent 0000000000000000
> [ 4823.664560] kobject: 'gpiochip533' (000000003541481f): calling ktype release
> [ 4823.664564] kobject: 'gpiochip533': free name
> [ 4823.924314] kobject: 'gpiochip588' (0000000075b76b85): kobject_cleanup, parent 0000000000000000
> [ 4823.924334] kobject: 'gpiochip588' (0000000075b76b85): calling ktype release
> [ 4823.924340] kobject: 'gpiochip588': free name
> [ 4823.924344] kobject: 'gpiochip13' (00000000dc27f1aa): kobject_cleanup, parent 0000000000000000
> [ 4823.924346] kobject: 'gpiochip13' (00000000dc27f1aa): calling ktype release
> [ 4823.924354] kobject: 'gpiochip13': free name
> [ 4823.924371] kobject: 'gpiochip16' (000000002be607c1): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4824.176336] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4824.176381] kobject: 'gpiochip19' (000000008385ae47): kobject_release, parent 0000000000000000 (delayed 250)
> [ 4824.400258] kobject: 'gpiochip1' (00000000c6873be9): kobject_cleanup, parent 0000000000000000
> [ 4824.400264] kobject: 'gpiochip1' (00000000c6873be9): calling ktype release
> [ 4824.400276] kobject: 'gpiochip1': free name
> [ 4824.432259] kobject: 'gpiochip0' (000000007063b679): kobject_cleanup, parent 0000000000000000
> [ 4824.432266] kobject: 'gpiochip0' (000000007063b679): calling ktype release
> [ 4824.432278] kobject: 'gpiochip0': free name
> [ 4824.688241] kobject: 'gpiochip5' (000000004ef2dca6): kobject_cleanup, parent 0000000000000000
> [ 4824.688248] kobject: 'gpiochip5' (000000004ef2dca6): calling ktype release
> [ 4824.688261] kobject: 'gpiochip5': free name
> [ 4824.692227] kobject: 'gpiochip11' (00000000b39a3c5f): kobject_cleanup, parent 0000000000000000
> [ 4824.692246] kobject: 'gpiochip11' (00000000b39a3c5f): calling ktype release
> [ 4824.692278] kobject: 'gpiochip11': free name
> [ 4824.692290] kobject: 'gpiochip6' (00000000a22546a6): kobject_cleanup, parent 0000000000000000
> [ 4824.692317] kobject: 'gpiochip6' (00000000a22546a6): calling ktype release
> [ 4824.692338] kobject: 'gpiochip6': free name
> [ 4824.948242] kobject: 'gpiochip15' (00000000b0535d8d): kobject_cleanup, parent 0000000000000000
> [ 4824.948247] kobject: 'gpiochip15' (00000000b0535d8d): calling ktype release
> [ 4824.948251] kobject: 'gpiochip16' (000000002be607c1): kobject_cleanup, parent 0000000000000000
> [ 4824.948255] kobject: 'gpiochip16' (000000002be607c1): calling ktype release
> [ 4824.948259] kobject: 'gpiochip15': free name
> [ 4824.948272] kobject: 'gpiochip16': free name
> [ 4825.200324] kobject: 'gpiochip18' (00000000a8ca1f8c): kobject_cleanup, parent 0000000000000000
> [ 4825.200330] kobject: 'gpiochip18' (00000000a8ca1f8c): calling ktype release
> [ 4825.200343] kobject: 'gpiochip18': free name
> [ 4825.204318] kobject: 'gpiochip19' (000000008385ae47): kobject_cleanup, parent 0000000000000000
> [ 4825.204325] kobject: 'gpiochip19' (000000008385ae47): calling ktype release
> [ 4825.204340] kobject: 'gpiochip19': free name
> [ 4825.680450] kobject: 'gpiochip10' (00000000e3becc55): kobject_cleanup, parent 0000000000000000
> [ 4825.680470] kobject: 'gpiochip10' (00000000e3becc55): calling ktype release
> [ 4825.680488] kobject: 'gpiochip10': free name
> [ 4825.716394] kobject: 'gpiochip9' (00000000bdaa0a47): kobject_cleanup, parent 0000000000000000
> [ 4825.716396] kobject: 'gpiochip9' (00000000bdaa0a47): calling ktype release
> [ 4825.716405] kobject: 'gpiochip9': free name
> [ 4825.968324] kobject: 'gpiochip17' (000000006ee06561): kobject_cleanup, parent 0000000000000000
> [ 4825.968331] kobject: 'gpiochip17' (000000006ee06561): calling ktype release
> [ 4825.968360] kobject: 'gpiochip17': free name
> [ 4825.968363] kobject: 'gpiochip14' (0000000092a079b6): kobject_cleanup, parent 0000000000000000
> [ 4825.968367] kobject: 'gpiochip14' (0000000092a079b6): calling ktype release
> [ 4825.968377] kobject: 'gpiochip14': free name
> [ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
> [ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
> [ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
> [ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
> [ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
> [ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> [ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
> [ 4833.908315] kobject: 'gpio-sim': free name
> [ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
> [ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
> [ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
> [ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
> [ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
> [ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
> [ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
> [ 4835.952448] kobject: 'gpio_sim': free name
>
>
> > > > > > Or maybe the chip->gc.parent should be changed to something else (actual GPIO
> > > > > > device, but then it's unclear how to provide the attributes in non-racy way
> > > > > Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
> > > > >
> > > > > I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> > > > > device expert nor would be appropriate to try the craft not earned ;-)
>
> With all of these additional debugging, cat /sys/kernel/debug/kmemleak
> showed nothing new.
>
> I believe this is reasonably safe.
>
> However, I was unsuccessful in seeing gpio trace, even with
> echo 1 > /sys/kernel/tracing/events/gpio/enable ... :-/
It's available in the trace buffer (you need to read a documentation to
understand how it works).
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-02-27 23:13 ` Andy Shevchenko
@ 2023-03-08 13:11 ` Mirsad Todorovac
2023-03-08 15:20 ` Andy Shevchenko
0 siblings, 1 reply; 23+ messages in thread
From: Mirsad Todorovac @ 2023-03-08 13:11 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 2/28/23 00:13, Andy Shevchenko wrote:
Hi, Andy!
>> The logs are extensive if you are willing to skim over them, but I believe the interesting
>> part is this:
>
> I'm not sure I understand where the problem is.
>
>> [ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
>> [ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
>> [ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
>> [ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
>> [ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
>> [ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
>> [ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
>> [ 4833.908315] kobject: 'gpio-sim': free name
>> [ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
>> [ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
>> [ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
>> [ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
>> [ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
>> [ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
>> [ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
>> [ 4835.952448] kobject: 'gpio_sim': free name
>>
>> Or, with CONFIG_DEBUG_DEVRES=y, it looks like this:
>
> I don't see that been enabled (it requires to pass a command line option to the kernel).
I don't think I have found this command line option to LK.
So far it seems that the kobject_release() was called for both /bus/platform/drivers/gpio-sim
and /module/gpio_sim . Is there soemthing I'm missing?
However, I've found one relatively unrelated failure to call kobject_release().
This happens during shutdown, after the syslog is shutdown, so I can only provide
a screenshot as a proof and for diagnostics:
https://domac.alu.hr/~mtodorov/linux/bugreports/integrity/20230308_123748.jpg
https://domac.alu.hr/~mtodorov/linux/bugreports/integrity/20230308_123752.jpg
I failed to locate the driver and responsible maintainers to the present moment.
It is happening on shutdown and it isn't that critical IMHO, except if it shows
some other problem in the code :-/
>>>>>>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>>>>>>> device, but then it's unclear how to provide the attributes in non-racy way
>>>>>> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>>>>>>
>>>>>> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
>>>>>> device expert nor would be appropriate to try the craft not earned ;-)
>>
>> With all of these additional debugging, cat /sys/kernel/debug/kmemleak
>> showed nothing new.
>>
>> I believe this is reasonably safe.
>>
>> However, I was unsuccessful in seeing gpio trace, even with
>> echo 1 > /sys/kernel/tracing/events/gpio/enable ... :-/
>
> It's available in the trace buffer (you need to read a documentation to
> understand how it works).
Still working on that, had other tasks to do ... So far I got to this:
1020 echo "1" > /sys/kernel/tracing/events/gpio/enable
1021 more /sys/kernel/tracing/trace
1022 cd ~marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio/
1023 ls
1024 ./gpio-sim.sh
1025 more /sys/kernel/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 9/9 #P:6
#
# _-----=> irqs-off/BH-disabled
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / _-=> migrate-disable
# |||| / delay
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
gpio-sim.sh-21157 [000] ..... 2705.538025: gpio_direction: 560 in (0)
gpio-mockup-cde-21471 [000] ..... 2705.579730: gpio_direction: 564 in (0)
gpio-mockup-cde-21471 [000] ..... 2705.579745: gpio_value: 564 get 1
gpio-mockup-cde-21478 [003] ..... 2705.589475: gpio_direction: 565 in (0)
gpio-mockup-cde-21478 [003] ..... 2705.589488: gpio_value: 565 get 0
gpio-mockup-cde-21561 [003] ..... 2705.721427: gpio_value: 589 set 1
gpio-mockup-cde-21561 [003] ..... 2705.721427: gpio_direction: 589 out (0)
gpio-mockup-cde-21595 [000] ..... 2705.855861: gpio_direction: 597 in (0)
gpio-mockup-cde-21595 [000] ..... 2705.855875: gpio_value: 597 get 1
[root@pc-mtodorov gpio]#
I hope I did this right. However, I have to play a bit with these results before
I could make any interpretation.
I just wanted to provide some feedback.
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-03-08 13:11 ` Mirsad Todorovac
@ 2023-03-08 15:20 ` Andy Shevchenko
2023-03-08 23:17 ` Mirsad Goran Todorovac
0 siblings, 1 reply; 23+ messages in thread
From: Andy Shevchenko @ 2023-03-08 15:20 UTC (permalink / raw)
To: Mirsad Todorovac
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On Wed, Mar 08, 2023 at 02:11:39PM +0100, Mirsad Todorovac wrote:
> On 2/28/23 00:13, Andy Shevchenko wrote:
...
> > > The logs are extensive if you are willing to skim over them, but I believe the interesting
> > > part is this:
> >
> > I'm not sure I understand where the problem is.
> >
> > > [ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
> > > [ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
> > > [ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
> > > [ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
> > > [ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
> > > [ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
> > > [ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
> > > [ 4833.908315] kobject: 'gpio-sim': free name
> > > [ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
> > > [ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
> > > [ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
> > > [ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
> > > [ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
> > > [ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
> > > [ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
> > > [ 4835.952448] kobject: 'gpio_sim': free name
> > >
> > > Or, with CONFIG_DEBUG_DEVRES=y, it looks like this:
> >
> > I don't see that been enabled (it requires to pass a command line option to the kernel).
>
> I don't think I have found this command line option to LK.
>
> So far it seems that the kobject_release() was called for both /bus/platform/drivers/gpio-sim
> and /module/gpio_sim . Is there soemthing I'm missing?
Have you read the code in drivers/base/devres.c?
https://elixir.bootlin.com/linux/v6.3-rc1/source/drivers/base/devres.c#L53
> However, I've found one relatively unrelated failure to call kobject_release().
> This happens during shutdown, after the syslog is shutdown, so I can only provide
> a screenshot as a proof and for diagnostics:
>
> https://domac.alu.hr/~mtodorov/linux/bugreports/integrity/20230308_123748.jpg
>
> https://domac.alu.hr/~mtodorov/linux/bugreports/integrity/20230308_123752.jpg
>
> I failed to locate the driver and responsible maintainers to the present moment.
> It is happening on shutdown and it isn't that critical IMHO, except if it shows
> some other problem in the code :-/
Congrats, you found a real issue somewhere. `git grep` usually helps
with this, like `git grep -n -w '"integrity"'` shows a few files, most
likely security/integrity/iint.c is the culprit.
> > > > > > > > Or maybe the chip->gc.parent should be changed to something else (actual GPIO
> > > > > > > > device, but then it's unclear how to provide the attributes in non-racy way
> > > > > > > Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
> > > > > > >
> > > > > > > I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
> > > > > > > device expert nor would be appropriate to try the craft not earned ;-)
> > >
> > > With all of these additional debugging, cat /sys/kernel/debug/kmemleak
> > > showed nothing new.
> > >
> > > I believe this is reasonably safe.
> > >
> > > However, I was unsuccessful in seeing gpio trace, even with
> > > echo 1 > /sys/kernel/tracing/events/gpio/enable ... :-/
> >
> > It's available in the trace buffer (you need to read a documentation to
> > understand how it works).
>
> Still working on that, had other tasks to do ... So far I got to this:
>
> 1020 echo "1" > /sys/kernel/tracing/events/gpio/enable
> 1021 more /sys/kernel/tracing/trace
> 1022 cd ~marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio/
> 1023 ls
> 1024 ./gpio-sim.sh
> 1025 more /sys/kernel/tracing/trace
> # tracer: nop
> #
> # entries-in-buffer/entries-written: 9/9 #P:6
> #
> # _-----=> irqs-off/BH-disabled
> # / _----=> need-resched
> # | / _---=> hardirq/softirq
> # || / _--=> preempt-depth
> # ||| / _-=> migrate-disable
> # |||| / delay
> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> # | | | ||||| | |
> gpio-sim.sh-21157 [000] ..... 2705.538025: gpio_direction: 560 in (0)
> gpio-mockup-cde-21471 [000] ..... 2705.579730: gpio_direction: 564 in (0)
> gpio-mockup-cde-21471 [000] ..... 2705.579745: gpio_value: 564 get 1
> gpio-mockup-cde-21478 [003] ..... 2705.589475: gpio_direction: 565 in (0)
> gpio-mockup-cde-21478 [003] ..... 2705.589488: gpio_value: 565 get 0
> gpio-mockup-cde-21561 [003] ..... 2705.721427: gpio_value: 589 set 1
> gpio-mockup-cde-21561 [003] ..... 2705.721427: gpio_direction: 589 out (0)
> gpio-mockup-cde-21595 [000] ..... 2705.855861: gpio_direction: 597 in (0)
> gpio-mockup-cde-21595 [000] ..... 2705.855875: gpio_value: 597 get 1
> I hope I did this right. However, I have to play a bit with these results before
> I could make any interpretation.
Yes. Just be sure you have all data dumped.
> I just wanted to provide some feedback.
Thanks.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: INFO: REPRODUCED: memory leak in gpio device in 6.2-rc6
2023-03-08 15:20 ` Andy Shevchenko
@ 2023-03-08 23:17 ` Mirsad Goran Todorovac
0 siblings, 0 replies; 23+ messages in thread
From: Mirsad Goran Todorovac @ 2023-03-08 23:17 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Bartosz Golaszewski, linux-gpio, Linus Walleij, linux-kernel,
Thorsten Leemhuis
On 08. 03. 2023. 16:20, Andy Shevchenko wrote:
> On Wed, Mar 08, 2023 at 02:11:39PM +0100, Mirsad Todorovac wrote:
>> On 2/28/23 00:13, Andy Shevchenko wrote:
>
> ...
>
>>>> The logs are extensive if you are willing to skim over them, but I believe the interesting
>>>> part is this:
>>>
>>> I'm not sure I understand where the problem is.
>>>
>>>> [ 4830.764748] kobject: 'gpio-sim' (000000005b8d0726): kobject_release, parent 000000007425b13f (delayed 750)
>>>> [ 4833.908238] kobject: 'gpio-sim' (000000005b8d0726): kobject_cleanup, parent 000000007425b13f
>>>> [ 4833.908244] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup kobject_del
>>>> [ 4833.908245] kobject: 'gpio-sim' (000000005b8d0726): auto cleanup 'remove' event
>>>> [ 4833.908247] kobject: 'gpio-sim' (000000005b8d0726): kobject_uevent_env
>>>> [ 4833.908273] kobject: 'gpio-sim' (000000005b8d0726): fill_kobj_path: path = '/bus/platform/drivers/gpio-sim'
>>>> [ 4833.908311] kobject: 'gpio-sim' (000000005b8d0726): calling ktype release
>>>> [ 4833.908315] kobject: 'gpio-sim': free name
>>>> [ 4834.932303] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_release, parent 0000000093357d30 (delayed 250)
>>>> [ 4835.952388] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_cleanup, parent 0000000093357d30
>>>> [ 4835.952413] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup kobject_del
>>>> [ 4835.952415] kobject: 'gpio_sim' (0000000096ea0bb1): auto cleanup 'remove' event
>>>> [ 4835.952416] kobject: 'gpio_sim' (0000000096ea0bb1): kobject_uevent_env
>>>> [ 4835.952424] kobject: 'gpio_sim' (0000000096ea0bb1): fill_kobj_path: path = '/module/gpio_sim'
>>>> [ 4835.952445] kobject: 'gpio_sim' (0000000096ea0bb1): calling ktype release
>>>> [ 4835.952448] kobject: 'gpio_sim': free name
>>>>
>>>> Or, with CONFIG_DEBUG_DEVRES=y, it looks like this:
>>>
>>> I don't see that been enabled (it requires to pass a command line option to the kernel).
>>
>> I don't think I have found this command line option to LK.
>>
>> So far it seems that the kobject_release() was called for both /bus/platform/drivers/gpio-sim
>> and /module/gpio_sim . Is there soemthing I'm missing?
>
> Have you read the code in drivers/base/devres.c?
>
> https://elixir.bootlin.com/linux/v6.3-rc1/source/drivers/base/devres.c#L53
Actually, it was in the manual all the time, but I have missed it somehow:
https://cateee.net/lkddb/web-lkddb/DEBUG_DEVRES.html
Here is the log (DEVRES debug was not in dmesg output, so after rsyslog dies,
there is no output to debug deallocs past that point):
https://domac.alu.hr/~mtodorov/linux/bugreports/gpio/devres_trace.log
Note that last 5 lines occur after `rmmod gpio-sim`:
Mar 8 23:23:55 pc-mtodorov kernel: kobject: 'gpio-sim' (00000000f0ead702): kobject_release, parent 00000000bb6cefbb (delayed 500)
Mar 8 23:23:55 pc-mtodorov kernel: kobject: 'drivers' (000000004fba12b9): kobject_release, parent 000000002c2cc344 (delayed 750)
Mar 8 23:23:55 pc-mtodorov kernel: kobject: 'holders' (00000000bfd7f234): kobject_release, parent 000000002c2cc344 (delayed 750)
Mar 8 23:23:55 pc-mtodorov kernel: kobject: 'notes' (00000000f2d17f82): kobject_release, parent 000000002c2cc344 (delayed 250)
Mar 8 23:23:59 pc-mtodorov kernel: kobject: 'gpio_sim' (000000002c2cc344): kobject_release, parent 00000000061436c2 (delayed 250)
>> However, I've found one relatively unrelated failure to call kobject_release().
>> This happens during shutdown, after the syslog is shutdown, so I can only provide
>> a screenshot as a proof and for diagnostics:
>>
>> https://domac.alu.hr/~mtodorov/linux/bugreports/integrity/20230308_123748.jpg
>>
>> https://domac.alu.hr/~mtodorov/linux/bugreports/integrity/20230308_123752.jpg
>>
>> I failed to locate the driver and responsible maintainers to the present moment.
>> It is happening on shutdown and it isn't that critical IMHO, except if it shows
>> some other problem in the code :-/
>
> Congrats, you found a real issue somewhere. `git grep` usually helps
> with this, like `git grep -n -w '"integrity"'` shows a few files, most
> likely security/integrity/iint.c is the culprit.
Looking more closely, this seems to be suspicious:
DEFINE_LSM(integrity) = {
.name = "integrity",
.init = integrity_iintcache_init,
};
.release member is missing.
However, struct lsm_info doesn't have the "release" member, so I wonder
what called release() in the first place?
https://elixir.bootlin.com/linux/v6.3-rc1/source/include/linux/lsm_hooks.h#L1733
struct lsm_info {
const char *name; /* Required. */
enum lsm_order order; /* Optional: default is LSM_ORDER_MUTABLE */
unsigned long flags; /* Optional: flags describing LSM */
int *enabled; /* Optional: controlled by CONFIG_LSM */
int (*init)(void); /* Required. */
struct lsm_blob_sizes *blobs; /* Optional: for blob sharing. */
};
extern struct lsm_info __start_lsm_info[], __end_lsm_info[];
extern struct lsm_info __start_early_lsm_info[], __end_early_lsm_info[];
#define DEFINE_LSM(lsm) \
static struct lsm_info __lsm_##lsm \
__used __section(".lsm_info.init") \
__aligned(sizeof(unsigned long))
But maybe the object is just deallocated and doesn't need kobject cleanup?
Actually it allocated "iint_cache".
Just a thought - if lsm_info had a release() member, it would be simple to
fix this:
static int __init integrity_iintcache_init(void)
{
iint_cache =
kmem_cache_create("iint_cache", sizeof(struct integrity_iint_cache),
0, SLAB_PANIC, init_once);
return 0;
}
+static void integrity_iintcache_destroy(void)
+{
+ kmem_cache_destroy(iint_cache);
+}
DEFINE_LSM(integrity) = {
.name = "integrity",
.init = integrity_iintcache_init,
+ .release = integrity_iintcache_destroy,
};
However, modifying include/linux/lsm_hooks.h is not for the faint of the heart,
I suppose.
But it seems too simple (too good to be true) that they just forgot to destroy
cache, maybe something is still alive when module is unloaded?
I still can't figure out what calls release() for "integrity" and where is the
"integrity" kobject allocated?
>>>>>>>>> Or maybe the chip->gc.parent should be changed to something else (actual GPIO
>>>>>>>>> device, but then it's unclear how to provide the attributes in non-racy way
>>>>>>>> Really, dunno. I have to repeat that my learning curve cannot adapt so quickly.
>>>>>>>>
>>>>>>>> I merely gave the report of KMEMLEAK, otherwise I am not a Linux kernel
>>>>>>>> device expert nor would be appropriate to try the craft not earned ;-)
>>>>
>>>> With all of these additional debugging, cat /sys/kernel/debug/kmemleak
>>>> showed nothing new.
>>>>
>>>> I believe this is reasonably safe.
>>>>
>>>> However, I was unsuccessful in seeing gpio trace, even with
>>>> echo 1 > /sys/kernel/tracing/events/gpio/enable ... :-/
>>>
>>> It's available in the trace buffer (you need to read a documentation to
>>> understand how it works).
>>
>> Still working on that, had other tasks to do ... So far I got to this:
>>
>> 1020 echo "1" > /sys/kernel/tracing/events/gpio/enable
>> 1021 more /sys/kernel/tracing/trace
>> 1022 cd ~marvin/linux/kernel/linux_torvalds/tools/testing/selftests/gpio/
>> 1023 ls
>> 1024 ./gpio-sim.sh
>> 1025 more /sys/kernel/tracing/trace
>> # tracer: nop
>> #
>> # entries-in-buffer/entries-written: 9/9 #P:6
>> #
>> # _-----=> irqs-off/BH-disabled
>> # / _----=> need-resched
>> # | / _---=> hardirq/softirq
>> # || / _--=> preempt-depth
>> # ||| / _-=> migrate-disable
>> # |||| / delay
>> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
>> # | | | ||||| | |
>> gpio-sim.sh-21157 [000] ..... 2705.538025: gpio_direction: 560 in (0)
>> gpio-mockup-cde-21471 [000] ..... 2705.579730: gpio_direction: 564 in (0)
>> gpio-mockup-cde-21471 [000] ..... 2705.579745: gpio_value: 564 get 1
>> gpio-mockup-cde-21478 [003] ..... 2705.589475: gpio_direction: 565 in (0)
>> gpio-mockup-cde-21478 [003] ..... 2705.589488: gpio_value: 565 get 0
>> gpio-mockup-cde-21561 [003] ..... 2705.721427: gpio_value: 589 set 1
>> gpio-mockup-cde-21561 [003] ..... 2705.721427: gpio_direction: 589 out (0)
>> gpio-mockup-cde-21595 [000] ..... 2705.855861: gpio_direction: 597 in (0)
>> gpio-mockup-cde-21595 [000] ..... 2705.855875: gpio_value: 597 get 1
>
>> I hope I did this right. However, I have to play a bit with these results before
>> I could make any interpretation.
>
> Yes. Just be sure you have all data dumped.
The trace from this run is:
[root@pc-mtodorov gpio]# more /sys/kernel/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 18/18 #P:6
#
# _-----=> irqs-off/BH-disabled
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / _-=> migrate-disable
# |||| / delay
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
gpio-sim.sh-4199 [004] ..... 1271.367102: gpio_direction: 560 in (0)
gpio-mockup-cde-4514 [004] ..... 1271.406523: gpio_direction: 564 in (0)
gpio-mockup-cde-4514 [004] ..... 1271.406537: gpio_value: 564 get 1
gpio-mockup-cde-4521 [004] ..... 1271.416032: gpio_direction: 565 in (0)
gpio-mockup-cde-4521 [004] ..... 1271.416045: gpio_value: 565 get 0
gpio-mockup-cde-4601 [000] ..... 1271.537872: gpio_value: 589 set 1
gpio-mockup-cde-4601 [000] ..... 1271.537873: gpio_direction: 589 out (0)
gpio-mockup-cde-4626 [002] ..... 1271.669199: gpio_direction: 597 in (0)
gpio-mockup-cde-4626 [002] ..... 1271.669212: gpio_value: 597 get 1
gpio-sim.sh-4742 [001] ..... 1472.168366: gpio_direction: 560 in (0)
gpio-mockup-cde-5055 [003] ..... 1472.207338: gpio_direction: 564 in (0)
gpio-mockup-cde-5055 [003] ..... 1472.207351: gpio_value: 564 get 1
gpio-mockup-cde-5062 [003] ..... 1472.216635: gpio_direction: 565 in (0)
gpio-mockup-cde-5062 [003] ..... 1472.216646: gpio_value: 565 get 0
gpio-mockup-cde-5142 [002] ..... 1472.338589: gpio_value: 589 set 1
gpio-mockup-cde-5142 [002] ..... 1472.338589: gpio_direction: 589 out (0)
gpio-mockup-cde-5167 [000] ..... 1472.469917: gpio_direction: 597 in (0)
gpio-mockup-cde-5167 [000] ..... 1472.469934: gpio_value: 597 get 1
[root@pc-mtodorov gpio]#
>> I just wanted to provide some feedback.
>
> Thanks.
Not at all, I am really having a great time while assisting your team. :-)
Regards,
Mirsad
--
Mirsad Goran Todorovac
Sistem inženjer
Grafički fakultet | Akademija likovnih umjetnosti
Sveučilište u Zagrebu
System engineer
Faculty of Graphic Arts | Academy of Fine Arts
University of Zagreb, Republic of Croatia
The European Union
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2023-03-08 23:17 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-31 9:36 INFO: memory leak in gpio device in 6.2-rc6 Mirsad Goran Todorovac
2023-02-08 19:55 ` INFO: REPRODUCED: " Mirsad Goran Todorovac
2023-02-12 14:19 ` Andy Shevchenko
2023-02-14 18:19 ` Mirsad Goran Todorovac
2023-02-15 10:53 ` Andy Shevchenko
2023-02-15 14:45 ` Mirsad Goran Todorovac
2023-02-16 14:16 ` Bartosz Golaszewski
2023-02-20 13:10 ` Mirsad Todorovac
2023-02-20 13:43 ` Andy Shevchenko
2023-02-21 13:52 ` Mirsad Goran Todorovac
2023-02-21 14:20 ` Mirsad Goran Todorovac
2023-02-21 14:32 ` Mirsad Goran Todorovac
2023-02-21 15:39 ` Andy Shevchenko
2023-02-22 10:53 ` Bartosz Golaszewski
2023-02-22 21:27 ` Mirsad Goran Todorovac
2023-02-24 15:12 ` Mirsad Todorovac
2023-02-24 17:40 ` Andy Shevchenko
2023-02-27 18:38 ` Mirsad Todorovac
2023-02-27 23:13 ` Andy Shevchenko
2023-03-08 13:11 ` Mirsad Todorovac
2023-03-08 15:20 ` Andy Shevchenko
2023-03-08 23:17 ` Mirsad Goran Todorovac
2023-02-14 20:54 ` INFO: BISECTED: " Mirsad Goran Todorovac
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).