All of lore.kernel.org
 help / color / mirror / Atom feed
From: Fox Chen <foxhlchen@gmail.com>
To: Ian Kent <raven@themaw.net>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Tejun Heo <tj@kernel.org>, Brice Goglin <brice.goglin@gmail.com>,
	Rick Lindsley <ricklind@linux.vnet.ibm.com>,
	Al Viro <viro@zeniv.linux.org.uk>,
	Miklos Szeredi <miklos@szeredi.hu>,
	David Howells <dhowells@redhat.com>,
	Eric Sandeen <sandeen@sandeen.net>,
	Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [PATCH v3 0/4] kernfs: proposed locking and concurrency improvement
Date: Mon, 19 Apr 2021 15:56:06 +0800	[thread overview]
Message-ID: <CAC2o3DKNc=sL2n8291Dpiyb0bRHaX=nd33ogvO_LkJqpBj-YmA@mail.gmail.com> (raw)
In-Reply-To: <161793058309.10062.17056551235139961080.stgit@mickey.themaw.net>

[-- Attachment #1: Type: text/plain, Size: 3214 bytes --]

On Fri, Apr 9, 2021 at 9:14 AM Ian Kent <raven@themaw.net> wrote:
>
> There have been a few instances of contention on the kernfs_mutex during
> path walks, a case on very large IBM systems seen by myself, a report by
> Brice Goglin and followed up by Fox Chen, and I've since seen a couple
> of other reports by CoreOS users.
>
> The common thread is a large number of kernfs path walks leading to
> slowness of path walks due to kernfs_mutex contention.
>
> The problem being that changes to the VFS over some time have increased
> it's concurrency capabilities to an extent that kernfs's use of a mutex
> is no longer appropriate. There's also an issue of walks for non-existent
> paths causing contention if there are quite a few of them which is a less
> common problem.
>
> This patch series is relatively straight forward.
>
> All it does is add the ability to take advantage of VFS negative dentry
> caching to avoid needless dentry alloc/free cycles for lookups of paths
> that don't exit and change the kernfs_mutex to a read/write semaphore.
>
> The patch that tried to stay in VFS rcu-walk mode during path walks has
> been dropped for two reasons. First, it doesn't actually give very much
> improvement and, second, if there's a place where mistakes could go
> unnoticed it would be in that path. This makes the patch series simpler
> to review and reduces the likelihood of problems going unnoticed and
> popping up later.
>
> The patch to use a revision to identify if a directory has changed has
> also been dropped. If the directory has changed the dentry revision
> needs to be updated to avoid subsequent rb tree searches and after
> changing to use a read/write semaphore the update also requires a lock.
> But the d_lock is the only lock available at this point which might
> itself be contended.
>
> Changes since v2:
> - actually fix the inode attribute update locking.
> - drop the patch that tried to stay in rcu-walk mode.
> - drop the use a revision to identify if a directory has changed patch.
>
> Changes since v1:
> - fix locking in .permission() and .getattr() by re-factoring the attribute
>   handling code.
>
> ---
>
> Ian Kent (4):
>       kernfs: move revalidate to be near lookup
>       kernfs: use VFS negative dentry caching
>       kernfs: switch kernfs to use an rwsem
>       kernfs: use i_lock to protect concurrent inode updates
>
>
>  fs/kernfs/dir.c             |  240 +++++++++++++++++++++++--------------------
>  fs/kernfs/file.c            |    4 -
>  fs/kernfs/inode.c           |   18 ++-
>  fs/kernfs/kernfs-internal.h |    5 +
>  fs/kernfs/mount.c           |   12 +-
>  fs/kernfs/symlink.c         |    4 -
>  include/linux/kernfs.h      |    2
>  7 files changed, 155 insertions(+), 130 deletions(-)
>
> --
>

Hi Ian,

I tested this patchset with my
benchmark(https://github.com/foxhlchen/sysfs_benchmark) on a 96 CPUs
(aws c5) machine.

The result was promising:
Before, one open+read+close cycle took 500us without much variation.
With this patch, the fastest one only takes 30us, though the slowest
is still around 100us(due to the spinlock). perf report shows no more
significant mutex contention.

FYR, I put outputs in the attachment.


thanks,
fox

[-- Attachment #2: result.after --]
[-- Type: application/octet-stream, Size: 4927 bytes --]

single: total 3.006701ms per 3.006701us
concur: total 32.057796ms per 32.057796us  CPU 51
concur: total 32.492103ms per 32.492103us  CPU 3
concur: total 32.737743ms per 32.737743us  CPU 7
concur: total 33.087308ms per 33.087308us  CPU 2
concur: total 33.271875ms per 33.271875us  CPU 49
concur: total 33.346198ms per 33.346198us  CPU 50
concur: total 33.483394ms per 33.483394us  CPU 55
concur: total 33.575117ms per 33.575117us  CPU 1
concur: total 33.873799ms per 33.873799us  CPU 6
concur: total 34.056747ms per 34.056747us  CPU 5
concur: total 34.139490ms per 34.139490us  CPU 54
concur: total 34.317747ms per 34.317747us  CPU 53
concur: total 35.430002ms per 35.430002us  CPU 48
concur: total 35.467414ms per 35.467414us  CPU 0
concur: total 36.634091ms per 36.634091us  CPU 4
concur: total 36.803717ms per 36.803717us  CPU 52
concur: total 36.959244ms per 36.959244us  CPU 11
concur: total 37.446623ms per 37.446623us  CPU 59
concur: total 37.607590ms per 37.607590us  CPU 58
concur: total 37.916612ms per 37.916612us  CPU 10
concur: total 38.044637ms per 38.044637us  CPU 9
concur: total 38.379601ms per 38.379601us  CPU 57
concur: total 38.582989ms per 38.582989us  CPU 15
concur: total 39.408612ms per 39.408612us  CPU 62
concur: total 40.001796ms per 40.001796us  CPU 63
concur: total 40.123991ms per 40.123991us  CPU 14
concur: total 41.816000ms per 41.816000us  CPU 13
concur: total 43.134413ms per 43.134413us  CPU 8
concur: total 43.167374ms per 43.167374us  CPU 61
concur: total 43.673585ms per 43.673585us  CPU 56
concur: total 45.289910ms per 45.289910us  CPU 12
concur: total 45.342382ms per 45.342382us  CPU 60
concur: total 64.314757ms per 64.314757us  CPU 67
concur: total 64.571128ms per 64.571128us  CPU 23
concur: total 64.650921ms per 64.650921us  CPU 71
concur: total 64.734934ms per 64.734934us  CPU 19
concur: total 64.863810ms per 64.863810us  CPU 17
concur: total 64.916032ms per 64.916032us  CPU 66
concur: total 64.988355ms per 64.988355us  CPU 21
concur: total 65.004955ms per 65.004955us  CPU 65
concur: total 65.134421ms per 65.134421us  CPU 18
concur: total 65.293527ms per 65.293527us  CPU 70
concur: total 65.410172ms per 65.410172us  CPU 22
concur: total 65.494995ms per 65.494995us  CPU 69
concur: total 65.597959ms per 65.597959us  CPU 64
concur: total 65.891331ms per 65.891331us  CPU 20
concur: total 66.150142ms per 66.150142us  CPU 68
concur: total 66.263326ms per 66.263326us  CPU 16
concur: total 80.115175ms per 80.115175us  CPU 27
concur: total 80.402707ms per 80.402707us  CPU 31
concur: total 80.565403ms per 80.565403us  CPU 75
concur: total 80.757601ms per 80.757601us  CPU 79
concur: total 80.988801ms per 80.988801us  CPU 73
concur: total 81.060023ms per 81.060023us  CPU 78
concur: total 81.169068ms per 81.169068us  CPU 25
concur: total 81.525702ms per 81.525702us  CPU 30
concur: total 81.678108ms per 81.678108us  CPU 26
concur: total 81.737981ms per 81.737981us  CPU 29
concur: total 82.072014ms per 82.072014us  CPU 77
concur: total 82.073764ms per 82.073764us  CPU 74
concur: total 82.673568ms per 82.673568us  CPU 72concur: total 82.687729ms per 82.687729us  CPU 76

concur: total 83.002492ms per 83.002492us  CPU 24
concur: total 83.163668ms per 83.163668us  CPU 28
concur: total 102.776790ms per 102.776790us  CPU 87
concur: total 102.857793ms per 102.857793us  CPU 35
concur: total 103.023630ms per 103.023630us  CPU 39
concur: total 103.143735ms per 103.143735us  CPU 43
concur: total 103.251629ms per 103.251629us  CPU 83
concur: total 103.304701ms per 103.304701us  CPU 33
concur: total 103.393765ms per 103.393765us  CPU 91
concur: total 103.416055ms per 103.416055us concur: total 103.419316ms per 103.419316us concur: total 103.429036ms per 103.429036us concur: total 103.424245ms per 103.424245us  CPU 86 CPU 95
 CPU 34

 CPU 85
concur: total 103.465647ms per 103.465647us  CPU 37
concur: total 103.556421ms per 103.556421us  CPU 81
concur: total 103.600983ms per 103.600983us  CPU 89concur: total 103.608003ms per 103.608003us  CPU 82concur: total 103.622134ms per 103.622134us  CPU 41


concur: total 103.659765ms per 103.659765us  CPU 38
concur: total 103.759289ms per 103.759289us  CPU 47
concur: total 103.795671ms per 103.795671us  CPU 90
concur: total 103.835662ms per 103.835662us  CPU 45
concur: total 103.874204ms per 103.874204us  CPU 42
concur: total 103.921736ms per 103.921736us  CPU 36
concur: total 104.031040ms per 104.031040us  CPU 94
concur: total 104.070472ms per 104.070472us  CPU 80
concur: total 104.078142ms per 104.078142us  CPU 93
concur: total 104.124384ms per 104.124384us  CPU 32
concur: total 104.160405ms per 104.160405us  CPU 46
concur: total 104.185506ms per 104.185506us  CPU 40
concur: total 104.207247ms per 104.207247us  CPU 92
concur: total 104.246149ms per 104.246149us  CPU 84
concur: total 104.293941ms per 104.293941us  CPU 88
concur: total 104.314402ms per 104.314402us  CPU 44
times: 1000 threads: 96 cpus: 96

[-- Attachment #3: result.before --]
[-- Type: application/octet-stream, Size: 5055 bytes --]

single: total 2.884608ms per 2.884608us
concur: total 493.478294ms per 493.478294us  CPU 54
concur: total 494.759468ms per 494.759468us  CPU 16
concur: total 495.123051ms per 495.123051us  CPU 17
concur: total 495.336105ms per 495.336105us  CPU 64
concur: total 495.430841ms per 495.430841us  CPU 14
concur: total 495.448193ms per 495.448193us  CPU 70
concur: total 495.468434ms per 495.468434us  CPU 67
concur: total 496.068303ms per 496.068303us  CPU 43concur: total 496.073834ms per 496.073834us  CPU 83

concur: total 496.214473ms per 496.214473us  CPU 22
concur: total 496.388984ms per 496.388984us  CPU 84
concur: total 496.615719ms per 496.615719us  CPU 38
concur: total 496.662582ms per 496.662582us  CPU 5
concur: total 496.954701ms per 496.954701us  CPU 69
concur: total 496.963772ms per 496.963772us  CPU 66
concur: total 497.130163ms per 497.130163us  CPU 35
concur: total 497.164755ms per 497.164755us  CPU 18concur: total 497.170125ms per 497.170125us  CPU 74

concur: total 497.243370ms per 497.243370us  CPU 19
concur: total 497.348467ms per 497.348467us  CPU 85
concur: total 497.616995ms per 497.616995us  CPU 79
concur: total 498.182402ms per 498.182402us  CPU 15
concur: total 498.346592ms per 498.346592us  CPU 78
concur: total 498.560636ms per 498.560636us  CPU 71
concur: total 498.793862ms per 498.793862us  CPU 36
concur: total 498.795112ms per 498.795112us  CPU 10
concur: total 498.836754ms per 498.836754us  CPU 94
concur: total 499.016226ms per 499.016226us  CPU 80
concur: total 499.117983ms per 499.117983us  CPU 24
concur: total 499.402951ms per 499.402951us  CPU 2
concur: total 499.489227ms per 499.489227us  CPU 32
concur: total 499.509828ms per 499.509828us  CPU 81
concur: total 499.574623ms per 499.574623us concur: total 499.577163ms per 499.577163us  CPU 37
concur: total 499.644097ms per 499.644097us  CPU 91
 CPU 63
concur: total 499.729263ms per 499.729263us  CPU 30
concur: total 499.760315ms per 499.760315us  CPU 65
concur: total 499.829729ms per 499.829729us  CPU 8
concur: total 499.848731ms per 499.848731us  CPU 21
concur: total 499.723043ms per 499.723043us  CPU 55
concur: total 499.940567ms per 499.940567us  CPU 62
concur: total 499.976419ms per 499.976419us  CPU 34
concur: total 500.024402ms per 500.024402us  CPU 53
concur: total 500.088066ms per 500.088066us  CPU 44
concur: total 500.104787ms per 500.104787us  CPU 11
concur: total 500.132569ms per 500.132569us  CPU 27
concur: total 500.175832ms per 500.175832us  CPU 86
concur: total 500.209424ms per 500.209424us  CPU 46
concur: total 500.224665ms per 500.224665us  CPU 73
concur: total 500.227475ms per 500.227475us  CPU 50
concur: total 500.282449ms per 500.282449us  CPU 23
concur: total 500.292870ms per 500.292870us  CPU 41
concur: total 500.370375ms per 500.370375us  CPU 6
concur: total 500.540066ms per 500.540066us  CPU 33
concur: total 500.597190ms per 500.597190us  CPU 61
concur: total 500.657503ms per 500.657503us  CPU 90
concur: total 500.727758ms per 500.727758us  CPU 92
concur: total 500.767171ms per 500.767171us  CPU 3
concur: total 500.782442ms per 500.782442us  CPU 56
concur: total 500.848026ms per 500.848026us  CPU 47concur: total 500.853006ms per 500.853006us concur: total 500.861377ms per 500.861377us  CPU 52 CPU 39


concur: total 500.883958ms per 500.883958us  CPU 26
concur: total 500.948133ms per 500.948133us  CPU 89
concur: total 500.954853ms per 500.954853us  CPU 88
concur: total 501.007586ms per 501.007586us  CPU 76
concur: total 501.106263ms per 501.106263us  CPU 28
concur: total 501.153606ms per 501.153606us  CPU 75
concur: total 501.174267ms per 501.174267us  CPU 58
concur: total 501.187538ms per 501.187538us  CPU 51
concur: total 501.249452ms per 501.249452us  CPU 72
concur: total 501.258213ms per 501.258213us  CPU 40
concur: total 501.340298ms per 501.340298us  CPU 82
concur: total 501.346649ms per 501.346649us  CPU 12
concur: total 501.371100ms per 501.371100us  CPU 9
concur: total 501.405882ms per 501.405882us  CPU 4
concur: total 501.449815ms per 501.449815us  CPU 87
concur: total 501.468097ms per 501.468097us  CPU 20concur: total 501.464897ms per 501.464897us  CPU 95

concur: total 501.525250ms per 501.525250us  CPU 45
concur: total 501.482868ms per 501.482868us  CPU 25
concur: total 501.641478ms per 501.641478us  CPU 42
concur: total 501.646838ms per 501.646838us  CPU 68
concur: total 501.702382ms per 501.702382us  CPU 59
concur: total 501.726964ms per 501.726964us  CPU 31
concur: total 501.742874ms per 501.742874us  CPU 7concur: total 501.728604ms per 501.728604us  CPU 29

concur: total 501.780907ms per 501.780907us  CPU 0
concur: total 501.818790ms per 501.818790us  CPU 48
concur: total 501.827900ms per 501.827900us  CPU 60
concur: total 501.849432ms per 501.849432us  CPU 57
concur: total 501.862902ms per 501.862902us  CPU 93
concur: total 501.879784ms per 501.879784us  CPU 13
concur: total 501.938867ms per 501.938867us  CPU 77
concur: total 502.001791ms per 502.001791us  CPU 1
concur: total 502.009782ms per 502.009782us  CPU 49
times: 1000 threads: 96 cpus: 96

[-- Attachment #4: perf_report --]
[-- Type: application/octet-stream, Size: 230766 bytes --]

# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 2K of event 'cycles'
# Event count (approx.): 513478020
#
# Children      Self  Command          Shared Object       Symbol                                        
# ........  ........  ...............  ..................  ..............................................
#
    91.01%     0.20%  bin_sysbm        [kernel.kallsyms]   [k] entry_SYSCALL_64_after_hwframe
            |          
             --90.80%--entry_SYSCALL_64_after_hwframe
                       |          
                        --90.66%--do_syscall_64
                                  |          
                                  |--56.89%--__x64_sys_openat
                                  |          |          
                                  |           --56.82%--do_sys_open
                                  |                     do_sys_openat2
                                  |                     |          
                                  |                     |--22.24%--fd_install
                                  |                     |          |          
                                  |                     |           --22.08%--_raw_spin_lock
                                  |                     |                     native_queued_spin_lock_slowpath
                                  |                     |          
                                  |                     |--18.23%--do_filp_open
                                  |                     |          path_openat
                                  |                     |          |          
                                  |                     |          |--8.39%--link_path_walk.part.0
                                  |                     |          |          |          
                                  |                     |          |          |--4.47%--inode_permission
                                  |                     |          |          |          |          
                                  |                     |          |          |          |--2.84%--kernfs_iop_permission
                                  |                     |          |          |          |          |          
                                  |                     |          |          |          |          |--1.19%--down_read
                                  |                     |          |          |          |          |          
                                  |                     |          |          |          |          |--0.68%--generic_permission
                                  |                     |          |          |          |          |          
                                  |                     |          |          |          |           --0.66%--kernfs_refresh_inode
                                  |                     |          |          |          |                     |          
                                  |                     |          |          |          |                      --0.58%--_raw_spin_lock
                                  |                     |          |          |          |                                native_queued_spin_lock_slowpath
                                  |                     |          |          |          |          
                                  |                     |          |          |           --0.57%--up_read
                                  |                     |          |          |          
                                  |                     |          |           --3.49%--walk_component
                                  |                     |          |                     |          
                                  |                     |          |                      --3.04%--lookup_fast
                                  |                     |          |                                |          
                                  |                     |          |                                |--1.02%--kernfs_dop_revalidate
                                  |                     |          |                                |          |          
                                  |                     |          |                                |           --0.94%--down_read
                                  |                     |          |                                |          
                                  |                     |          |                                |--0.99%--__d_lookup
                                  |                     |          |                                |          _raw_spin_lock
                                  |                     |          |                                |          native_queued_spin_lock_slowpath
                                  |                     |          |                                |          
                                  |                     |          |                                 --0.59%--_raw_spin_lock
                                  |                     |          |          
                                  |                     |          |--4.18%--vfs_open
                                  |                     |          |          do_dentry_open
                                  |                     |          |          |          
                                  |                     |          |           --3.21%--kernfs_fop_open
                                  |                     |          |                     |          
                                  |                     |          |                      --2.50%--mutex_lock
                                  |                     |          |                                |          
                                  |                     |          |                                 --2.18%--__mutex_lock_slowpath
                                  |                     |          |                                           |          
                                  |                     |          |                                            --2.10%--__mutex_lock.isra.0
                                  |                     |          |                                                      |          
                                  |                     |          |                                                       --1.78%--osq_lock
                                  |                     |          |          
                                  |                     |          |--1.88%--alloc_empty_file
                                  |                     |          |          __alloc_file
                                  |                     |          |          |          
                                  |                     |          |           --1.09%--kmem_cache_alloc
                                  |                     |          |          
                                  |                     |          |--1.35%--may_open
                                  |                     |          |          |          
                                  |                     |          |           --1.27%--inode_permission
                                  |                     |          |                     |          
                                  |                     |          |                      --1.05%--kernfs_iop_permission
                                  |                     |          |                                |          
                                  |                     |          |                                 --0.64%--kernfs_refresh_inode
                                  |                     |          |                                           |          
                                  |                     |          |                                            --0.56%--_raw_spin_lock
                                  |                     |          |                                                      native_queued_spin_lock_slowpath
                                  |                     |          |          
                                  |                     |           --1.20%--lookup_fast
                                  |                     |                     |          
                                  |                     |                      --0.70%--__d_lookup
                                  |                     |                                |          
                                  |                     |                                 --0.65%--_raw_spin_lock
                                  |                     |                                           native_queued_spin_lock_slowpath
                                  |                     |          
                                  |                      --15.25%--get_unused_fd_flags
                                  |                                |          
                                  |                                 --14.76%--alloc_fd
                                  |                                           |          
                                  |                                            --13.95%--_raw_spin_lock
                                  |                                                      native_queued_spin_lock_slowpath
                                  |          
                                  |--20.63%--__x64_sys_close
                                  |          close_fd
                                  |          |          
                                  |          |--19.93%--pick_file
                                  |          |          |          
                                  |          |           --19.80%--_raw_spin_lock
                                  |          |                     native_queued_spin_lock_slowpath
                                  |          |          
                                  |           --0.62%--_raw_spin_lock
                                  |          
                                  |--9.22%--syscall_exit_to_user_mode
                                  |          |          
                                  |           --9.13%--exit_to_user_mode_prepare
                                  |                     |          
                                  |                      --8.98%--task_work_run
                                  |                                |          
                                  |                                 --8.93%--____fput
                                  |                                           |          
                                  |                                            --8.85%--__fput
                                  |                                                      |          
                                  |                                                      |--7.13%--kernfs_fop_release
                                  |                                                      |          |          
                                  |                                                      |          |--5.46%--kernfs_put_open_node.isra.0
                                  |                                                      |          |          |          
                                  |                                                      |          |           --5.19%--mutex_lock
                                  |                                                      |          |                     |          
                                  |                                                      |          |                      --5.01%--__mutex_lock_slowpath
                                  |                                                      |          |                                |          
                                  |                                                      |          |                                 --4.88%--__mutex_lock.isra.0
                                  |                                                      |          |                                           |          
                                  |                                                      |          |                                            --4.57%--osq_lock
                                  |                                                      |          |          
                                  |                                                      |           --1.36%--seq_release
                                  |                                                      |                     |          
                                  |                                                      |                      --1.10%--kvfree
                                  |                                                      |                                kfree
                                  |                                                      |                                |          
                                  |                                                      |                                 --1.02%--obj_cgroup_uncharge
                                  |                                                      |                                           refill_obj_stock
                                  |                                                      |                                           |          
                                  |                                                      |                                            --0.67%--drain_obj_stock.isra.0
                                  |                                                      |                                                      |          
                                  |                                                      |                                                       --0.51%--__memcg_kmem_uncharge
                                  |                                                      |          
                                  |                                                       --0.82%--dput
                                  |                                                                 _raw_spin_lock
                                  |                                                                 native_queued_spin_lock_slowpath
                                  |          
                                   --3.78%--__x64_sys_read
                                             ksys_read
                                             |          
                                              --3.62%--vfs_read
                                                        |          
                                                         --3.15%--new_sync_read
                                                                   |          
                                                                    --3.02%--kernfs_fop_read_iter
                                                                              |          
                                                                               --2.87%--seq_read_iter
                                                                                         |          
                                                                                          --1.74%--kvmalloc_node
                                                                                                    |          
                                                                                                     --1.59%--__kmalloc_node
                                                                                                               |          
                                                                                                                --1.32%--obj_cgroup_charge
                                                                                                                          |          
                                                                                                                           --0.83%--refill_obj_stock
                                                                                                                                     drain_obj_stock.isra.0

    90.66%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] do_syscall_64
            |
            ---do_syscall_64
               |          
               |--56.89%--__x64_sys_openat
               |          |          
               |           --56.82%--do_sys_open
               |                     do_sys_openat2
               |                     |          
               |                     |--22.24%--fd_install
               |                     |          |          
               |                     |           --22.08%--_raw_spin_lock
               |                     |                     native_queued_spin_lock_slowpath
               |                     |          
               |                     |--18.23%--do_filp_open
               |                     |          path_openat
               |                     |          |          
               |                     |          |--8.39%--link_path_walk.part.0
               |                     |          |          |          
               |                     |          |          |--4.47%--inode_permission
               |                     |          |          |          |          
               |                     |          |          |          |--2.84%--kernfs_iop_permission
               |                     |          |          |          |          |          
               |                     |          |          |          |          |--1.19%--down_read
               |                     |          |          |          |          |          
               |                     |          |          |          |          |--0.68%--generic_permission
               |                     |          |          |          |          |          
               |                     |          |          |          |           --0.66%--kernfs_refresh_inode
               |                     |          |          |          |                     |          
               |                     |          |          |          |                      --0.58%--_raw_spin_lock
               |                     |          |          |          |                                native_queued_spin_lock_slowpath
               |                     |          |          |          |          
               |                     |          |          |           --0.57%--up_read
               |                     |          |          |          
               |                     |          |           --3.49%--walk_component
               |                     |          |                     |          
               |                     |          |                      --3.04%--lookup_fast
               |                     |          |                                |          
               |                     |          |                                |--1.02%--kernfs_dop_revalidate
               |                     |          |                                |          |          
               |                     |          |                                |           --0.94%--down_read
               |                     |          |                                |          
               |                     |          |                                |--0.99%--__d_lookup
               |                     |          |                                |          _raw_spin_lock
               |                     |          |                                |          native_queued_spin_lock_slowpath
               |                     |          |                                |          
               |                     |          |                                 --0.59%--_raw_spin_lock
               |                     |          |          
               |                     |          |--4.18%--vfs_open
               |                     |          |          do_dentry_open
               |                     |          |          |          
               |                     |          |           --3.21%--kernfs_fop_open
               |                     |          |                     |          
               |                     |          |                      --2.50%--mutex_lock
               |                     |          |                                |          
               |                     |          |                                 --2.18%--__mutex_lock_slowpath
               |                     |          |                                           |          
               |                     |          |                                            --2.10%--__mutex_lock.isra.0
               |                     |          |                                                      |          
               |                     |          |                                                       --1.78%--osq_lock
               |                     |          |          
               |                     |          |--1.88%--alloc_empty_file
               |                     |          |          __alloc_file
               |                     |          |          |          
               |                     |          |           --1.09%--kmem_cache_alloc
               |                     |          |          
               |                     |          |--1.35%--may_open
               |                     |          |          |          
               |                     |          |           --1.27%--inode_permission
               |                     |          |                     |          
               |                     |          |                      --1.05%--kernfs_iop_permission
               |                     |          |                                |          
               |                     |          |                                 --0.64%--kernfs_refresh_inode
               |                     |          |                                           |          
               |                     |          |                                            --0.56%--_raw_spin_lock
               |                     |          |                                                      native_queued_spin_lock_slowpath
               |                     |          |          
               |                     |           --1.20%--lookup_fast
               |                     |                     |          
               |                     |                      --0.70%--__d_lookup
               |                     |                                |          
               |                     |                                 --0.65%--_raw_spin_lock
               |                     |                                           native_queued_spin_lock_slowpath
               |                     |          
               |                      --15.25%--get_unused_fd_flags
               |                                |          
               |                                 --14.76%--alloc_fd
               |                                           |          
               |                                            --13.95%--_raw_spin_lock
               |                                                      native_queued_spin_lock_slowpath
               |          
               |--20.63%--__x64_sys_close
               |          close_fd
               |          |          
               |          |--19.93%--pick_file
               |          |          |          
               |          |           --19.80%--_raw_spin_lock
               |          |                     native_queued_spin_lock_slowpath
               |          |          
               |           --0.62%--_raw_spin_lock
               |          
               |--9.22%--syscall_exit_to_user_mode
               |          |          
               |           --9.13%--exit_to_user_mode_prepare
               |                     |          
               |                      --8.98%--task_work_run
               |                                |          
               |                                 --8.93%--____fput
               |                                           |          
               |                                            --8.85%--__fput
               |                                                      |          
               |                                                      |--7.13%--kernfs_fop_release
               |                                                      |          |          
               |                                                      |          |--5.46%--kernfs_put_open_node.isra.0
               |                                                      |          |          |          
               |                                                      |          |           --5.19%--mutex_lock
               |                                                      |          |                     |          
               |                                                      |          |                      --5.01%--__mutex_lock_slowpath
               |                                                      |          |                                |          
               |                                                      |          |                                 --4.88%--__mutex_lock.isra.0
               |                                                      |          |                                           |          
               |                                                      |          |                                            --4.57%--osq_lock
               |                                                      |          |          
               |                                                      |           --1.36%--seq_release
               |                                                      |                     |          
               |                                                      |                      --1.10%--kvfree
               |                                                      |                                kfree
               |                                                      |                                |          
               |                                                      |                                 --1.02%--obj_cgroup_uncharge
               |                                                      |                                           refill_obj_stock
               |                                                      |                                           |          
               |                                                      |                                            --0.67%--drain_obj_stock.isra.0
               |                                                      |                                                      |          
               |                                                      |                                                       --0.51%--__memcg_kmem_uncharge
               |                                                      |          
               |                                                       --0.82%--dput
               |                                                                 _raw_spin_lock
               |                                                                 native_queued_spin_lock_slowpath
               |          
                --3.78%--__x64_sys_read
                          ksys_read
                          |          
                           --3.62%--vfs_read
                                     |          
                                      --3.15%--new_sync_read
                                                |          
                                                 --3.02%--kernfs_fop_read_iter
                                                           |          
                                                            --2.87%--seq_read_iter
                                                                      |          
                                                                       --1.74%--kvmalloc_node
                                                                                 |          
                                                                                  --1.59%--__kmalloc_node
                                                                                            |          
                                                                                             --1.32%--obj_cgroup_charge
                                                                                                       |          
                                                                                                        --0.83%--refill_obj_stock
                                                                                                                  drain_obj_stock.isra.0

    61.98%     2.26%  bin_sysbm        [kernel.kallsyms]   [k] _raw_spin_lock
            |          
            |--59.71%--_raw_spin_lock
            |          native_queued_spin_lock_slowpath
            |          
            |--1.64%--0x7379732f73656369
            |          0x7f88520d7ad4
            |          entry_SYSCALL_64_after_hwframe
            |          do_syscall_64
            |          __x64_sys_openat
            |          do_sys_open
            |          do_sys_openat2
            |          |          
            |           --1.03%--do_filp_open
            |                     path_openat
            |                     |          
            |                      --0.81%--link_path_walk.part.0
            |                                |          
            |                                 --0.59%--walk_component
            |                                           lookup_fast
            |                                           _raw_spin_lock
            |          
             --0.62%--0x7f88520cc609
                       0x56303487c721
                       0x7f88520d73fb
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       __x64_sys_close
                       close_fd
                       _raw_spin_lock

    59.80%    59.80%  bin_sysbm        [kernel.kallsyms]   [k] native_queued_spin_lock_slowpath
            |          
            |--39.17%--0x7379732f73656369
            |          0x7f88520d7ad4
            |          entry_SYSCALL_64_after_hwframe
            |          do_syscall_64
            |          __x64_sys_openat
            |          do_sys_open
            |          do_sys_openat2
            |          |          
            |          |--22.08%--fd_install
            |          |          _raw_spin_lock
            |          |          native_queued_spin_lock_slowpath
            |          |          
            |          |--14.03%--get_unused_fd_flags
            |          |          alloc_fd
            |          |          |          
            |          |           --13.95%--_raw_spin_lock
            |          |                     native_queued_spin_lock_slowpath
            |          |          
            |           --3.07%--do_filp_open
            |                     path_openat
            |                     |          
            |                     |--1.65%--link_path_walk.part.0
            |                     |          |          
            |                     |          |--1.07%--walk_component
            |                     |          |          |          
            |                     |          |           --0.99%--lookup_fast
            |                     |          |                     __d_lookup
            |                     |          |                     _raw_spin_lock
            |                     |          |                     native_queued_spin_lock_slowpath
            |                     |          |          
            |                     |           --0.58%--inode_permission
            |                     |                     kernfs_iop_permission
            |                     |                     kernfs_refresh_inode
            |                     |                     _raw_spin_lock
            |                     |                     native_queued_spin_lock_slowpath
            |                     |          
            |                     |--0.65%--lookup_fast
            |                     |          __d_lookup
            |                     |          _raw_spin_lock
            |                     |          native_queued_spin_lock_slowpath
            |                     |          
            |                      --0.56%--may_open
            |                                inode_permission
            |                                kernfs_iop_permission
            |                                kernfs_refresh_inode
            |                                _raw_spin_lock
            |                                native_queued_spin_lock_slowpath
            |          
             --20.62%--0x7f88520cc609
                       0x56303487c721
                       0x7f88520d73fb
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       |          
                       |--19.80%--__x64_sys_close
                       |          close_fd
                       |          pick_file
                       |          _raw_spin_lock
                       |          native_queued_spin_lock_slowpath
                       |          
                        --0.82%--syscall_exit_to_user_mode
                                  exit_to_user_mode_prepare
                                  task_work_run
                                  ____fput
                                  __fput
                                  dput
                                  _raw_spin_lock
                                  native_queued_spin_lock_slowpath

    57.29%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520d7ad4
            |
            ---0x7f88520d7ad4
               |          
                --57.09%--entry_SYSCALL_64_after_hwframe
                          |          
                           --56.97%--do_syscall_64
                                     |          
                                      --56.89%--__x64_sys_openat
                                                |          
                                                 --56.82%--do_sys_open
                                                           do_sys_openat2
                                                           |          
                                                           |--22.24%--fd_install
                                                           |          |          
                                                           |           --22.08%--_raw_spin_lock
                                                           |                     native_queued_spin_lock_slowpath
                                                           |          
                                                           |--18.23%--do_filp_open
                                                           |          path_openat
                                                           |          |          
                                                           |          |--8.39%--link_path_walk.part.0
                                                           |          |          |          
                                                           |          |          |--4.47%--inode_permission
                                                           |          |          |          |          
                                                           |          |          |          |--2.84%--kernfs_iop_permission
                                                           |          |          |          |          |          
                                                           |          |          |          |          |--1.19%--down_read
                                                           |          |          |          |          |          
                                                           |          |          |          |          |--0.68%--generic_permission
                                                           |          |          |          |          |          
                                                           |          |          |          |           --0.66%--kernfs_refresh_inode
                                                           |          |          |          |                     |          
                                                           |          |          |          |                      --0.58%--_raw_spin_lock
                                                           |          |          |          |                                native_queued_spin_lock_slowpath
                                                           |          |          |          |          
                                                           |          |          |           --0.57%--up_read
                                                           |          |          |          
                                                           |          |           --3.49%--walk_component
                                                           |          |                     |          
                                                           |          |                      --3.04%--lookup_fast
                                                           |          |                                |          
                                                           |          |                                |--1.02%--kernfs_dop_revalidate
                                                           |          |                                |          |          
                                                           |          |                                |           --0.94%--down_read
                                                           |          |                                |          
                                                           |          |                                |--0.99%--__d_lookup
                                                           |          |                                |          _raw_spin_lock
                                                           |          |                                |          native_queued_spin_lock_slowpath
                                                           |          |                                |          
                                                           |          |                                 --0.59%--_raw_spin_lock
                                                           |          |          
                                                           |          |--4.18%--vfs_open
                                                           |          |          do_dentry_open
                                                           |          |          |          
                                                           |          |           --3.21%--kernfs_fop_open
                                                           |          |                     |          
                                                           |          |                      --2.50%--mutex_lock
                                                           |          |                                |          
                                                           |          |                                 --2.18%--__mutex_lock_slowpath
                                                           |          |                                           |          
                                                           |          |                                            --2.10%--__mutex_lock.isra.0
                                                           |          |                                                      |          
                                                           |          |                                                       --1.78%--osq_lock
                                                           |          |          
                                                           |          |--1.88%--alloc_empty_file
                                                           |          |          __alloc_file
                                                           |          |          |          
                                                           |          |           --1.09%--kmem_cache_alloc
                                                           |          |          
                                                           |          |--1.35%--may_open
                                                           |          |          |          
                                                           |          |           --1.27%--inode_permission
                                                           |          |                     |          
                                                           |          |                      --1.05%--kernfs_iop_permission
                                                           |          |                                |          
                                                           |          |                                 --0.64%--kernfs_refresh_inode
                                                           |          |                                           |          
                                                           |          |                                            --0.56%--_raw_spin_lock
                                                           |          |                                                      native_queued_spin_lock_slowpath
                                                           |          |          
                                                           |           --1.20%--lookup_fast
                                                           |                     |          
                                                           |                      --0.70%--__d_lookup
                                                           |                                |          
                                                           |                                 --0.65%--_raw_spin_lock
                                                           |                                           native_queued_spin_lock_slowpath
                                                           |          
                                                            --15.25%--get_unused_fd_flags
                                                                      |          
                                                                       --14.76%--alloc_fd
                                                                                 |          
                                                                                  --13.95%--_raw_spin_lock
                                                                                            native_queued_spin_lock_slowpath

    57.25%     0.00%  bin_sysbm        [unknown]           [k] 0x7379732f73656369
            |
            ---0x7379732f73656369
               0x7f88520d7ad4
               |          
                --57.05%--entry_SYSCALL_64_after_hwframe
                          |          
                           --56.97%--do_syscall_64
                                     |          
                                      --56.89%--__x64_sys_openat
                                                |          
                                                 --56.82%--do_sys_open
                                                           do_sys_openat2
                                                           |          
                                                           |--22.24%--fd_install
                                                           |          |          
                                                           |           --22.08%--_raw_spin_lock
                                                           |                     native_queued_spin_lock_slowpath
                                                           |          
                                                           |--18.23%--do_filp_open
                                                           |          path_openat
                                                           |          |          
                                                           |          |--8.39%--link_path_walk.part.0
                                                           |          |          |          
                                                           |          |          |--4.47%--inode_permission
                                                           |          |          |          |          
                                                           |          |          |          |--2.84%--kernfs_iop_permission
                                                           |          |          |          |          |          
                                                           |          |          |          |          |--1.19%--down_read
                                                           |          |          |          |          |          
                                                           |          |          |          |          |--0.68%--generic_permission
                                                           |          |          |          |          |          
                                                           |          |          |          |           --0.66%--kernfs_refresh_inode
                                                           |          |          |          |                     |          
                                                           |          |          |          |                      --0.58%--_raw_spin_lock
                                                           |          |          |          |                                native_queued_spin_lock_slowpath
                                                           |          |          |          |          
                                                           |          |          |           --0.57%--up_read
                                                           |          |          |          
                                                           |          |           --3.49%--walk_component
                                                           |          |                     |          
                                                           |          |                      --3.04%--lookup_fast
                                                           |          |                                |          
                                                           |          |                                |--1.02%--kernfs_dop_revalidate
                                                           |          |                                |          |          
                                                           |          |                                |           --0.94%--down_read
                                                           |          |                                |          
                                                           |          |                                |--0.99%--__d_lookup
                                                           |          |                                |          _raw_spin_lock
                                                           |          |                                |          native_queued_spin_lock_slowpath
                                                           |          |                                |          
                                                           |          |                                 --0.59%--_raw_spin_lock
                                                           |          |          
                                                           |          |--4.18%--vfs_open
                                                           |          |          do_dentry_open
                                                           |          |          |          
                                                           |          |           --3.21%--kernfs_fop_open
                                                           |          |                     |          
                                                           |          |                      --2.50%--mutex_lock
                                                           |          |                                |          
                                                           |          |                                 --2.18%--__mutex_lock_slowpath
                                                           |          |                                           |          
                                                           |          |                                            --2.10%--__mutex_lock.isra.0
                                                           |          |                                                      |          
                                                           |          |                                                       --1.78%--osq_lock
                                                           |          |          
                                                           |          |--1.88%--alloc_empty_file
                                                           |          |          __alloc_file
                                                           |          |          |          
                                                           |          |           --1.09%--kmem_cache_alloc
                                                           |          |          
                                                           |          |--1.35%--may_open
                                                           |          |          |          
                                                           |          |           --1.27%--inode_permission
                                                           |          |                     |          
                                                           |          |                      --1.05%--kernfs_iop_permission
                                                           |          |                                |          
                                                           |          |                                 --0.64%--kernfs_refresh_inode
                                                           |          |                                           |          
                                                           |          |                                            --0.56%--_raw_spin_lock
                                                           |          |                                                      native_queued_spin_lock_slowpath
                                                           |          |          
                                                           |           --1.20%--lookup_fast
                                                           |                     |          
                                                           |                      --0.70%--__d_lookup
                                                           |                                |          
                                                           |                                 --0.65%--_raw_spin_lock
                                                           |                                           native_queued_spin_lock_slowpath
                                                           |          
                                                            --15.25%--get_unused_fd_flags
                                                                      |          
                                                                       --14.76%--alloc_fd
                                                                                 |          
                                                                                  --13.95%--_raw_spin_lock
                                                                                            native_queued_spin_lock_slowpath

    56.89%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __x64_sys_openat
            |
            ---__x64_sys_openat
               |          
                --56.82%--do_sys_open
                          do_sys_openat2
                          |          
                          |--22.24%--fd_install
                          |          |          
                          |           --22.08%--_raw_spin_lock
                          |                     native_queued_spin_lock_slowpath
                          |          
                          |--18.23%--do_filp_open
                          |          path_openat
                          |          |          
                          |          |--8.39%--link_path_walk.part.0
                          |          |          |          
                          |          |          |--4.47%--inode_permission
                          |          |          |          |          
                          |          |          |          |--2.84%--kernfs_iop_permission
                          |          |          |          |          |          
                          |          |          |          |          |--1.19%--down_read
                          |          |          |          |          |          
                          |          |          |          |          |--0.68%--generic_permission
                          |          |          |          |          |          
                          |          |          |          |           --0.66%--kernfs_refresh_inode
                          |          |          |          |                     |          
                          |          |          |          |                      --0.58%--_raw_spin_lock
                          |          |          |          |                                native_queued_spin_lock_slowpath
                          |          |          |          |          
                          |          |          |           --0.57%--up_read
                          |          |          |          
                          |          |           --3.49%--walk_component
                          |          |                     |          
                          |          |                      --3.04%--lookup_fast
                          |          |                                |          
                          |          |                                |--1.02%--kernfs_dop_revalidate
                          |          |                                |          |          
                          |          |                                |           --0.94%--down_read
                          |          |                                |          
                          |          |                                |--0.99%--__d_lookup
                          |          |                                |          _raw_spin_lock
                          |          |                                |          native_queued_spin_lock_slowpath
                          |          |                                |          
                          |          |                                 --0.59%--_raw_spin_lock
                          |          |          
                          |          |--4.18%--vfs_open
                          |          |          do_dentry_open
                          |          |          |          
                          |          |           --3.21%--kernfs_fop_open
                          |          |                     |          
                          |          |                      --2.50%--mutex_lock
                          |          |                                |          
                          |          |                                 --2.18%--__mutex_lock_slowpath
                          |          |                                           |          
                          |          |                                            --2.10%--__mutex_lock.isra.0
                          |          |                                                      |          
                          |          |                                                       --1.78%--osq_lock
                          |          |          
                          |          |--1.88%--alloc_empty_file
                          |          |          __alloc_file
                          |          |          |          
                          |          |           --1.09%--kmem_cache_alloc
                          |          |          
                          |          |--1.35%--may_open
                          |          |          |          
                          |          |           --1.27%--inode_permission
                          |          |                     |          
                          |          |                      --1.05%--kernfs_iop_permission
                          |          |                                |          
                          |          |                                 --0.64%--kernfs_refresh_inode
                          |          |                                           |          
                          |          |                                            --0.56%--_raw_spin_lock
                          |          |                                                      native_queued_spin_lock_slowpath
                          |          |          
                          |           --1.20%--lookup_fast
                          |                     |          
                          |                      --0.70%--__d_lookup
                          |                                |          
                          |                                 --0.65%--_raw_spin_lock
                          |                                           native_queued_spin_lock_slowpath
                          |          
                           --15.25%--get_unused_fd_flags
                                     |          
                                      --14.76%--alloc_fd
                                                |          
                                                 --13.95%--_raw_spin_lock
                                                           native_queued_spin_lock_slowpath

    56.89%     0.31%  bin_sysbm        [kernel.kallsyms]   [k] do_sys_openat2
            |          
             --56.58%--do_sys_openat2
                       |          
                       |--22.24%--fd_install
                       |          |          
                       |           --22.08%--_raw_spin_lock
                       |                     native_queued_spin_lock_slowpath
                       |          
                       |--18.23%--do_filp_open
                       |          path_openat
                       |          |          
                       |          |--8.39%--link_path_walk.part.0
                       |          |          |          
                       |          |          |--4.47%--inode_permission
                       |          |          |          |          
                       |          |          |          |--2.84%--kernfs_iop_permission
                       |          |          |          |          |          
                       |          |          |          |          |--1.19%--down_read
                       |          |          |          |          |          
                       |          |          |          |          |--0.68%--generic_permission
                       |          |          |          |          |          
                       |          |          |          |           --0.66%--kernfs_refresh_inode
                       |          |          |          |                     |          
                       |          |          |          |                      --0.58%--_raw_spin_lock
                       |          |          |          |                                native_queued_spin_lock_slowpath
                       |          |          |          |          
                       |          |          |           --0.57%--up_read
                       |          |          |          
                       |          |           --3.49%--walk_component
                       |          |                     |          
                       |          |                      --3.04%--lookup_fast
                       |          |                                |          
                       |          |                                |--1.02%--kernfs_dop_revalidate
                       |          |                                |          |          
                       |          |                                |           --0.94%--down_read
                       |          |                                |          
                       |          |                                |--0.99%--__d_lookup
                       |          |                                |          _raw_spin_lock
                       |          |                                |          native_queued_spin_lock_slowpath
                       |          |                                |          
                       |          |                                 --0.59%--_raw_spin_lock
                       |          |          
                       |          |--4.18%--vfs_open
                       |          |          do_dentry_open
                       |          |          |          
                       |          |           --3.21%--kernfs_fop_open
                       |          |                     |          
                       |          |                      --2.50%--mutex_lock
                       |          |                                |          
                       |          |                                 --2.18%--__mutex_lock_slowpath
                       |          |                                           |          
                       |          |                                            --2.10%--__mutex_lock.isra.0
                       |          |                                                      |          
                       |          |                                                       --1.78%--osq_lock
                       |          |          
                       |          |--1.88%--alloc_empty_file
                       |          |          __alloc_file
                       |          |          |          
                       |          |           --1.09%--kmem_cache_alloc
                       |          |          
                       |          |--1.35%--may_open
                       |          |          |          
                       |          |           --1.27%--inode_permission
                       |          |                     |          
                       |          |                      --1.05%--kernfs_iop_permission
                       |          |                                |          
                       |          |                                 --0.64%--kernfs_refresh_inode
                       |          |                                           |          
                       |          |                                            --0.56%--_raw_spin_lock
                       |          |                                                      native_queued_spin_lock_slowpath
                       |          |          
                       |           --1.20%--lookup_fast
                       |                     |          
                       |                      --0.70%--__d_lookup
                       |                                |          
                       |                                 --0.65%--_raw_spin_lock
                       |                                           native_queued_spin_lock_slowpath
                       |          
                        --15.25%--get_unused_fd_flags
                                  |          
                                   --14.76%--alloc_fd
                                             |          
                                              --13.95%--_raw_spin_lock
                                                        native_queued_spin_lock_slowpath

    56.82%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] do_sys_open
            |
            ---do_sys_open
               do_sys_openat2
               |          
               |--22.24%--fd_install
               |          |          
               |           --22.08%--_raw_spin_lock
               |                     native_queued_spin_lock_slowpath
               |          
               |--18.23%--do_filp_open
               |          path_openat
               |          |          
               |          |--8.39%--link_path_walk.part.0
               |          |          |          
               |          |          |--4.47%--inode_permission
               |          |          |          |          
               |          |          |          |--2.84%--kernfs_iop_permission
               |          |          |          |          |          
               |          |          |          |          |--1.19%--down_read
               |          |          |          |          |          
               |          |          |          |          |--0.68%--generic_permission
               |          |          |          |          |          
               |          |          |          |           --0.66%--kernfs_refresh_inode
               |          |          |          |                     |          
               |          |          |          |                      --0.58%--_raw_spin_lock
               |          |          |          |                                native_queued_spin_lock_slowpath
               |          |          |          |          
               |          |          |           --0.57%--up_read
               |          |          |          
               |          |           --3.49%--walk_component
               |          |                     |          
               |          |                      --3.04%--lookup_fast
               |          |                                |          
               |          |                                |--1.02%--kernfs_dop_revalidate
               |          |                                |          |          
               |          |                                |           --0.94%--down_read
               |          |                                |          
               |          |                                |--0.99%--__d_lookup
               |          |                                |          _raw_spin_lock
               |          |                                |          native_queued_spin_lock_slowpath
               |          |                                |          
               |          |                                 --0.59%--_raw_spin_lock
               |          |          
               |          |--4.18%--vfs_open
               |          |          do_dentry_open
               |          |          |          
               |          |           --3.21%--kernfs_fop_open
               |          |                     |          
               |          |                      --2.50%--mutex_lock
               |          |                                |          
               |          |                                 --2.18%--__mutex_lock_slowpath
               |          |                                           |          
               |          |                                            --2.10%--__mutex_lock.isra.0
               |          |                                                      |          
               |          |                                                       --1.78%--osq_lock
               |          |          
               |          |--1.88%--alloc_empty_file
               |          |          __alloc_file
               |          |          |          
               |          |           --1.09%--kmem_cache_alloc
               |          |          
               |          |--1.35%--may_open
               |          |          |          
               |          |           --1.27%--inode_permission
               |          |                     |          
               |          |                      --1.05%--kernfs_iop_permission
               |          |                                |          
               |          |                                 --0.64%--kernfs_refresh_inode
               |          |                                           |          
               |          |                                            --0.56%--_raw_spin_lock
               |          |                                                      native_queued_spin_lock_slowpath
               |          |          
               |           --1.20%--lookup_fast
               |                     |          
               |                      --0.70%--__d_lookup
               |                                |          
               |                                 --0.65%--_raw_spin_lock
               |                                           native_queued_spin_lock_slowpath
               |          
                --15.25%--get_unused_fd_flags
                          |          
                           --14.76%--alloc_fd
                                     |          
                                      --13.95%--_raw_spin_lock
                                                native_queued_spin_lock_slowpath

    34.29%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520cc609
            |
            ---0x7f88520cc609
               |          
                --34.20%--0x56303487c721
                          |          
                          |--30.00%--0x7f88520d73fb
                          |          entry_SYSCALL_64_after_hwframe
                          |          |          
                          |           --29.84%--do_syscall_64
                          |                     |          
                          |                     |--20.63%--__x64_sys_close
                          |                     |          close_fd
                          |                     |          |          
                          |                     |          |--19.93%--pick_file
                          |                     |          |          |          
                          |                     |          |           --19.80%--_raw_spin_lock
                          |                     |          |                     native_queued_spin_lock_slowpath
                          |                     |          |          
                          |                     |           --0.62%--_raw_spin_lock
                          |                     |          
                          |                      --9.13%--syscall_exit_to_user_mode
                          |                                exit_to_user_mode_prepare
                          |                                |          
                          |                                 --8.98%--task_work_run
                          |                                           |          
                          |                                            --8.93%--____fput
                          |                                                      |          
                          |                                                       --8.85%--__fput
                          |                                                                 |          
                          |                                                                 |--7.13%--kernfs_fop_release
                          |                                                                 |          |          
                          |                                                                 |          |--5.46%--kernfs_put_open_node.isra.0
                          |                                                                 |          |          |          
                          |                                                                 |          |           --5.19%--mutex_lock
                          |                                                                 |          |                     |          
                          |                                                                 |          |                      --5.01%--__mutex_lock_slowpath
                          |                                                                 |          |                                |          
                          |                                                                 |          |                                 --4.88%--__mutex_lock.isra.0
                          |                                                                 |          |                                           |          
                          |                                                                 |          |                                            --4.57%--osq_lock
                          |                                                                 |          |          
                          |                                                                 |           --1.36%--seq_release
                          |                                                                 |                     |          
                          |                                                                 |                      --1.10%--kvfree
                          |                                                                 |                                kfree
                          |                                                                 |                                |          
                          |                                                                 |                                 --1.02%--obj_cgroup_uncharge
                          |                                                                 |                                           refill_obj_stock
                          |                                                                 |                                           |          
                          |                                                                 |                                            --0.67%--drain_obj_stock.isra.0
                          |                                                                 |                                                      |          
                          |                                                                 |                                                       --0.51%--__memcg_kmem_uncharge
                          |                                                                 |          
                          |                                                                  --0.82%--dput
                          |                                                                            _raw_spin_lock
                          |                                                                            native_queued_spin_lock_slowpath
                          |          
                           --3.83%--0x7f88520d736c
                                     entry_SYSCALL_64_after_hwframe
                                     |          
                                      --3.78%--do_syscall_64
                                                __x64_sys_read
                                                ksys_read
                                                |          
                                                 --3.62%--vfs_read
                                                           |          
                                                            --3.15%--new_sync_read
                                                                      |          
                                                                       --3.02%--kernfs_fop_read_iter
                                                                                 |          
                                                                                  --2.87%--seq_read_iter
                                                                                            |          
                                                                                             --1.74%--kvmalloc_node
                                                                                                       |          
                                                                                                        --1.59%--__kmalloc_node
                                                                                                                  |          
                                                                                                                   --1.32%--obj_cgroup_charge
                                                                                                                             |          
                                                                                                                              --0.83%--refill_obj_stock
                                                                                                                                        drain_obj_stock.isra.0

    34.20%     0.00%  bin_sysbm        bin_sysbm           [.] 0x000056303487c721
            |
            ---0x56303487c721
               |          
               |--30.00%--0x7f88520d73fb
               |          entry_SYSCALL_64_after_hwframe
               |          |          
               |           --29.84%--do_syscall_64
               |                     |          
               |                     |--20.63%--__x64_sys_close
               |                     |          close_fd
               |                     |          |          
               |                     |          |--19.93%--pick_file
               |                     |          |          |          
               |                     |          |           --19.80%--_raw_spin_lock
               |                     |          |                     native_queued_spin_lock_slowpath
               |                     |          |          
               |                     |           --0.62%--_raw_spin_lock
               |                     |          
               |                      --9.13%--syscall_exit_to_user_mode
               |                                exit_to_user_mode_prepare
               |                                |          
               |                                 --8.98%--task_work_run
               |                                           |          
               |                                            --8.93%--____fput
               |                                                      |          
               |                                                       --8.85%--__fput
               |                                                                 |          
               |                                                                 |--7.13%--kernfs_fop_release
               |                                                                 |          |          
               |                                                                 |          |--5.46%--kernfs_put_open_node.isra.0
               |                                                                 |          |          |          
               |                                                                 |          |           --5.19%--mutex_lock
               |                                                                 |          |                     |          
               |                                                                 |          |                      --5.01%--__mutex_lock_slowpath
               |                                                                 |          |                                |          
               |                                                                 |          |                                 --4.88%--__mutex_lock.isra.0
               |                                                                 |          |                                           |          
               |                                                                 |          |                                            --4.57%--osq_lock
               |                                                                 |          |          
               |                                                                 |           --1.36%--seq_release
               |                                                                 |                     |          
               |                                                                 |                      --1.10%--kvfree
               |                                                                 |                                kfree
               |                                                                 |                                |          
               |                                                                 |                                 --1.02%--obj_cgroup_uncharge
               |                                                                 |                                           refill_obj_stock
               |                                                                 |                                           |          
               |                                                                 |                                            --0.67%--drain_obj_stock.isra.0
               |                                                                 |                                                      |          
               |                                                                 |                                                       --0.51%--__memcg_kmem_uncharge
               |                                                                 |          
               |                                                                  --0.82%--dput
               |                                                                            _raw_spin_lock
               |                                                                            native_queued_spin_lock_slowpath
               |          
                --3.83%--0x7f88520d736c
                          entry_SYSCALL_64_after_hwframe
                          |          
                           --3.78%--do_syscall_64
                                     __x64_sys_read
                                     ksys_read
                                     |          
                                      --3.62%--vfs_read
                                                |          
                                                 --3.15%--new_sync_read
                                                           |          
                                                            --3.02%--kernfs_fop_read_iter
                                                                      |          
                                                                       --2.87%--seq_read_iter
                                                                                 |          
                                                                                  --1.74%--kvmalloc_node
                                                                                            |          
                                                                                             --1.59%--__kmalloc_node
                                                                                                       |          
                                                                                                        --1.32%--obj_cgroup_charge
                                                                                                                  |          
                                                                                                                   --0.83%--refill_obj_stock
                                                                                                                             drain_obj_stock.isra.0

    30.00%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520d73fb
            |
            ---0x7f88520d73fb
               entry_SYSCALL_64_after_hwframe
               |          
                --29.84%--do_syscall_64
                          |          
                          |--20.63%--__x64_sys_close
                          |          close_fd
                          |          |          
                          |          |--19.93%--pick_file
                          |          |          |          
                          |          |           --19.80%--_raw_spin_lock
                          |          |                     native_queued_spin_lock_slowpath
                          |          |          
                          |           --0.62%--_raw_spin_lock
                          |          
                           --9.13%--syscall_exit_to_user_mode
                                     exit_to_user_mode_prepare
                                     |          
                                      --8.98%--task_work_run
                                                |          
                                                 --8.93%--____fput
                                                           |          
                                                            --8.85%--__fput
                                                                      |          
                                                                      |--7.13%--kernfs_fop_release
                                                                      |          |          
                                                                      |          |--5.46%--kernfs_put_open_node.isra.0
                                                                      |          |          |          
                                                                      |          |           --5.19%--mutex_lock
                                                                      |          |                     |          
                                                                      |          |                      --5.01%--__mutex_lock_slowpath
                                                                      |          |                                |          
                                                                      |          |                                 --4.88%--__mutex_lock.isra.0
                                                                      |          |                                           |          
                                                                      |          |                                            --4.57%--osq_lock
                                                                      |          |          
                                                                      |           --1.36%--seq_release
                                                                      |                     |          
                                                                      |                      --1.10%--kvfree
                                                                      |                                kfree
                                                                      |                                |          
                                                                      |                                 --1.02%--obj_cgroup_uncharge
                                                                      |                                           refill_obj_stock
                                                                      |                                           |          
                                                                      |                                            --0.67%--drain_obj_stock.isra.0
                                                                      |                                                      |          
                                                                      |                                                       --0.51%--__memcg_kmem_uncharge
                                                                      |          
                                                                       --0.82%--dput
                                                                                 _raw_spin_lock
                                                                                 native_queued_spin_lock_slowpath

    22.24%     0.16%  bin_sysbm        [kernel.kallsyms]   [k] fd_install
            |          
             --22.08%--fd_install
                       _raw_spin_lock
                       native_queued_spin_lock_slowpath

    20.71%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] close_fd
            |          
             --20.63%--close_fd
                       |          
                       |--19.93%--pick_file
                       |          |          
                       |           --19.80%--_raw_spin_lock
                       |                     native_queued_spin_lock_slowpath
                       |          
                        --0.62%--_raw_spin_lock

    20.63%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __x64_sys_close
            |
            ---__x64_sys_close
               close_fd
               |          
               |--19.93%--pick_file
               |          |          
               |           --19.80%--_raw_spin_lock
               |                     native_queued_spin_lock_slowpath
               |          
                --0.62%--_raw_spin_lock

    19.93%     0.13%  bin_sysbm        [kernel.kallsyms]   [k] pick_file
            |          
             --19.80%--pick_file
                       _raw_spin_lock
                       native_queued_spin_lock_slowpath

    18.23%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] do_filp_open
            |
            ---do_filp_open
               path_openat
               |          
               |--8.39%--link_path_walk.part.0
               |          |          
               |          |--4.47%--inode_permission
               |          |          |          
               |          |          |--2.84%--kernfs_iop_permission
               |          |          |          |          
               |          |          |          |--1.19%--down_read
               |          |          |          |          
               |          |          |          |--0.68%--generic_permission
               |          |          |          |          
               |          |          |           --0.66%--kernfs_refresh_inode
               |          |          |                     |          
               |          |          |                      --0.58%--_raw_spin_lock
               |          |          |                                native_queued_spin_lock_slowpath
               |          |          |          
               |          |           --0.57%--up_read
               |          |          
               |           --3.49%--walk_component
               |                     |          
               |                      --3.04%--lookup_fast
               |                                |          
               |                                |--1.02%--kernfs_dop_revalidate
               |                                |          |          
               |                                |           --0.94%--down_read
               |                                |          
               |                                |--0.99%--__d_lookup
               |                                |          _raw_spin_lock
               |                                |          native_queued_spin_lock_slowpath
               |                                |          
               |                                 --0.59%--_raw_spin_lock
               |          
               |--4.18%--vfs_open
               |          do_dentry_open
               |          |          
               |           --3.21%--kernfs_fop_open
               |                     |          
               |                      --2.50%--mutex_lock
               |                                |          
               |                                 --2.18%--__mutex_lock_slowpath
               |                                           |          
               |                                            --2.10%--__mutex_lock.isra.0
               |                                                      |          
               |                                                       --1.78%--osq_lock
               |          
               |--1.88%--alloc_empty_file
               |          __alloc_file
               |          |          
               |           --1.09%--kmem_cache_alloc
               |          
               |--1.35%--may_open
               |          |          
               |           --1.27%--inode_permission
               |                     |          
               |                      --1.05%--kernfs_iop_permission
               |                                |          
               |                                 --0.64%--kernfs_refresh_inode
               |                                           |          
               |                                            --0.56%--_raw_spin_lock
               |                                                      native_queued_spin_lock_slowpath
               |          
                --1.20%--lookup_fast
                          |          
                           --0.70%--__d_lookup
                                     |          
                                      --0.65%--_raw_spin_lock
                                                native_queued_spin_lock_slowpath

    18.23%     0.50%  bin_sysbm        [kernel.kallsyms]   [k] path_openat
            |          
            |--17.72%--path_openat
            |          |          
            |          |--8.39%--link_path_walk.part.0
            |          |          |          
            |          |          |--4.47%--inode_permission
            |          |          |          |          
            |          |          |          |--2.84%--kernfs_iop_permission
            |          |          |          |          |          
            |          |          |          |          |--1.19%--down_read
            |          |          |          |          |          
            |          |          |          |          |--0.68%--generic_permission
            |          |          |          |          |          
            |          |          |          |           --0.66%--kernfs_refresh_inode
            |          |          |          |                     |          
            |          |          |          |                      --0.58%--_raw_spin_lock
            |          |          |          |                                native_queued_spin_lock_slowpath
            |          |          |          |          
            |          |          |           --0.57%--up_read
            |          |          |          
            |          |           --3.49%--walk_component
            |          |                     |          
            |          |                      --3.04%--lookup_fast
            |          |                                |          
            |          |                                |--1.02%--kernfs_dop_revalidate
            |          |                                |          |          
            |          |                                |           --0.94%--down_read
            |          |                                |          
            |          |                                |--0.99%--__d_lookup
            |          |                                |          _raw_spin_lock
            |          |                                |          native_queued_spin_lock_slowpath
            |          |                                |          
            |          |                                 --0.59%--_raw_spin_lock
            |          |          
            |          |--4.18%--vfs_open
            |          |          do_dentry_open
            |          |          |          
            |          |           --3.21%--kernfs_fop_open
            |          |                     |          
            |          |                      --2.50%--mutex_lock
            |          |                                |          
            |          |                                 --2.18%--__mutex_lock_slowpath
            |          |                                           |          
            |          |                                            --2.10%--__mutex_lock.isra.0
            |          |                                                      |          
            |          |                                                       --1.78%--osq_lock
            |          |          
            |          |--1.88%--alloc_empty_file
            |          |          __alloc_file
            |          |          |          
            |          |           --1.09%--kmem_cache_alloc
            |          |          
            |          |--1.35%--may_open
            |          |          |          
            |          |           --1.27%--inode_permission
            |          |                     |          
            |          |                      --1.05%--kernfs_iop_permission
            |          |                                |          
            |          |                                 --0.64%--kernfs_refresh_inode
            |          |                                           |          
            |          |                                            --0.56%--_raw_spin_lock
            |          |                                                      native_queued_spin_lock_slowpath
            |          |          
            |           --1.20%--lookup_fast
            |                     |          
            |                      --0.70%--__d_lookup
            |                                |          
            |                                 --0.65%--_raw_spin_lock
            |                                           native_queued_spin_lock_slowpath
            |          
             --0.50%--0x7379732f73656369
                       0x7f88520d7ad4
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       __x64_sys_openat
                       do_sys_open
                       do_sys_openat2
                       do_filp_open
                       path_openat

    15.25%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] get_unused_fd_flags
            |
            ---get_unused_fd_flags
               |          
                --14.76%--alloc_fd
                          |          
                           --13.95%--_raw_spin_lock
                                     native_queued_spin_lock_slowpath

    14.76%     0.54%  bin_sysbm        [kernel.kallsyms]   [k] alloc_fd
            |          
            |--14.21%--alloc_fd
            |          |          
            |           --13.95%--_raw_spin_lock
            |                     native_queued_spin_lock_slowpath
            |          
             --0.54%--0x7379732f73656369
                       0x7f88520d7ad4
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       __x64_sys_openat
                       do_sys_open
                       do_sys_openat2
                       get_unused_fd_flags
                       alloc_fd

     9.30%     0.17%  bin_sysbm        [kernel.kallsyms]   [k] syscall_exit_to_user_mode
            |          
             --9.13%--syscall_exit_to_user_mode
                       |          
                        --9.13%--exit_to_user_mode_prepare
                                  |          
                                   --8.98%--task_work_run
                                             |          
                                              --8.93%--____fput
                                                        |          
                                                         --8.85%--__fput
                                                                   |          
                                                                   |--7.13%--kernfs_fop_release
                                                                   |          |          
                                                                   |          |--5.46%--kernfs_put_open_node.isra.0
                                                                   |          |          |          
                                                                   |          |           --5.19%--mutex_lock
                                                                   |          |                     |          
                                                                   |          |                      --5.01%--__mutex_lock_slowpath
                                                                   |          |                                |          
                                                                   |          |                                 --4.88%--__mutex_lock.isra.0
                                                                   |          |                                           |          
                                                                   |          |                                            --4.57%--osq_lock
                                                                   |          |          
                                                                   |           --1.36%--seq_release
                                                                   |                     |          
                                                                   |                      --1.10%--kvfree
                                                                   |                                kfree
                                                                   |                                |          
                                                                   |                                 --1.02%--obj_cgroup_uncharge
                                                                   |                                           refill_obj_stock
                                                                   |                                           |          
                                                                   |                                            --0.67%--drain_obj_stock.isra.0
                                                                   |                                                      |          
                                                                   |                                                       --0.51%--__memcg_kmem_uncharge
                                                                   |          
                                                                    --0.82%--dput
                                                                              _raw_spin_lock
                                                                              native_queued_spin_lock_slowpath

     9.13%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] exit_to_user_mode_prepare
            |
            ---exit_to_user_mode_prepare
               |          
                --8.98%--task_work_run
                          |          
                           --8.93%--____fput
                                     |          
                                      --8.85%--__fput
                                                |          
                                                |--7.13%--kernfs_fop_release
                                                |          |          
                                                |          |--5.46%--kernfs_put_open_node.isra.0
                                                |          |          |          
                                                |          |           --5.19%--mutex_lock
                                                |          |                     |          
                                                |          |                      --5.01%--__mutex_lock_slowpath
                                                |          |                                |          
                                                |          |                                 --4.88%--__mutex_lock.isra.0
                                                |          |                                           |          
                                                |          |                                            --4.57%--osq_lock
                                                |          |          
                                                |           --1.36%--seq_release
                                                |                     |          
                                                |                      --1.10%--kvfree
                                                |                                kfree
                                                |                                |          
                                                |                                 --1.02%--obj_cgroup_uncharge
                                                |                                           refill_obj_stock
                                                |                                           |          
                                                |                                            --0.67%--drain_obj_stock.isra.0
                                                |                                                      |          
                                                |                                                       --0.51%--__memcg_kmem_uncharge
                                                |          
                                                 --0.82%--dput
                                                           _raw_spin_lock
                                                           native_queued_spin_lock_slowpath

     9.00%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] ____fput
            |          
             --8.93%--____fput
                       |          
                        --8.85%--__fput
                                  |          
                                  |--7.13%--kernfs_fop_release
                                  |          |          
                                  |          |--5.46%--kernfs_put_open_node.isra.0
                                  |          |          |          
                                  |          |           --5.19%--mutex_lock
                                  |          |                     |          
                                  |          |                      --5.01%--__mutex_lock_slowpath
                                  |          |                                |          
                                  |          |                                 --4.88%--__mutex_lock.isra.0
                                  |          |                                           |          
                                  |          |                                            --4.57%--osq_lock
                                  |          |          
                                  |           --1.36%--seq_release
                                  |                     |          
                                  |                      --1.10%--kvfree
                                  |                                kfree
                                  |                                |          
                                  |                                 --1.02%--obj_cgroup_uncharge
                                  |                                           refill_obj_stock
                                  |                                           |          
                                  |                                            --0.67%--drain_obj_stock.isra.0
                                  |                                                      |          
                                  |                                                       --0.51%--__memcg_kmem_uncharge
                                  |          
                                   --0.82%--dput
                                             _raw_spin_lock
                                             native_queued_spin_lock_slowpath

     8.98%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] task_work_run
            |
            ---task_work_run
               |          
                --8.93%--____fput
                          |          
                           --8.85%--__fput
                                     |          
                                     |--7.13%--kernfs_fop_release
                                     |          |          
                                     |          |--5.46%--kernfs_put_open_node.isra.0
                                     |          |          |          
                                     |          |           --5.19%--mutex_lock
                                     |          |                     |          
                                     |          |                      --5.01%--__mutex_lock_slowpath
                                     |          |                                |          
                                     |          |                                 --4.88%--__mutex_lock.isra.0
                                     |          |                                           |          
                                     |          |                                            --4.57%--osq_lock
                                     |          |          
                                     |           --1.36%--seq_release
                                     |                     |          
                                     |                      --1.10%--kvfree
                                     |                                kfree
                                     |                                |          
                                     |                                 --1.02%--obj_cgroup_uncharge
                                     |                                           refill_obj_stock
                                     |                                           |          
                                     |                                            --0.67%--drain_obj_stock.isra.0
                                     |                                                      |          
                                     |                                                       --0.51%--__memcg_kmem_uncharge
                                     |          
                                      --0.82%--dput
                                                _raw_spin_lock
                                                native_queued_spin_lock_slowpath

     8.85%     0.23%  bin_sysbm        [kernel.kallsyms]   [k] __fput
            |          
             --8.61%--__fput
                       |          
                       |--7.13%--kernfs_fop_release
                       |          |          
                       |          |--5.46%--kernfs_put_open_node.isra.0
                       |          |          |          
                       |          |           --5.19%--mutex_lock
                       |          |                     |          
                       |          |                      --5.01%--__mutex_lock_slowpath
                       |          |                                |          
                       |          |                                 --4.88%--__mutex_lock.isra.0
                       |          |                                           |          
                       |          |                                            --4.57%--osq_lock
                       |          |          
                       |           --1.36%--seq_release
                       |                     |          
                       |                      --1.10%--kvfree
                       |                                kfree
                       |                                |          
                       |                                 --1.02%--obj_cgroup_uncharge
                       |                                           refill_obj_stock
                       |                                           |          
                       |                                            --0.67%--drain_obj_stock.isra.0
                       |                                                      |          
                       |                                                       --0.51%--__memcg_kmem_uncharge
                       |          
                        --0.82%--dput
                                  _raw_spin_lock
                                  native_queued_spin_lock_slowpath

     8.39%     0.10%  bin_sysbm        [kernel.kallsyms]   [k] link_path_walk.part.0
            |          
             --8.29%--link_path_walk.part.0
                       |          
                       |--4.47%--inode_permission
                       |          |          
                       |          |--2.84%--kernfs_iop_permission
                       |          |          |          
                       |          |          |--1.19%--down_read
                       |          |          |          
                       |          |          |--0.68%--generic_permission
                       |          |          |          
                       |          |           --0.66%--kernfs_refresh_inode
                       |          |                     |          
                       |          |                      --0.58%--_raw_spin_lock
                       |          |                                native_queued_spin_lock_slowpath
                       |          |          
                       |           --0.57%--up_read
                       |          
                        --3.49%--walk_component
                                  |          
                                   --3.04%--lookup_fast
                                             |          
                                             |--1.02%--kernfs_dop_revalidate
                                             |          |          
                                             |           --0.94%--down_read
                                             |          
                                             |--0.99%--__d_lookup
                                             |          _raw_spin_lock
                                             |          native_queued_spin_lock_slowpath
                                             |          
                                              --0.59%--_raw_spin_lock

     8.29%     0.00%  swapper          [kernel.kallsyms]   [k] secondary_startup_64_no_verify
            |
            ---secondary_startup_64_no_verify
               |          
                --8.19%--start_secondary
                          cpu_startup_entry
                          |          
                           --8.18%--do_idle
                                     |          
                                      --7.83%--call_cpuidle
                                                cpuidle_enter
                                                |          
                                                 --7.79%--cpuidle_enter_state
                                                           |          
                                                           |--6.14%--asm_sysvec_apic_timer_interrupt
                                                           |          sysvec_apic_timer_interrupt
                                                           |          |          
                                                           |          |--4.02%--irq_exit_rcu
                                                           |          |          |          
                                                           |          |           --3.98%--__softirqentry_text_start
                                                           |          |                     |          
                                                           |          |                     |--3.11%--rcu_core_si
                                                           |          |                     |          |          
                                                           |          |                     |           --3.03%--rcu_core
                                                           |          |                     |                     |          
                                                           |          |                     |                      --3.00%--file_free_rcu
                                                           |          |                     |                                |          
                                                           |          |                     |                                 --1.63%--kmem_cache_free
                                                           |          |                     |                                           |          
                                                           |          |                     |                                            --0.75%--obj_cgroup_uncharge
                                                           |          |                     |                                                      refill_obj_stock
                                                           |          |                     |          
                                                           |          |                      --0.87%--run_rebalance_domains
                                                           |          |                                |          
                                                           |          |                                 --0.81%--rebalance_domains
                                                           |          |                                           |          
                                                           |          |                                            --0.65%--load_balance
                                                           |          |          
                                                           |           --2.06%--__sysvec_apic_timer_interrupt
                                                           |                     |          
                                                           |                      --1.85%--hrtimer_interrupt
                                                           |                                |          
                                                           |                                |--0.85%--tick_program_event
                                                           |                                |          |          
                                                           |                                |           --0.82%--clockevents_program_event
                                                           |                                |                     |          
                                                           |                                |                      --0.79%--native_apic_mem_write
                                                           |                                |          
                                                           |                                 --0.82%--__hrtimer_run_queues
                                                           |                                           |          
                                                           |                                            --0.80%--tick_sched_timer
                                                           |                                                      |          
                                                           |                                                       --0.73%--tick_sched_handle.isra.0
                                                           |                                                                 |          
                                                           |                                                                  --0.72%--update_process_times
                                                           |          
                                                            --1.51%--acpi_idle_enter
                                                                      |          
                                                                       --1.51%--acpi_idle_do_entry

     8.29%     0.00%  swapper          [kernel.kallsyms]   [k] cpu_startup_entry
            |
            ---cpu_startup_entry
               |          
                --8.28%--do_idle
                          |          
                           --7.90%--call_cpuidle
                                     cpuidle_enter
                                     |          
                                      --7.86%--cpuidle_enter_state
                                                |          
                                                |--6.19%--asm_sysvec_apic_timer_interrupt
                                                |          sysvec_apic_timer_interrupt
                                                |          |          
                                                |          |--4.04%--irq_exit_rcu
                                                |          |          |          
                                                |          |           --4.00%--__softirqentry_text_start
                                                |          |                     |          
                                                |          |                     |--3.11%--rcu_core_si
                                                |          |                     |          |          
                                                |          |                     |           --3.03%--rcu_core
                                                |          |                     |                     |          
                                                |          |                     |                      --3.00%--file_free_rcu
                                                |          |                     |                                |          
                                                |          |                     |                                 --1.63%--kmem_cache_free
                                                |          |                     |                                           |          
                                                |          |                     |                                            --0.75%--obj_cgroup_uncharge
                                                |          |                     |                                                      refill_obj_stock
                                                |          |                     |          
                                                |          |                      --0.88%--run_rebalance_domains
                                                |          |                                |          
                                                |          |                                 --0.83%--rebalance_domains
                                                |          |                                           |          
                                                |          |                                            --0.66%--load_balance
                                                |          |          
                                                |           --2.09%--__sysvec_apic_timer_interrupt
                                                |                     |          
                                                |                      --1.88%--hrtimer_interrupt
                                                |                                |          
                                                |                                |--0.86%--tick_program_event
                                                |                                |          |          
                                                |                                |           --0.83%--clockevents_program_event
                                                |                                |                     |          
                                                |                                |                      --0.80%--native_apic_mem_write
                                                |                                |          
                                                |                                 --0.83%--__hrtimer_run_queues
                                                |                                           |          
                                                |                                            --0.81%--tick_sched_timer
                                                |                                                      |          
                                                |                                                       --0.73%--tick_sched_handle.isra.0
                                                |                                                                 |          
                                                |                                                                  --0.72%--update_process_times
                                                |          
                                                 --1.52%--acpi_idle_enter
                                                           |          
                                                            --1.51%--acpi_idle_do_entry

     8.28%     0.03%  swapper          [kernel.kallsyms]   [k] do_idle
            |          
             --8.25%--do_idle
                       |          
                        --7.90%--call_cpuidle
                                  cpuidle_enter
                                  |          
                                   --7.86%--cpuidle_enter_state
                                             |          
                                             |--6.19%--asm_sysvec_apic_timer_interrupt
                                             |          sysvec_apic_timer_interrupt
                                             |          |          
                                             |          |--4.04%--irq_exit_rcu
                                             |          |          |          
                                             |          |           --4.00%--__softirqentry_text_start
                                             |          |                     |          
                                             |          |                     |--3.11%--rcu_core_si
                                             |          |                     |          |          
                                             |          |                     |           --3.03%--rcu_core
                                             |          |                     |                     |          
                                             |          |                     |                      --3.00%--file_free_rcu
                                             |          |                     |                                |          
                                             |          |                     |                                 --1.63%--kmem_cache_free
                                             |          |                     |                                           |          
                                             |          |                     |                                            --0.75%--obj_cgroup_uncharge
                                             |          |                     |                                                      refill_obj_stock
                                             |          |                     |          
                                             |          |                      --0.88%--run_rebalance_domains
                                             |          |                                |          
                                             |          |                                 --0.83%--rebalance_domains
                                             |          |                                           |          
                                             |          |                                            --0.66%--load_balance
                                             |          |          
                                             |           --2.09%--__sysvec_apic_timer_interrupt
                                             |                     |          
                                             |                      --1.88%--hrtimer_interrupt
                                             |                                |          
                                             |                                |--0.86%--tick_program_event
                                             |                                |          |          
                                             |                                |           --0.83%--clockevents_program_event
                                             |                                |                     |          
                                             |                                |                      --0.80%--native_apic_mem_write
                                             |                                |          
                                             |                                 --0.83%--__hrtimer_run_queues
                                             |                                           |          
                                             |                                            --0.81%--tick_sched_timer
                                             |                                                      |          
                                             |                                                       --0.73%--tick_sched_handle.isra.0
                                             |                                                                 |          
                                             |                                                                  --0.72%--update_process_times
                                             |          
                                              --1.52%--acpi_idle_enter
                                                        |          
                                                         --1.51%--acpi_idle_do_entry

     8.19%     0.00%  swapper          [kernel.kallsyms]   [k] start_secondary
            |
            ---start_secondary
               cpu_startup_entry
               |          
                --8.18%--do_idle
                          |          
                           --7.83%--call_cpuidle
                                     cpuidle_enter
                                     |          
                                      --7.79%--cpuidle_enter_state
                                                |          
                                                |--6.14%--asm_sysvec_apic_timer_interrupt
                                                |          sysvec_apic_timer_interrupt
                                                |          |          
                                                |          |--4.02%--irq_exit_rcu
                                                |          |          |          
                                                |          |           --3.98%--__softirqentry_text_start
                                                |          |                     |          
                                                |          |                     |--3.11%--rcu_core_si
                                                |          |                     |          |          
                                                |          |                     |           --3.03%--rcu_core
                                                |          |                     |                     |          
                                                |          |                     |                      --3.00%--file_free_rcu
                                                |          |                     |                                |          
                                                |          |                     |                                 --1.63%--kmem_cache_free
                                                |          |                     |                                           |          
                                                |          |                     |                                            --0.75%--obj_cgroup_uncharge
                                                |          |                     |                                                      refill_obj_stock
                                                |          |                     |          
                                                |          |                      --0.87%--run_rebalance_domains
                                                |          |                                |          
                                                |          |                                 --0.81%--rebalance_domains
                                                |          |                                           |          
                                                |          |                                            --0.65%--load_balance
                                                |          |          
                                                |           --2.06%--__sysvec_apic_timer_interrupt
                                                |                     |          
                                                |                      --1.85%--hrtimer_interrupt
                                                |                                |          
                                                |                                |--0.85%--tick_program_event
                                                |                                |          |          
                                                |                                |           --0.82%--clockevents_program_event
                                                |                                |                     |          
                                                |                                |                      --0.79%--native_apic_mem_write
                                                |                                |          
                                                |                                 --0.82%--__hrtimer_run_queues
                                                |                                           |          
                                                |                                            --0.80%--tick_sched_timer
                                                |                                                      |          
                                                |                                                       --0.73%--tick_sched_handle.isra.0
                                                |                                                                 |          
                                                |                                                                  --0.72%--update_process_times
                                                |          
                                                 --1.51%--acpi_idle_enter
                                                           |          
                                                            --1.51%--acpi_idle_do_entry

     7.90%     0.00%  swapper          [kernel.kallsyms]   [k] call_cpuidle
            |
            ---call_cpuidle
               cpuidle_enter
               |          
                --7.86%--cpuidle_enter_state
                          |          
                          |--6.19%--asm_sysvec_apic_timer_interrupt
                          |          sysvec_apic_timer_interrupt
                          |          |          
                          |          |--4.04%--irq_exit_rcu
                          |          |          |          
                          |          |           --4.00%--__softirqentry_text_start
                          |          |                     |          
                          |          |                     |--3.11%--rcu_core_si
                          |          |                     |          |          
                          |          |                     |           --3.03%--rcu_core
                          |          |                     |                     |          
                          |          |                     |                      --3.00%--file_free_rcu
                          |          |                     |                                |          
                          |          |                     |                                 --1.63%--kmem_cache_free
                          |          |                     |                                           |          
                          |          |                     |                                            --0.75%--obj_cgroup_uncharge
                          |          |                     |                                                      refill_obj_stock
                          |          |                     |          
                          |          |                      --0.88%--run_rebalance_domains
                          |          |                                |          
                          |          |                                 --0.83%--rebalance_domains
                          |          |                                           |          
                          |          |                                            --0.66%--load_balance
                          |          |          
                          |           --2.09%--__sysvec_apic_timer_interrupt
                          |                     |          
                          |                      --1.88%--hrtimer_interrupt
                          |                                |          
                          |                                |--0.86%--tick_program_event
                          |                                |          |          
                          |                                |           --0.83%--clockevents_program_event
                          |                                |                     |          
                          |                                |                      --0.80%--native_apic_mem_write
                          |                                |          
                          |                                 --0.83%--__hrtimer_run_queues
                          |                                           |          
                          |                                            --0.81%--tick_sched_timer
                          |                                                      |          
                          |                                                       --0.73%--tick_sched_handle.isra.0
                          |                                                                 |          
                          |                                                                  --0.72%--update_process_times
                          |          
                           --1.52%--acpi_idle_enter
                                     |          
                                      --1.51%--acpi_idle_do_entry

     7.90%     0.01%  swapper          [kernel.kallsyms]   [k] cpuidle_enter
            |          
             --7.89%--cpuidle_enter
                       |          
                        --7.86%--cpuidle_enter_state
                                  |          
                                  |--6.19%--asm_sysvec_apic_timer_interrupt
                                  |          sysvec_apic_timer_interrupt
                                  |          |          
                                  |          |--4.04%--irq_exit_rcu
                                  |          |          |          
                                  |          |           --4.00%--__softirqentry_text_start
                                  |          |                     |          
                                  |          |                     |--3.11%--rcu_core_si
                                  |          |                     |          |          
                                  |          |                     |           --3.03%--rcu_core
                                  |          |                     |                     |          
                                  |          |                     |                      --3.00%--file_free_rcu
                                  |          |                     |                                |          
                                  |          |                     |                                 --1.63%--kmem_cache_free
                                  |          |                     |                                           |          
                                  |          |                     |                                            --0.75%--obj_cgroup_uncharge
                                  |          |                     |                                                      refill_obj_stock
                                  |          |                     |          
                                  |          |                      --0.88%--run_rebalance_domains
                                  |          |                                |          
                                  |          |                                 --0.83%--rebalance_domains
                                  |          |                                           |          
                                  |          |                                            --0.66%--load_balance
                                  |          |          
                                  |           --2.09%--__sysvec_apic_timer_interrupt
                                  |                     |          
                                  |                      --1.88%--hrtimer_interrupt
                                  |                                |          
                                  |                                |--0.86%--tick_program_event
                                  |                                |          |          
                                  |                                |           --0.83%--clockevents_program_event
                                  |                                |                     |          
                                  |                                |                      --0.80%--native_apic_mem_write
                                  |                                |          
                                  |                                 --0.83%--__hrtimer_run_queues
                                  |                                           |          
                                  |                                            --0.81%--tick_sched_timer
                                  |                                                      |          
                                  |                                                       --0.73%--tick_sched_handle.isra.0
                                  |                                                                 |          
                                  |                                                                  --0.72%--update_process_times
                                  |          
                                   --1.52%--acpi_idle_enter
                                             |          
                                              --1.51%--acpi_idle_do_entry

     7.86%     0.08%  swapper          [kernel.kallsyms]   [k] cpuidle_enter_state
            |          
             --7.78%--cpuidle_enter_state
                       |          
                       |--6.19%--asm_sysvec_apic_timer_interrupt
                       |          sysvec_apic_timer_interrupt
                       |          |          
                       |          |--4.04%--irq_exit_rcu
                       |          |          |          
                       |          |           --4.00%--__softirqentry_text_start
                       |          |                     |          
                       |          |                     |--3.11%--rcu_core_si
                       |          |                     |          |          
                       |          |                     |           --3.03%--rcu_core
                       |          |                     |                     |          
                       |          |                     |                      --3.00%--file_free_rcu
                       |          |                     |                                |          
                       |          |                     |                                 --1.63%--kmem_cache_free
                       |          |                     |                                           |          
                       |          |                     |                                            --0.75%--obj_cgroup_uncharge
                       |          |                     |                                                      refill_obj_stock
                       |          |                     |          
                       |          |                      --0.88%--run_rebalance_domains
                       |          |                                |          
                       |          |                                 --0.83%--rebalance_domains
                       |          |                                           |          
                       |          |                                            --0.66%--load_balance
                       |          |          
                       |           --2.09%--__sysvec_apic_timer_interrupt
                       |                     |          
                       |                      --1.88%--hrtimer_interrupt
                       |                                |          
                       |                                |--0.86%--tick_program_event
                       |                                |          |          
                       |                                |           --0.83%--clockevents_program_event
                       |                                |                     |          
                       |                                |                      --0.80%--native_apic_mem_write
                       |                                |          
                       |                                 --0.83%--__hrtimer_run_queues
                       |                                           |          
                       |                                            --0.81%--tick_sched_timer
                       |                                                      |          
                       |                                                       --0.73%--tick_sched_handle.isra.0
                       |                                                                 |          
                       |                                                                  --0.72%--update_process_times
                       |          
                        --1.52%--acpi_idle_enter
                                  |          
                                   --1.51%--acpi_idle_do_entry

     7.78%     0.51%  bin_sysbm        [kernel.kallsyms]   [k] mutex_lock
            |          
             --7.27%--mutex_lock
                       |          
                        --7.19%--__mutex_lock_slowpath
                                  |          
                                   --6.97%--__mutex_lock.isra.0
                                             |          
                                              --6.35%--osq_lock

     7.25%     0.06%  bin_sysbm        [kernel.kallsyms]   [k] __mutex_lock_slowpath
            |          
             --7.19%--__mutex_lock_slowpath
                       |          
                        --6.97%--__mutex_lock.isra.0
                                  |          
                                   --6.35%--osq_lock

     7.21%     0.19%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_fop_release
            |          
             --7.02%--kernfs_fop_release
                       |          
                       |--5.46%--kernfs_put_open_node.isra.0
                       |          |          
                       |           --5.19%--mutex_lock
                       |                     |          
                       |                      --5.01%--__mutex_lock_slowpath
                       |                                |          
                       |                                 --4.88%--__mutex_lock.isra.0
                       |                                           |          
                       |                                            --4.57%--osq_lock
                       |          
                        --1.36%--seq_release
                                  |          
                                   --1.10%--kvfree
                                             kfree
                                             |          
                                              --1.02%--obj_cgroup_uncharge
                                                        refill_obj_stock
                                                        |          
                                                         --0.67%--drain_obj_stock.isra.0
                                                                   |          
                                                                    --0.51%--__memcg_kmem_uncharge

     6.97%     0.25%  bin_sysbm        [kernel.kallsyms]   [k] __mutex_lock.isra.0
            |          
             --6.73%--__mutex_lock.isra.0
                       |          
                        --6.35%--osq_lock

     6.57%     6.57%  bin_sysbm        [kernel.kallsyms]   [k] osq_lock
            |          
            |--4.71%--0x7f88520cc609
            |          0x56303487c721
            |          0x7f88520d73fb
            |          entry_SYSCALL_64_after_hwframe
            |          do_syscall_64
            |          syscall_exit_to_user_mode
            |          exit_to_user_mode_prepare
            |          task_work_run
            |          ____fput
            |          __fput
            |          kernfs_fop_release
            |          kernfs_put_open_node.isra.0
            |          mutex_lock
            |          __mutex_lock_slowpath
            |          |          
            |           --4.57%--__mutex_lock.isra.0
            |                     osq_lock
            |          
             --1.86%--0x7379732f73656369
                       0x7f88520d7ad4
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       __x64_sys_openat
                       do_sys_open
                       do_sys_openat2
                       do_filp_open
                       path_openat
                       vfs_open
                       do_dentry_open
                       kernfs_fop_open
                       mutex_lock
                       __mutex_lock_slowpath
                       |          
                        --1.78%--__mutex_lock.isra.0
                                  osq_lock

     6.19%     0.00%  swapper          [kernel.kallsyms]   [k] asm_sysvec_apic_timer_interrupt
            |
            ---asm_sysvec_apic_timer_interrupt
               sysvec_apic_timer_interrupt
               |          
               |--4.04%--irq_exit_rcu
               |          |          
               |           --4.00%--__softirqentry_text_start
               |                     |          
               |                     |--3.11%--rcu_core_si
               |                     |          |          
               |                     |           --3.03%--rcu_core
               |                     |                     |          
               |                     |                      --3.00%--file_free_rcu
               |                     |                                |          
               |                     |                                 --1.63%--kmem_cache_free
               |                     |                                           |          
               |                     |                                            --0.75%--obj_cgroup_uncharge
               |                     |                                                      refill_obj_stock
               |                     |          
               |                      --0.88%--run_rebalance_domains
               |                                |          
               |                                 --0.83%--rebalance_domains
               |                                           |          
               |                                            --0.66%--load_balance
               |          
                --2.09%--__sysvec_apic_timer_interrupt
                          |          
                           --1.88%--hrtimer_interrupt
                                     |          
                                     |--0.86%--tick_program_event
                                     |          |          
                                     |           --0.83%--clockevents_program_event
                                     |                     |          
                                     |                      --0.80%--native_apic_mem_write
                                     |          
                                      --0.83%--__hrtimer_run_queues
                                                |          
                                                 --0.81%--tick_sched_timer
                                                           |          
                                                            --0.73%--tick_sched_handle.isra.0
                                                                      |          
                                                                       --0.72%--update_process_times

     6.19%     0.00%  swapper          [kernel.kallsyms]   [k] sysvec_apic_timer_interrupt
            |
            ---sysvec_apic_timer_interrupt
               |          
               |--4.04%--irq_exit_rcu
               |          |          
               |           --4.00%--__softirqentry_text_start
               |                     |          
               |                     |--3.11%--rcu_core_si
               |                     |          |          
               |                     |           --3.03%--rcu_core
               |                     |                     |          
               |                     |                      --3.00%--file_free_rcu
               |                     |                                |          
               |                     |                                 --1.63%--kmem_cache_free
               |                     |                                           |          
               |                     |                                            --0.75%--obj_cgroup_uncharge
               |                     |                                                      refill_obj_stock
               |                     |          
               |                      --0.88%--run_rebalance_domains
               |                                |          
               |                                 --0.83%--rebalance_domains
               |                                           |          
               |                                            --0.66%--load_balance
               |          
                --2.09%--__sysvec_apic_timer_interrupt
                          |          
                           --1.88%--hrtimer_interrupt
                                     |          
                                     |--0.86%--tick_program_event
                                     |          |          
                                     |           --0.83%--clockevents_program_event
                                     |                     |          
                                     |                      --0.80%--native_apic_mem_write
                                     |          
                                      --0.83%--__hrtimer_run_queues
                                                |          
                                                 --0.81%--tick_sched_timer
                                                           |          
                                                            --0.73%--tick_sched_handle.isra.0
                                                                      |          
                                                                       --0.72%--update_process_times

     5.75%     0.80%  bin_sysbm        [kernel.kallsyms]   [k] inode_permission
            |          
            |--4.95%--inode_permission
            |          |          
            |          |--3.89%--kernfs_iop_permission
            |          |          |          
            |          |          |--1.52%--down_read
            |          |          |          
            |          |          |--1.31%--kernfs_refresh_inode
            |          |          |          |          
            |          |          |           --1.14%--_raw_spin_lock
            |          |          |                     native_queued_spin_lock_slowpath
            |          |          |          
            |          |           --0.68%--generic_permission
            |          |          
            |           --0.80%--up_read
            |          
             --0.80%--0x7379732f73656369
                       0x7f88520d7ad4
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       __x64_sys_openat
                       do_sys_open
                       do_sys_openat2
                       do_filp_open
                       path_openat
                       link_path_walk.part.0
                       inode_permission

     5.46%     0.21%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_put_open_node.isra.0
            |          
             --5.25%--kernfs_put_open_node.isra.0
                       |          
                        --5.19%--mutex_lock
                                  |          
                                   --5.01%--__mutex_lock_slowpath
                                             |          
                                              --4.88%--__mutex_lock.isra.0
                                                        |          
                                                         --4.57%--osq_lock

     4.24%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] lookup_fast
            |
            ---lookup_fast
               |          
               |--1.69%--__d_lookup
               |          |          
               |           --1.64%--_raw_spin_lock
               |                     native_queued_spin_lock_slowpath
               |          
               |--1.31%--kernfs_dop_revalidate
               |          |          
               |           --1.22%--down_read
               |          
                --0.72%--_raw_spin_lock

     4.18%     0.36%  bin_sysbm        [kernel.kallsyms]   [k] do_dentry_open
            |          
             --3.82%--do_dentry_open
                       |          
                        --3.21%--kernfs_fop_open
                                  |          
                                   --2.50%--mutex_lock
                                             |          
                                              --2.18%--__mutex_lock_slowpath
                                                        |          
                                                         --2.10%--__mutex_lock.isra.0
                                                                   |          
                                                                    --1.78%--osq_lock

     4.18%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] vfs_open
            |
            ---vfs_open
               do_dentry_open
               |          
                --3.21%--kernfs_fop_open
                          |          
                           --2.50%--mutex_lock
                                     |          
                                      --2.18%--__mutex_lock_slowpath
                                                |          
                                                 --2.10%--__mutex_lock.isra.0
                                                           |          
                                                            --1.78%--osq_lock

     4.05%     0.00%  swapper          [kernel.kallsyms]   [k] irq_exit_rcu
            |
            ---irq_exit_rcu
               |          
                --4.01%--__softirqentry_text_start
                          |          
                          |--3.11%--rcu_core_si
                          |          |          
                          |           --3.03%--rcu_core
                          |                     |          
                          |                      --3.00%--file_free_rcu
                          |                                |          
                          |                                 --1.63%--kmem_cache_free
                          |                                           |          
                          |                                            --0.75%--obj_cgroup_uncharge
                          |                                                      refill_obj_stock
                          |          
                           --0.89%--run_rebalance_domains
                                     |          
                                      --0.83%--rebalance_domains
                                                |          
                                                 --0.66%--load_balance

     4.01%     0.00%  swapper          [kernel.kallsyms]   [k] __softirqentry_text_start
            |          
             --4.01%--__softirqentry_text_start
                       |          
                       |--3.11%--rcu_core_si
                       |          |          
                       |           --3.03%--rcu_core
                       |                     |          
                       |                      --3.00%--file_free_rcu
                       |                                |          
                       |                                 --1.63%--kmem_cache_free
                       |                                           |          
                       |                                            --0.75%--obj_cgroup_uncharge
                       |                                                      refill_obj_stock
                       |          
                        --0.89%--run_rebalance_domains
                                  |          
                                   --0.83%--rebalance_domains
                                             |          
                                              --0.66%--load_balance

     3.99%     0.18%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_iop_permission
            |          
             --3.81%--kernfs_iop_permission
                       |          
                       |--1.52%--down_read
                       |          
                       |--1.31%--kernfs_refresh_inode
                       |          |          
                       |           --1.14%--_raw_spin_lock
                       |                     native_queued_spin_lock_slowpath
                       |          
                        --0.68%--generic_permission

     3.83%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520d736c
            |
            ---0x7f88520d736c
               entry_SYSCALL_64_after_hwframe
               |          
                --3.78%--do_syscall_64
                          __x64_sys_read
                          ksys_read
                          |          
                           --3.62%--vfs_read
                                     |          
                                      --3.15%--new_sync_read
                                                |          
                                                 --3.02%--kernfs_fop_read_iter
                                                           |          
                                                            --2.87%--seq_read_iter
                                                                      |          
                                                                       --1.74%--kvmalloc_node
                                                                                 |          
                                                                                  --1.59%--__kmalloc_node
                                                                                            |          
                                                                                             --1.32%--obj_cgroup_charge
                                                                                                       |          
                                                                                                        --0.83%--refill_obj_stock
                                                                                                                  drain_obj_stock.isra.0

     3.83%     0.05%  bin_sysbm        [kernel.kallsyms]   [k] __x64_sys_read
            |          
             --3.78%--__x64_sys_read
                       ksys_read
                       |          
                        --3.62%--vfs_read
                                  |          
                                   --3.15%--new_sync_read
                                             |          
                                              --3.02%--kernfs_fop_read_iter
                                                        |          
                                                         --2.87%--seq_read_iter
                                                                   |          
                                                                    --1.74%--kvmalloc_node
                                                                              |          
                                                                               --1.59%--__kmalloc_node
                                                                                         |          
                                                                                          --1.32%--obj_cgroup_charge
                                                                                                    |          
                                                                                                     --0.83%--refill_obj_stock
                                                                                                               drain_obj_stock.isra.0

     3.78%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] ksys_read
            |
            ---ksys_read
               |          
                --3.62%--vfs_read
                          |          
                           --3.15%--new_sync_read
                                     |          
                                      --3.02%--kernfs_fop_read_iter
                                                |          
                                                 --2.87%--seq_read_iter
                                                           |          
                                                            --1.74%--kvmalloc_node
                                                                      |          
                                                                       --1.59%--__kmalloc_node
                                                                                 |          
                                                                                  --1.32%--obj_cgroup_charge
                                                                                            |          
                                                                                             --0.83%--refill_obj_stock
                                                                                                       drain_obj_stock.isra.0

     3.62%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] vfs_read
            |          
             --3.55%--vfs_read
                       |          
                        --3.15%--new_sync_read
                                  |          
                                   --3.02%--kernfs_fop_read_iter
                                             |          
                                              --2.87%--seq_read_iter
                                                        |          
                                                         --1.74%--kvmalloc_node
                                                                   |          
                                                                    --1.59%--__kmalloc_node
                                                                              |          
                                                                               --1.32%--obj_cgroup_charge
                                                                                         |          
                                                                                          --0.83%--refill_obj_stock
                                                                                                    drain_obj_stock.isra.0

     3.49%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] walk_component
            |          
             --3.42%--walk_component
                       |          
                        --3.04%--lookup_fast
                                  |          
                                  |--1.02%--kernfs_dop_revalidate
                                  |          |          
                                  |           --0.94%--down_read
                                  |          
                                  |--0.99%--__d_lookup
                                  |          _raw_spin_lock
                                  |          native_queued_spin_lock_slowpath
                                  |          
                                   --0.59%--_raw_spin_lock

     3.21%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_fop_open
            |
            ---kernfs_fop_open
               |          
                --2.50%--mutex_lock
                          |          
                           --2.18%--__mutex_lock_slowpath
                                     |          
                                      --2.10%--__mutex_lock.isra.0
                                                |          
                                                 --1.78%--osq_lock

     3.15%     0.13%  bin_sysbm        [kernel.kallsyms]   [k] new_sync_read
            |          
             --3.02%--new_sync_read
                       kernfs_fop_read_iter
                       |          
                        --2.87%--seq_read_iter
                                  |          
                                   --1.74%--kvmalloc_node
                                             |          
                                              --1.59%--__kmalloc_node
                                                        |          
                                                         --1.32%--obj_cgroup_charge
                                                                   |          
                                                                    --0.83%--refill_obj_stock
                                                                              drain_obj_stock.isra.0

     3.11%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_core_si
            |
            ---rcu_core_si
               |          
                --3.03%--rcu_core
                          |          
                           --3.00%--file_free_rcu
                                     |          
                                      --1.63%--kmem_cache_free
                                                |          
                                                 --0.75%--obj_cgroup_uncharge
                                                           refill_obj_stock

     3.03%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_core
            |          
             --3.02%--rcu_core
                       |          
                        --3.00%--file_free_rcu
                                  |          
                                   --1.63%--kmem_cache_free
                                             |          
                                              --0.75%--obj_cgroup_uncharge
                                                        refill_obj_stock

     3.02%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_fop_read_iter
            |
            ---kernfs_fop_read_iter
               |          
                --2.87%--seq_read_iter
                          |          
                           --1.74%--kvmalloc_node
                                     |          
                                      --1.59%--__kmalloc_node
                                                |          
                                                 --1.32%--obj_cgroup_charge
                                                           |          
                                                            --0.83%--refill_obj_stock
                                                                      drain_obj_stock.isra.0

     3.00%     1.37%  swapper          [kernel.kallsyms]   [k] file_free_rcu
            |          
            |--1.63%--file_free_rcu
            |          |          
            |           --1.63%--kmem_cache_free
            |                     |          
            |                      --0.75%--obj_cgroup_uncharge
            |                                refill_obj_stock
            |          
             --1.37%--secondary_startup_64_no_verify
                       start_secondary
                       cpu_startup_entry
                       do_idle
                       call_cpuidle
                       cpuidle_enter
                       cpuidle_enter_state
                       asm_sysvec_apic_timer_interrupt
                       sysvec_apic_timer_interrupt
                       irq_exit_rcu
                       __softirqentry_text_start
                       rcu_core_si
                       rcu_core
                       file_free_rcu

     2.87%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] seq_read_iter
            |
            ---seq_read_iter
               |          
                --1.74%--kvmalloc_node
                          |          
                           --1.59%--__kmalloc_node
                                     |          
                                      --1.32%--obj_cgroup_charge
                                                |          
                                                 --0.83%--refill_obj_stock
                                                           drain_obj_stock.isra.0

     2.74%     2.72%  bin_sysbm        [kernel.kallsyms]   [k] down_read
            |          
             --2.72%--0x7379732f73656369
                       0x7f88520d7ad4
                       entry_SYSCALL_64_after_hwframe
                       do_syscall_64
                       __x64_sys_openat
                       do_sys_open
                       do_sys_openat2
                       do_filp_open
                       path_openat
                       |          
                        --2.11%--link_path_walk.part.0
                                  |          
                                  |--1.17%--inode_permission
                                  |          kernfs_iop_permission
                                  |          down_read
                                  |          
                                   --0.94%--walk_component
                                             lookup_fast
                                             kernfs_dop_revalidate
                                             down_read

     2.09%     0.00%  swapper          [kernel.kallsyms]   [k] __sysvec_apic_timer_interrupt
            |
            ---__sysvec_apic_timer_interrupt
               |          
                --1.88%--hrtimer_interrupt
                          |          
                          |--0.86%--tick_program_event
                          |          |          
                          |           --0.83%--clockevents_program_event
                          |                     |          
                          |                      --0.80%--native_apic_mem_write
                          |          
                           --0.83%--__hrtimer_run_queues
                                     |          
                                      --0.81%--tick_sched_timer
                                                |          
                                                 --0.73%--tick_sched_handle.isra.0
                                                           |          
                                                            --0.72%--update_process_times

     2.03%     0.50%  bin_sysbm        [kernel.kallsyms]   [k] refill_obj_stock
            |          
             --1.53%--refill_obj_stock
                       drain_obj_stock.isra.0
                       |          
                        --0.99%--__memcg_kmem_uncharge
                                  |          
                                   --0.84%--page_counter_uncharge
                                             |          
                                              --0.76%--page_counter_cancel

     1.88%     0.00%  swapper          [kernel.kallsyms]   [k] hrtimer_interrupt
            |          
             --1.88%--hrtimer_interrupt
                       |          
                       |--0.86%--tick_program_event
                       |          |          
                       |           --0.83%--clockevents_program_event
                       |                     |          
                       |                      --0.80%--native_apic_mem_write
                       |          
                        --0.83%--__hrtimer_run_queues
                                  |          
                                   --0.81%--tick_sched_timer
                                             |          
                                              --0.73%--tick_sched_handle.isra.0
                                                        |          
                                                         --0.72%--update_process_times

     1.88%     0.42%  bin_sysbm        [kernel.kallsyms]   [k] __alloc_file
            |          
             --1.46%--__alloc_file
                       |          
                        --1.09%--kmem_cache_alloc

     1.88%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] alloc_empty_file
            |
            ---alloc_empty_file
               __alloc_file
               |          
                --1.09%--kmem_cache_alloc

     1.74%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] kvmalloc_node
            |
            ---kvmalloc_node
               |          
                --1.59%--__kmalloc_node
                          |          
                           --1.32%--obj_cgroup_charge
                                     |          
                                      --0.83%--refill_obj_stock
                                                drain_obj_stock.isra.0

     1.70%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] obj_cgroup_charge
            |          
             --1.62%--obj_cgroup_charge
                       |          
                       |--0.83%--refill_obj_stock
                       |          drain_obj_stock.isra.0
                       |          
                        --0.79%--__memcg_kmem_charge
                                  |          
                                   --0.65%--page_counter_try_charge

     1.69%     0.05%  bin_sysbm        [kernel.kallsyms]   [k] __d_lookup
            |          
             --1.64%--__d_lookup
                       _raw_spin_lock
                       native_queued_spin_lock_slowpath

     1.63%     0.68%  swapper          [kernel.kallsyms]   [k] kmem_cache_free
            |          
            |--0.94%--kmem_cache_free
            |          |          
            |           --0.75%--obj_cgroup_uncharge
            |                     refill_obj_stock
            |          
             --0.68%--secondary_startup_64_no_verify
                       start_secondary
                       cpu_startup_entry
                       do_idle
                       call_cpuidle
                       cpuidle_enter
                       cpuidle_enter_state
                       asm_sysvec_apic_timer_interrupt
                       sysvec_apic_timer_interrupt
                       irq_exit_rcu
                       __softirqentry_text_start
                       rcu_core_si
                       rcu_core
                       file_free_rcu
                       kmem_cache_free

     1.59%     0.13%  bin_sysbm        [kernel.kallsyms]   [k] __kmalloc_node
            |          
             --1.46%--__kmalloc_node
                       |          
                        --1.32%--obj_cgroup_charge
                                  |          
                                   --0.83%--refill_obj_stock
                                             drain_obj_stock.isra.0

     1.53%     0.02%  swapper          [kernel.kallsyms]   [k] acpi_idle_enter
            |          
             --1.51%--acpi_idle_enter
                       acpi_idle_do_entry

     1.53%     0.54%  bin_sysbm        [kernel.kallsyms]   [k] drain_obj_stock.isra.0
            |          
            |--0.99%--drain_obj_stock.isra.0
            |          __memcg_kmem_uncharge
            |          |          
            |           --0.84%--page_counter_uncharge
            |                     |          
            |                      --0.76%--page_counter_cancel
            |          
             --0.54%--0x7f88520cc609
                       0x56303487c721

     1.51%     1.51%  swapper          [kernel.kallsyms]   [k] acpi_idle_do_entry
            |          
             --1.51%--secondary_startup_64_no_verify
                       start_secondary
                       cpu_startup_entry
                       do_idle
                       call_cpuidle
                       cpuidle_enter
                       cpuidle_enter_state
                       acpi_idle_enter
                       acpi_idle_do_entry

     1.43%     0.34%  bin_sysbm        [kernel.kallsyms]   [k] kmem_cache_alloc
            |          
             --1.09%--kmem_cache_alloc

     1.36%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] seq_release
            |
            ---seq_release
               |          
                --1.10%--kvfree
                          kfree
                          |          
                           --1.02%--obj_cgroup_uncharge
                                     refill_obj_stock
                                     |          
                                      --0.67%--drain_obj_stock.isra.0
                                                |          
                                                 --0.51%--__memcg_kmem_uncharge

     1.35%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] may_open
            |          
             --1.27%--may_open
                       inode_permission
                       |          
                        --1.05%--kernfs_iop_permission
                                  |          
                                   --0.64%--kernfs_refresh_inode
                                             |          
                                              --0.56%--_raw_spin_lock
                                                        native_queued_spin_lock_slowpath

     1.31%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_dop_revalidate
            |          
             --1.22%--kernfs_dop_revalidate
                       down_read

     1.31%     0.16%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_refresh_inode
            |          
             --1.14%--kernfs_refresh_inode
                       _raw_spin_lock
                       native_queued_spin_lock_slowpath

     1.20%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] obj_cgroup_uncharge
            |
            ---obj_cgroup_uncharge
               refill_obj_stock
               |          
                --0.70%--drain_obj_stock.isra.0
                          |          
                           --0.54%--__memcg_kmem_uncharge

     1.18%     0.16%  bin_sysbm        [kernel.kallsyms]   [k] kfree
            |          
             --1.02%--kfree
                       obj_cgroup_uncharge
                       refill_obj_stock
                       |          
                        --0.67%--drain_obj_stock.isra.0
                                  |          
                                   --0.51%--__memcg_kmem_uncharge

     1.10%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] kvfree
            |
            ---kvfree
               kfree
               |          
                --1.02%--obj_cgroup_uncharge
                          refill_obj_stock
                          |          
                           --0.67%--drain_obj_stock.isra.0
                                     |          
                                      --0.51%--__memcg_kmem_uncharge

     1.09%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] dput
            |
            ---dput
               |          
                --1.01%--_raw_spin_lock
                          native_queued_spin_lock_slowpath

     1.02%     1.02%  swapper          [kernel.kallsyms]   [k] native_apic_mem_write
            |
            ---secondary_startup_64_no_verify
               |          
                --1.01%--start_secondary
                          cpu_startup_entry
                          do_idle
                          |          
                           --1.00%--call_cpuidle
                                     cpuidle_enter
                                     cpuidle_enter_state
                                     asm_sysvec_apic_timer_interrupt
                                     sysvec_apic_timer_interrupt
                                     __sysvec_apic_timer_interrupt
                                     |          
                                      --0.79%--hrtimer_interrupt
                                                tick_program_event
                                                clockevents_program_event
                                                native_apic_mem_write

     0.99%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __memcg_kmem_uncharge
            |
            ---__memcg_kmem_uncharge
               |          
                --0.84%--page_counter_uncharge
                          |          
                           --0.76%--page_counter_cancel

     0.96%     0.96%  bin_sysbm        [kernel.kallsyms]   [k] up_read
            |
            ---0x7379732f73656369
               0x7f88520d7ad4
               entry_SYSCALL_64_after_hwframe
               do_syscall_64
               __x64_sys_openat
               do_sys_open
               do_sys_openat2
               do_filp_open
               path_openat
               |          
                --0.65%--link_path_walk.part.0
                          |          
                           --0.57%--inode_permission
                                     up_read

     0.92%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] page_counter_uncharge
            |
            ---page_counter_uncharge
               |          
                --0.84%--page_counter_cancel

     0.89%     0.00%  swapper          [kernel.kallsyms]   [k] run_rebalance_domains
            |
            ---run_rebalance_domains
               |          
                --0.83%--rebalance_domains
                          |          
                           --0.66%--load_balance

     0.88%     0.01%  swapper          [kernel.kallsyms]   [k] tick_program_event
            |          
             --0.86%--tick_program_event
                       |          
                        --0.85%--clockevents_program_event
                                  |          
                                   --0.82%--native_apic_mem_write

     0.85%     0.00%  swapper          [kernel.kallsyms]   [k] clockevents_program_event
            |          
             --0.85%--clockevents_program_event
                       |          
                        --0.82%--native_apic_mem_write

     0.84%     0.84%  bin_sysbm        [kernel.kallsyms]   [k] page_counter_cancel
            |
            ---0x7f88520cc609
               0x56303487c721

     0.83%     0.16%  swapper          [kernel.kallsyms]   [k] rebalance_domains
            |          
             --0.67%--rebalance_domains
                       |          
                        --0.67%--load_balance

     0.83%     0.01%  swapper          [kernel.kallsyms]   [k] __hrtimer_run_queues
            |          
             --0.82%--__hrtimer_run_queues
                       |          
                        --0.81%--tick_sched_timer
                                  |          
                                   --0.73%--tick_sched_handle.isra.0
                                             |          
                                              --0.72%--update_process_times

     0.82%     0.01%  swapper          [kernel.kallsyms]   [k] tick_sched_timer
            |          
             --0.81%--tick_sched_timer
                       |          
                        --0.73%--tick_sched_handle.isra.0
                                  |          
                                   --0.72%--update_process_times

     0.81%     0.81%  bin_sysbm        [kernel.kallsyms]   [k] generic_permission
            |
            ---0x7379732f73656369
               0x7f88520d7ad4
               entry_SYSCALL_64_after_hwframe
               do_syscall_64
               __x64_sys_openat
               do_sys_open
               do_sys_openat2
               do_filp_open
               path_openat
               link_path_walk.part.0
               inode_permission
               |          
                --0.68%--kernfs_iop_permission
                          generic_permission

     0.79%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __memcg_kmem_charge
            |
            ---__memcg_kmem_charge
               |          
                --0.65%--page_counter_try_charge

     0.75%     0.34%  swapper          [kernel.kallsyms]   [k] refill_obj_stock
     0.75%     0.00%  swapper          [kernel.kallsyms]   [k] obj_cgroup_uncharge
            |
            ---obj_cgroup_uncharge
               refill_obj_stock

     0.74%     0.01%  swapper          [kernel.kallsyms]   [k] tick_sched_handle.isra.0
            |          
             --0.73%--tick_sched_handle.isra.0
                       |          
                        --0.72%--update_process_times

     0.74%     0.04%  swapper          [kernel.kallsyms]   [k] update_process_times
            |          
             --0.70%--update_process_times

     0.73%     0.55%  bin_sysbm        [kernel.kallsyms]   [k] kmem_cache_free
     0.73%     0.73%  bin_sysbm        [kernel.kallsyms]   [k] page_counter_try_charge
     0.67%     0.23%  swapper          [kernel.kallsyms]   [k] load_balance
     0.62%     0.62%  bin_sysbm        [kernel.kallsyms]   [k] memset
     0.60%     0.03%  bin_sysbm        [kernel.kallsyms]   [k] step_into
            |          
             --0.57%--step_into

     0.51%     0.51%  bin_sysbm        [kernel.kallsyms]   [k] lockref_put_return
            |
            ---0x7379732f73656369
               0x7f88520d7ad4
               entry_SYSCALL_64_after_hwframe
               do_syscall_64
               __x64_sys_openat
               do_sys_open
               do_sys_openat2
               do_filp_open
               path_openat

     0.47%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] putname
     0.46%     0.46%  bin_sysbm        [kernel.kallsyms]   [k] get_obj_cgroup_from_current
     0.44%     0.02%  swapper          [kernel.kallsyms]   [k] scheduler_tick
     0.44%     0.02%  swapper          [kernel.kallsyms]   [k] find_busiest_group
     0.44%     0.44%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_put_active
     0.42%     0.12%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_seq_show
     0.42%     0.23%  swapper          [kernel.kallsyms]   [k] update_sd_lb_stats.constprop.0
     0.42%     0.00%  swapper          [kernel.kallsyms]   [k] perf_event_task_tick
     0.41%     0.00%  swapper          [kernel.kallsyms]   [k] amd_pmu_disable_all
     0.41%     0.00%  swapper          [kernel.kallsyms]   [k] perf_pmu_disable.part.0
     0.41%     0.00%  swapper          [kernel.kallsyms]   [k] x86_pmu_disable
     0.40%     0.24%  swapper          [kernel.kallsyms]   [k] drain_obj_stock.isra.0
     0.40%     0.11%  bin_sysbm        [kernel.kallsyms]   [k] rw_verify_area
     0.39%     0.29%  bin_sysbm        [kernel.kallsyms]   [k] lockref_get
     0.39%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] path_get
     0.38%     0.38%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_get_active
     0.37%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_seq_stop
     0.34%     0.02%  swapper          [kernel.kallsyms]   [k] x86_pmu_disable_all
     0.34%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] seq_open
     0.33%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] terminate_walk
     0.33%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_seq_start
     0.32%     0.32%  swapper          [kernel.kallsyms]   [k] native_read_msr
     0.31%     0.31%  bin_sysbm        [kernel.kallsyms]   [k] _raw_spin_lock_irq
     0.30%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] sysfs_kf_seq_show
     0.30%     0.30%  bin_sysbm        [kernel.kallsyms]   [k] apparmor_file_free_security
     0.30%     0.30%  bin_sysbm        [kernel.kallsyms]   [k] mutex_spin_on_owner
     0.29%     0.14%  bin_sysbm        [kernel.kallsyms]   [k] security_file_permission
     0.27%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] getname
     0.27%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] getname_flags
     0.27%     0.27%  bin_sysbm        [kernel.kallsyms]   [k] mutex_unlock
     0.26%     0.26%  bin_sysbm        [kernel.kallsyms]   [k] __cond_resched
     0.24%     0.24%  swapper          [kernel.kallsyms]   [k] rcu_sched_clock_irq
     0.24%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] security_file_alloc
     0.22%     0.06%  bin_sysbm        [kernel.kallsyms]   [k] dev_attr_show
     0.22%     0.22%  bin_sysbm        [kernel.kallsyms]   [k] strcmp
     0.22%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] security_file_free
     0.22%     0.00%  swapper          [kernel.kallsyms]   [k] cpuidle_select
     0.20%     0.14%  swapper          [kernel.kallsyms]   [k] menu_select
     0.20%     0.20%  bin_sysbm        libpthread-2.31.so  [.] 0x0000000000014ad4
     0.17%     0.17%  bin_sysbm        [kernel.kallsyms]   [k] _find_next_bit.constprop.0
     0.17%     0.09%  bin_sysbm        [kernel.kallsyms]   [k] __mod_memcg_lruvec_state
     0.16%     0.00%  swapper          [kernel.kallsyms]   [k] __memcg_kmem_uncharge
     0.16%     0.00%  swapper          [kernel.kallsyms]   [k] page_counter_uncharge
     0.16%     0.16%  swapper          [kernel.kallsyms]   [k] page_counter_cancel
     0.16%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] get_page_from_freelist
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __slab_alloc
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] ___slab_alloc
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] allocate_slab
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] alloc_pages_current
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __alloc_pages_nodemask
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] core_id_show
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] sysfs_emit
     0.16%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] vscnprintf
     0.16%     0.16%  bin_sysbm        libpthread-2.31.so  [.] 0x0000000000014003
     0.16%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520d7003
     0.16%     0.16%  swapper          [kernel.kallsyms]   [k] __slab_free
     0.15%     0.15%  bin_sysbm        [kernel.kallsyms]   [k] lockref_get_not_dead
     0.15%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] try_to_unlazy
     0.15%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __legitimize_path.isra.0
     0.15%     0.00%  swapper          [kernel.kallsyms]   [k] cpumask_next_and
     0.15%     0.15%  bin_sysbm        [kernel.kallsyms]   [k] locks_remove_file
     0.15%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] refill_stock
     0.15%     0.14%  bin_sysbm        [kernel.kallsyms]   [k] security_file_open
     0.15%     0.15%  bin_sysbm        [kernel.kallsyms]   [k] __d_lookup_rcu
     0.14%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] try_charge
     0.14%     0.14%  bin_sysbm        [kernel.kallsyms]   [k] propagate_protected_usage
     0.13%     0.13%  bin_sysbm        [kernel.kallsyms]   [k] rcu_all_qs
     0.12%     0.12%  swapper          [kernel.kallsyms]   [k] ktime_get
     0.11%     0.02%  bin_sysbm        [kernel.kallsyms]   [k] security_task_getsecid
     0.11%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] ima_file_check
     0.10%     0.02%  bin_sysbm        [kernel.kallsyms]   [k] __check_object_size
     0.10%     0.10%  swapper          [kernel.kallsyms]   [k] read_tsc
     0.10%     0.00%  swapper          [kernel.kallsyms]   [k] x86_64_start_kernel
     0.10%     0.00%  swapper          [kernel.kallsyms]   [k] x86_64_start_reservations
     0.10%     0.00%  swapper          [kernel.kallsyms]   [k] start_kernel
     0.10%     0.00%  swapper          [kernel.kallsyms]   [k] arch_call_rest_init
     0.10%     0.00%  swapper          [kernel.kallsyms]   [k] rest_init
     0.10%     0.10%  swapper          [kernel.kallsyms]   [k] find_next_and_bit
     0.09%     0.09%  bin_sysbm        [kernel.kallsyms]   [k] aa_get_task_label
     0.09%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] apparmor_task_getsecid
     0.09%     0.00%  bin_sysbm        libc-2.31.so        [.] 0x00007f8851fd689b
     0.09%     0.09%  swapper          [kernel.kallsyms]   [k] ktime_get_update_offsets_now
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] rcu_read_unlock_strict
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] memcpy
     0.08%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] drain_stock.isra.0
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] __mod_memcg_state
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] clear_page_rep
     0.08%     0.08%  migration/91     [kernel.kallsyms]   [k] balance_rt
     0.08%     0.00%  migration/91     [kernel.kallsyms]   [k] ret_from_fork
     0.08%     0.00%  migration/91     [kernel.kallsyms]   [k] kthread
     0.08%     0.00%  migration/91     [kernel.kallsyms]   [k] smpboot_thread_fn
     0.08%     0.00%  migration/91     [kernel.kallsyms]   [k] schedule
     0.08%     0.08%  bin_sysbm        bin_sysbm           [.] 0x000000000000144c
     0.08%     0.00%  bin_sysbm        bin_sysbm           [.] 0x000056303487c44c
     0.08%     0.08%  bin_sysbm        libpthread-2.31.so  [.] 0x0000000000014054
     0.08%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520d7054
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] mntput_no_expire
     0.08%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] mntput
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] security_inode_permission
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] __virt_addr_valid
     0.08%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] strncpy_from_user
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] common_file_perm
     0.08%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] apparmor_file_permission
     0.08%     0.08%  swapper          [kernel.kallsyms]   [k] menu_reflect
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] vsnprintf
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] locks_remove_posix
     0.08%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] filp_close
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] osq_unlock
     0.08%     0.08%  swapper          [kernel.kallsyms]   [k] rcu_cblist_dequeue
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] kernfs_seq_next
     0.08%     0.08%  bin_sysbm        [kernel.kallsyms]   [k] legitimize_root
     0.07%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] set_root
     0.07%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] path_init
     0.07%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] nd_jump_root
     0.07%     0.07%  swapper          [kernel.kallsyms]   [k] _find_next_bit.constprop.0
     0.07%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] apparmor_file_alloc_security
     0.07%     0.07%  swapper          [kernel.kallsyms]   [k] native_write_msr
     0.07%     0.07%  bin_sysbm        [kernel.kallsyms]   [k] __fsnotify_parent
     0.07%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] schedule
     0.07%     0.01%  bin_sysbm        [kernel.kallsyms]   [k] __schedule
     0.07%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __x64_sys_sched_yield
     0.06%     0.06%  bin_sysbm        [kernel.kallsyms]   [k] mem_cgroup_from_task
     0.06%     0.06%  bin_sysbm        [kernel.kallsyms]   [k] _raw_spin_lock_irqsave
     0.06%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] do_sched_yield
     0.05%     0.05%  bin_sysbm        [kernel.kallsyms]   [k] make_kuid
     0.05%     0.05%  swapper          [kernel.kallsyms]   [k] tick_nohz_next_event
     0.05%     0.03%  swapper          [kernel.kallsyms]   [k] update_blocked_averages
     0.05%     0.00%  swapper          [kernel.kallsyms]   [k] tick_nohz_get_sleep_length
     0.05%     0.01%  swapper          [kernel.kallsyms]   [k] tick_nohz_irq_exit
     0.04%     0.04%  bin_sysbm        [kernel.kallsyms]   [k] __lookup_mnt
     0.04%     0.01%  swapper          [kernel.kallsyms]   [k] irq_enter_rcu
     0.04%     0.04%  swapper          [kernel.kallsyms]   [k] __mod_memcg_state
     0.04%     0.04%  rcu_sched        [kernel.kallsyms]   [k] update_nohz_stats
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] ret_from_fork
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] kthread
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] rcu_gp_kthread
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] schedule_timeout
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] schedule
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] __schedule
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] pick_next_task_fair
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] newidle_balance
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] load_balance
     0.04%     0.00%  rcu_sched        [kernel.kallsyms]   [k] find_busiest_group
     0.04%     0.00%  swapper          [kernel.kallsyms]   [k] asm_sysvec_call_function_single
     0.04%     0.00%  swapper          [kernel.kallsyms]   [k] sysvec_call_function_single
     0.04%     0.01%  bin_sysbm        [kernel.kallsyms]   [k] pick_next_task_fair
     0.03%     0.00%  swapper          [kernel.kallsyms]   [k] sched_clock
     0.03%     0.00%  swapper          [kernel.kallsyms]   [k] sched_clock_cpu
     0.03%     0.03%  swapper          [kernel.kallsyms]   [k] _raw_spin_lock
     0.03%     0.00%  swapper          [kernel.kallsyms]   [k] tick_irq_enter
     0.03%     0.03%  swapper          [kernel.kallsyms]   [k] pvclock_clocksource_read
     0.03%     0.00%  swapper          [kernel.kallsyms]   [k] kvm_sched_clock_read
     0.03%     0.03%  swapper          [kernel.kallsyms]   [k] llist_reverse_order
     0.03%     0.00%  swapper          [kernel.kallsyms]   [k] __sysvec_call_function_single
     0.03%     0.00%  swapper          [kernel.kallsyms]   [k] generic_smp_call_function_single_interrupt
     0.03%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] asm_sysvec_apic_timer_interrupt
     0.03%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] sysvec_apic_timer_interrupt
     0.02%     0.00%  swapper          [kernel.kallsyms]   [k] arch_cpu_idle_enter
     0.02%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __softirqentry_text_start
     0.02%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] irq_exit_rcu
     0.02%     0.02%  swapper          [kernel.kallsyms]   [k] error_return
     0.02%     0.02%  swapper          [kernel.kallsyms]   [k] tsc_verify_tsc_adjust
     0.02%     0.02%  swapper          [kernel.kallsyms]   [k] __update_load_avg_cfs_rq
     0.02%     0.00%  swapper          [kernel.kallsyms]   [k] hrtimer_start_range_ns
     0.02%     0.00%  swapper          [kernel.kallsyms]   [k] hrtimer_reprogram
     0.02%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] load_balance
     0.02%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] find_busiest_group
     0.02%     0.01%  swapper          [kernel.kallsyms]   [k] update_load_avg
     0.02%     0.00%  swapper          [kernel.kallsyms]   [k] note_gp_changes
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] run_rebalance_domains
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] update_sd_lb_stats.constprop.0
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] update_blocked_averages
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] _nohz_idle_balance
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] lapic_next_event
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] sched_clock_idle_sleep_event
     0.01%     0.01%  kworker/75:1-mm  [kernel.kallsyms]   [k] __rdgsbase_inactive
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] schedule_idle
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] __schedule
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] tick_nohz_idle_exit
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] __tick_nohz_idle_restart_tick
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] nr_iowait_cpu
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] _raw_spin_lock_irqsave
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] update_curr
     0.01%     0.01%  bin_sysbm        [kernel.kallsyms]   [k] update_rq_clock
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] expand_files
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] newidle_balance
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] update_nohz_stats
     0.01%     0.01%  kworker/79:1-mm  [kernel.kallsyms]   [k] psi_group_change
     0.01%     0.00%  kworker/79:1-mm  [kernel.kallsyms]   [k] ret_from_fork
     0.01%     0.00%  kworker/79:1-mm  [kernel.kallsyms]   [k] kthread
     0.01%     0.00%  kworker/79:1-mm  [kernel.kallsyms]   [k] worker_thread
     0.01%     0.00%  kworker/79:1-mm  [kernel.kallsyms]   [k] schedule
     0.01%     0.00%  kworker/79:1-mm  [kernel.kallsyms]   [k] __schedule
     0.01%     0.00%  kworker/79:1-mm  [kernel.kallsyms]   [k] psi_task_switch
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] pick_next_task_fair
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] cpuidle_governor_latency_req
     0.01%     0.01%  bin_sysbm        [kernel.kallsyms]   [k] apparmor_file_open
     0.01%     0.01%  kworker/59:1-mm  [kernel.kallsyms]   [k] update_sd_lb_stats.constprop.0
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] ret_from_fork
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] kthread
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] worker_thread
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] schedule
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] __schedule
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] pick_next_task_fair
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] newidle_balance
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] load_balance
     0.01%     0.00%  kworker/59:1-mm  [kernel.kallsyms]   [k] find_busiest_group
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] __note_gp_changes
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] __remove_hrtimer
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] update_group_capacity
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] update_rq_clock
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] idle_cpu
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] cpuidle_reflect
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] irqentry_exit
     0.01%     0.00%  swapper          [kernel.kallsyms]   [k] hrtimer_update_next_event
     0.01%     0.00%  bin_sysbm        libc-2.31.so        [.] 0x00007f8851ef80b3
     0.01%     0.00%  bin_sysbm        bin_sysbm           [.] 0x000056303487cb61
     0.01%     0.00%  bin_sysbm        bin_sysbm           [.] 0x000056303487ca0f
     0.01%     0.00%  bin_sysbm        libpthread-2.31.so  [.] 0x00007f88520cd2d3
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] asm_exc_page_fault
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] exc_page_fault
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] do_user_addr_fault
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] handle_mm_fault
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __handle_mm_fault
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] lru_cache_add_inactive_or_unevictable
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] lru_cache_add
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __pagevec_lru_add
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __mod_lruvec_state
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] put_cpu_partial
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] rcu_eqs_enter.constprop.0
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] _raw_spin_trylock
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] get_cpu_device
     0.01%     0.01%  bin_sysbm        [kernel.kallsyms]   [k] native_apic_mem_write
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] rebalance_domains
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] __hrtimer_next_event_base
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] tick_check_broadcast_expired
     0.01%     0.01%  migration/85     [kernel.kallsyms]   [k] schedule
     0.01%     0.00%  migration/85     [kernel.kallsyms]   [k] ret_from_fork
     0.01%     0.00%  migration/85     [kernel.kallsyms]   [k] kthread
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] kvm_guest_apic_eoi_write
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] sched_idle_set_state
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __sysvec_apic_timer_interrupt
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] hrtimer_interrupt
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] tick_program_event
     0.01%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] clockevents_program_event
     0.01%     0.01%  swapper          [kernel.kallsyms]   [k] tick_nohz_tick_stopped
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_idle_enter
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] tick_nohz_idle_stop_tick
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] clear_buddies
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] pick_next_entity
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_segcblist_ready_cbs
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] get_nohz_timer_target
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] run_timer_softirq
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] __run_timers.part.0
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] call_timer_fn
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] blk_stat_timer_fn
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] wb_timer_fn
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rwb_arm_timer
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] mod_timer
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] note_gp_changes
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] rcu_core_si
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] rcu_core
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] tick_sched_do_timer
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] tick_do_update_jiffies64
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] update_wall_time
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] timekeeping_advance
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] acpi_processor_ffh_cstate_enter
     0.00%     0.00%  sshd             [kernel.kallsyms]   [k] sock_poll
     0.00%     0.00%  sshd             [unknown]           [k] 0000000000000000
     0.00%     0.00%  sshd             [unknown]           [k] 0x000055a63851e470
     0.00%     0.00%  sshd             libc-2.31.so        [.] 0x00007f661ce310da
     0.00%     0.00%  sshd             [kernel.kallsyms]   [k] entry_SYSCALL_64_after_hwframe
     0.00%     0.00%  sshd             [kernel.kallsyms]   [k] do_syscall_64
     0.00%     0.00%  sshd             [kernel.kallsyms]   [k] __x64_sys_select
     0.00%     0.00%  sshd             [kernel.kallsyms]   [k] kern_select
     0.00%     0.00%  sshd             [kernel.kallsyms]   [k] core_sys_select
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_dynticks_eqs_enter
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] __msecs_to_jiffies
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] syscall_enter_from_user_mode
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] cpuacct_charge
     0.00%     0.00%  bin_sysbm        libc-2.31.so        [.] 0x000000000010589b
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] entry_SYSCALL_64_safe_stack
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] fpregs_assert_state_consistent
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] cgroup_rstat_updated
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __cgroup_account_cputime
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] update_ts_time_stats
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] update_min_vruntime
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_advance_cbs
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_accelerate_cbs
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] __hrtimer_get_next_event
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] rcu_advance_cbs
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] error_return
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] asm_sysvec_call_function_single
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] sysvec_call_function_single
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __sysvec_call_function_single
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] pm_qos_read_value
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_segcblist_pend_cbs
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] __bitmap_and
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] update_group_capacity
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] group_balance_cpu
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] enqueue_hrtimer
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] propagate_protected_usage
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] local_touch_nmi
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] timerqueue_add
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __bitmap_and
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] __update_load_avg_cfs_rq
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_nmi_exit
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_read_unlock_strict
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rcu_segcblist_accelerate
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] __update_load_avg_se
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] finish_task_switch
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] trigger_load_balance
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] native_send_call_func_single_ipi
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] kick_ilb
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] smp_call_function_single_async
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] generic_exec_single
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] __smp_call_single_queue
     0.00%     0.00%  bin_sysbm        [kernel.kallsyms]   [k] ktime_get
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] rb_insert_color
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] run_posix_cpu_timers
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] profile_tick
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] nohz_balance_exit_idle
     0.00%     0.00%  swapper          [kernel.kallsyms]   [k] _raw_spin_lock_irq
     0.00%     0.00%  perf             libc-2.31.so        [.] 0x00007f4a016440b3
     0.00%     0.00%  perf             perf                [.] 0x0000559a782e6b7c
     0.00%     0.00%  perf             perf                [.] 0x0000559a783755ff
     0.00%     0.00%  perf             perf                [.] 0x0000559a78300622
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] entry_SYSCALL_64_after_hwframe
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] do_syscall_64
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] asm_sysvec_apic_timer_interrupt
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] sysvec_apic_timer_interrupt
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] __sysvec_apic_timer_interrupt
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] hrtimer_interrupt
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] nohz_balance_exit_idle
     0.00%     0.00%  perf             perf                [.] 0x0000559a7838aac4
     0.00%     0.00%  perf             libc-2.31.so        [.] 0x00007f4a017229cb
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] __x64_sys_sched_setaffinity
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] sched_setaffinity
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] __set_cpus_allowed_ptr
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] affine_move_task
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] _raw_spin_unlock_irqrestore
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] __hrtimer_run_queues
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] tick_sched_timer
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] tick_sched_handle.isra.0
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] update_process_times
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] scheduler_tick
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] trigger_load_balance
     0.00%     0.00%  perf             perf                [.] 0x0000559a7838ac1a
     0.00%     0.00%  perf             libc-2.31.so        [.] 0x00007f4a0173450b
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] __x64_sys_ioctl
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] perf_ioctl
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] _perf_ioctl
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] generic_exec_single
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] perf_event_for_each_child
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] _perf_event_enable
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] event_function_call
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] smp_call_function_single
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] native_apic_mem_write
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] tick_program_event
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] clockevents_program_event
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] syscall_exit_to_user_mode
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] mutex_unlock
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] put_ctx
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] __fget_light
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] fpregs_assert_state_consistent
     0.00%     0.00%  perf             [kernel.kallsyms]   [k] exit_to_user_mode_prepare
     0.00%     0.00%  perf             perf                [.] 0x000000000036dac2
     0.00%     0.00%  perf             perf                [.] 0x0000559a78491ac2
     0.00%     0.00%  perf             perf                [.] 0x0000000000266c33
     0.00%     0.00%  perf             perf                [.] 0x0000559a7838ac33


# Samples: 0  of event 'dummy:HG'
# Event count (approx.): 0
#
# Children      Self  Command  Shared Object  Symbol
# ........  ........  .......  .............  ......
#


#
# (Tip: List events using substring match: perf list <keyword>)
#

  parent reply	other threads:[~2021-04-19  7:56 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-09  1:14 [PATCH v3 0/4] kernfs: proposed locking and concurrency improvement Ian Kent
2021-04-09  1:14 ` [PATCH v3 1/4] kernfs: move revalidate to be near lookup Ian Kent
2021-04-09  3:39   ` kernel test robot
2021-04-09  3:39     ` kernel test robot
2021-04-09  1:15 ` [PATCH v3 2/4] kernfs: use VFS negative dentry caching Ian Kent
2021-04-09  1:35   ` Al Viro
2021-04-09  8:26     ` Ian Kent
2021-04-09  9:34       ` Ian Kent
2021-04-09  1:15 ` [PATCH v3 3/4] kernfs: switch kernfs to use an rwsem Ian Kent
2021-04-09  1:15 ` [PATCH v3 4/4] kernfs: use i_lock to protect concurrent inode updates Ian Kent
2021-04-19  7:56 ` Fox Chen [this message]
2021-04-19 12:25   ` [PATCH v3 0/4] kernfs: proposed locking and concurrency improvement Ian Kent

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAC2o3DKNc=sL2n8291Dpiyb0bRHaX=nd33ogvO_LkJqpBj-YmA@mail.gmail.com' \
    --to=foxhlchen@gmail.com \
    --cc=brice.goglin@gmail.com \
    --cc=dhowells@redhat.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=raven@themaw.net \
    --cc=ricklind@linux.vnet.ibm.com \
    --cc=sandeen@sandeen.net \
    --cc=tj@kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.