linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases
@ 2024-02-07 20:31 SeongJae Park
  2024-02-07 20:31 ` [PATCH 1/8] selftests/damon/_damon_sysfs: support DAMOS quota SeongJae Park
                   ` (7 more replies)
  0 siblings, 8 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Continue DAMON selftests' test coverage improvement works with a trivial
improvement of the test code itself.  The sequence of the patches in
patchset is as follows.

The first five patches add two DAMON core functionalities tests.  Those
begins with three patches (patches 1-3) that update the test-purpose
DAMON sysfs interface wrapper to support DAMOS quota, stats, and apply
interval features, respectively.  The fourth patch implements and adds a
selftest for DAMOS quota feature, using the DAMON sysfs interface
wrapper's newly added support of the quota and the stats feature.  The
fifth patch further implements and adds a selftest for DAMOS apply
interval using the DAMON sysfs interface wrapper's newly added support
of the apply interval and the stats feature.

Two patches (patches 6 and 7) for implementing and adding two corner
cases handling selftests follow.  Those try to avoid two previously
fixed bugs from recurring.

Finally, a patch for making DAMON debugfs selftests dependency checker
to use /proc/mounts instead of the hard-coded mount point assumption
follows.

SeongJae Park (8):
  selftests/damon/_damon_sysfs: support DAMOS quota
  selftests/damon/_damon_sysfs: support DAMOS stats
  selftests/damon/_damon_sysfs: support DAMOS apply interval
  selftests/damon: add a test for DAMOS quota
  selftests/damon: add a test for DAMOS apply intervals
  selftests/damon: add a test for a race between target_ids_read() and
    dbgfs_before_terminate()
  selftests/damon: add a test for the pid leak of
    dbgfs_target_ids_write()
  selftests/damon/_chk_dependency: get debugfs mount point from
    /proc/mounts

 tools/testing/selftests/damon/.gitignore      |  2 +
 tools/testing/selftests/damon/Makefile        |  5 ++
 .../selftests/damon/_chk_dependency.sh        |  9 ++-
 tools/testing/selftests/damon/_damon_sysfs.py | 77 ++++++++++++++++--
 .../selftests/damon/damos_apply_interval.py   | 67 ++++++++++++++++
 tools/testing/selftests/damon/damos_quota.py  | 67 ++++++++++++++++
 .../damon/debugfs_target_ids_pid_leak.c       | 68 ++++++++++++++++
 .../damon/debugfs_target_ids_pid_leak.sh      | 22 +++++
 ...fs_target_ids_read_before_terminate_race.c | 80 +++++++++++++++++++
 ...s_target_ids_read_before_terminate_race.sh | 14 ++++
 10 files changed, 403 insertions(+), 8 deletions(-)
 create mode 100755 tools/testing/selftests/damon/damos_apply_interval.py
 create mode 100755 tools/testing/selftests/damon/damos_quota.py
 create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c
 create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh
 create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c
 create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh


base-commit: f51e629727d8cc526a3156a2c80489b8f050410f
-- 
2.39.2


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 1/8] selftests/damon/_damon_sysfs: support DAMOS quota
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 2/8] selftests/damon/_damon_sysfs: support DAMOS stats SeongJae Park
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Update the test-purpose DAMON sysfs control Python module to support
DAMOS quota.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon_sysfs.py | 42 +++++++++++++++----
 1 file changed, 33 insertions(+), 9 deletions(-)

diff --git a/tools/testing/selftests/damon/_damon_sysfs.py b/tools/testing/selftests/damon/_damon_sysfs.py
index e98cf4b6a4b7..b4f6e385c564 100644
--- a/tools/testing/selftests/damon/_damon_sysfs.py
+++ b/tools/testing/selftests/damon/_damon_sysfs.py
@@ -70,18 +70,48 @@ class DamosAccessPattern:
         if err != None:
             return err
 
+class DamosQuota:
+    sz = None                   # size quota, in bytes
+    ms = None                   # time quota
+    reset_interval_ms = None    # quota reset interval
+    scheme = None               # owner scheme
+
+    def __init__(self, sz=0, ms=0, reset_interval_ms=0):
+        self.sz = sz
+        self.ms = ms
+        self.reset_interval_ms = reset_interval_ms
+
+    def sysfs_dir(self):
+        return os.path.join(self.scheme.sysfs_dir(), 'quotas')
+
+    def stage(self):
+        err = write_file(os.path.join(self.sysfs_dir(), 'bytes'), self.sz)
+        if err != None:
+            return err
+        err = write_file(os.path.join(self.sysfs_dir(), 'ms'), self.ms)
+        if err != None:
+            return err
+        err = write_file(os.path.join(self.sysfs_dir(), 'reset_interval_ms'),
+                         self.reset_interval_ms)
+        if err != None:
+            return err
+
 class Damos:
     action = None
     access_pattern = None
-    # todo: Support quotas, watermarks, stats, tried_regions
+    quota = None
+    # todo: Support watermarks, stats, tried_regions
     idx = None
     context = None
     tried_bytes = None
 
-    def __init__(self, action='stat', access_pattern=DamosAccessPattern()):
+    def __init__(self, action='stat', access_pattern=DamosAccessPattern(),
+                 quota=DamosQuota()):
         self.action = action
         self.access_pattern = access_pattern
         self.access_pattern.scheme = self
+        self.quota = quota
+        self.quota.scheme = self
 
     def sysfs_dir(self):
         return os.path.join(
@@ -94,13 +124,7 @@ class Damos:
         err = self.access_pattern.stage()
         if err != None:
             return err
-
-        # disable quotas
-        err = write_file(os.path.join(self.sysfs_dir(), 'quotas', 'ms'), '0')
-        if err != None:
-            return err
-        err = write_file(
-                os.path.join(self.sysfs_dir(), 'quotas', 'bytes'), '0')
+        err = self.quota.stage()
         if err != None:
             return err
 
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 2/8] selftests/damon/_damon_sysfs: support DAMOS stats
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
  2024-02-07 20:31 ` [PATCH 1/8] selftests/damon/_damon_sysfs: support DAMOS quota SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 3/8] selftests/damon/_damon_sysfs: support DAMOS apply interval SeongJae Park
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Update the test-purpose DAMON sysfs control Python module to support
DAMOS stats.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon_sysfs.py | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/tools/testing/selftests/damon/_damon_sysfs.py b/tools/testing/selftests/damon/_damon_sysfs.py
index b4f6e385c564..a75244451684 100644
--- a/tools/testing/selftests/damon/_damon_sysfs.py
+++ b/tools/testing/selftests/damon/_damon_sysfs.py
@@ -96,6 +96,20 @@ class DamosQuota:
         if err != None:
             return err
 
+class DamosStats:
+    nr_tried = None
+    sz_tried = None
+    nr_applied = None
+    sz_applied = None
+    qt_exceeds = None
+
+    def __init__(self, nr_tried, sz_tried, nr_applied, sz_applied, qt_exceeds):
+        self.nr_tried = nr_tried
+        self.sz_tried = sz_tried
+        self.nr_applied = nr_applied
+        self.sz_applied = sz_applied
+        self.qt_exceeds = qt_exceeds
+
 class Damos:
     action = None
     access_pattern = None
@@ -104,6 +118,7 @@ class Damos:
     idx = None
     context = None
     tried_bytes = None
+    stats = None
 
     def __init__(self, action='stat', access_pattern=DamosAccessPattern(),
                  quota=DamosQuota()):
@@ -322,6 +337,23 @@ class Kdamond:
                     return err
                 scheme.tried_bytes = int(content)
 
+    def update_schemes_stats(self):
+        err = write_file(os.path.join(self.sysfs_dir(), 'state'),
+                'update_schemes_stats')
+        if err != None:
+            return err
+        for context in self.contexts:
+            for scheme in context.schemes:
+                stat_values = []
+                for stat in ['nr_tried', 'sz_tried', 'nr_applied',
+                             'sz_applied', 'qt_exceeds']:
+                    content, err = read_file(
+                            os.path.join(scheme.sysfs_dir(), 'stats', stat))
+                    if err != None:
+                        return err
+                    stat_values.append(int(content))
+                scheme.stats = DamosStats(*stat_values)
+
 class Kdamonds:
     kdamonds = []
 
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 3/8] selftests/damon/_damon_sysfs: support DAMOS apply interval
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
  2024-02-07 20:31 ` [PATCH 1/8] selftests/damon/_damon_sysfs: support DAMOS quota SeongJae Park
  2024-02-07 20:31 ` [PATCH 2/8] selftests/damon/_damon_sysfs: support DAMOS stats SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 4/8] selftests/damon: add a test for DAMOS quota SeongJae Park
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Update the test-purpose DAMON sysfs control Python module to support
DAMOS apply interval.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_damon_sysfs.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/damon/_damon_sysfs.py b/tools/testing/selftests/damon/_damon_sysfs.py
index a75244451684..d23d7398a27a 100644
--- a/tools/testing/selftests/damon/_damon_sysfs.py
+++ b/tools/testing/selftests/damon/_damon_sysfs.py
@@ -114,6 +114,7 @@ class Damos:
     action = None
     access_pattern = None
     quota = None
+    apply_interval_us = None
     # todo: Support watermarks, stats, tried_regions
     idx = None
     context = None
@@ -121,12 +122,13 @@ class Damos:
     stats = None
 
     def __init__(self, action='stat', access_pattern=DamosAccessPattern(),
-                 quota=DamosQuota()):
+                 quota=DamosQuota(), apply_interval_us=0):
         self.action = action
         self.access_pattern = access_pattern
         self.access_pattern.scheme = self
         self.quota = quota
         self.quota.scheme = self
+        self.apply_interval_us = apply_interval_us
 
     def sysfs_dir(self):
         return os.path.join(
@@ -139,6 +141,11 @@ class Damos:
         err = self.access_pattern.stage()
         if err != None:
             return err
+        err = write_file(os.path.join(self.sysfs_dir(), 'apply_interval_us'),
+                         '%d' % self.apply_interval_us)
+        if err != None:
+            return err
+
         err = self.quota.stage()
         if err != None:
             return err
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 4/8] selftests/damon: add a test for DAMOS quota
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
                   ` (2 preceding siblings ...)
  2024-02-07 20:31 ` [PATCH 3/8] selftests/damon/_damon_sysfs: support DAMOS apply interval SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 5/8] selftests/damon: add a test for DAMOS apply intervals SeongJae Park
                   ` (3 subsequent siblings)
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Add a selftest for verifying the DAMOS quota feature.  The test is very
similar to sysfs_update_schemes_tried_regions_wss_estimation.py.  It
starts an artificial workload of 20 MiB working set, run DAMON to find
the working set size, but with 1 MiB/100 ms size quota.  Then, it
collect the DAMON-found working set size every 100 ms and check if the
quota was always applied as expected.  For the confirmation, the tests
shows the stat-applied region size and the qt_exceeds stat.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/Makefile       |  1 +
 tools/testing/selftests/damon/damos_quota.py | 67 ++++++++++++++++++++
 2 files changed, 68 insertions(+)
 create mode 100755 tools/testing/selftests/damon/damos_quota.py

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 8a1cc2bf1864..9c3783f1a39d 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -12,6 +12,7 @@ TEST_PROGS += debugfs_rm_non_contexts.sh
 TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
 TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py
 TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
+TEST_PROGS += damos_quota.py
 TEST_PROGS += reclaim.sh lru_sort.sh
 
 include ../lib.mk
diff --git a/tools/testing/selftests/damon/damos_quota.py b/tools/testing/selftests/damon/damos_quota.py
new file mode 100755
index 000000000000..7d4c6bb2e3cd
--- /dev/null
+++ b/tools/testing/selftests/damon/damos_quota.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+import subprocess
+import time
+
+import _damon_sysfs
+
+def main():
+    # access two 10 MiB memory regions, 2 second per each
+    sz_region = 10 * 1024 * 1024
+    proc = subprocess.Popen(['./access_memory', '2', '%d' % sz_region, '2000'])
+
+    # Set quota up to 1 MiB per 100 ms
+    sz_quota = 1024 * 1024 # 1 MiB
+    quota_reset_interval = 100 # 100 ms
+    kdamonds = _damon_sysfs.Kdamonds([_damon_sysfs.Kdamond(
+            contexts=[_damon_sysfs.DamonCtx(
+                ops='vaddr',
+                targets=[_damon_sysfs.DamonTarget(pid=proc.pid)],
+                schemes=[_damon_sysfs.Damos(
+                    access_pattern=_damon_sysfs.DamosAccessPattern(
+                        # >= 25% access rate, >= 200ms age
+                        nr_accesses=[5, 20], age=[2, 2**64 - 1]),
+                    quota=_damon_sysfs.DamosQuota(
+                        sz=sz_quota, reset_interval_ms=quota_reset_interval)
+                    )] # schemes
+                )] # contexts
+            )]) # kdamonds
+
+    err = kdamonds.start()
+    if err != None:
+        print('kdamond start failed: %s' % err)
+        exit(1)
+
+    wss_collected = []
+    nr_quota_exceeds = 0
+    while proc.poll() == None:
+        time.sleep(0.1)
+        err = kdamonds.kdamonds[0].update_schemes_tried_bytes()
+        if err != None:
+            print('tried bytes update failed: %s' % err)
+            exit(1)
+        err = kdamonds.kdamonds[0].update_schemes_stats()
+        if err != None:
+            print('stats update failed: %s' % err)
+            exit(1)
+
+        scheme = kdamonds.kdamonds[0].contexts[0].schemes[0]
+        wss_collected.append(scheme.tried_bytes)
+        nr_quota_exceeds = scheme.stats.qt_exceeds
+
+    wss_collected.sort()
+    for wss in wss_collected:
+        if wss > sz_quota:
+            print('quota is not kept: %s > %s' % (wss, sz_quota))
+            print('collected samples are as below')
+            print('\n'.join(['%d' % wss for wss in wss_collected]))
+            exit(1)
+
+    if nr_quota_exceeds < len(wss_collected):
+        print('quota is not always exceeded: %d > %d' %
+              (len(wss_collected), nr_quota_exceeds))
+        exit(1)
+
+if __name__ == '__main__':
+    main()
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 5/8] selftests/damon: add a test for DAMOS apply intervals
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
                   ` (3 preceding siblings ...)
  2024-02-07 20:31 ` [PATCH 4/8] selftests/damon: add a test for DAMOS quota SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 6/8] selftests/damon: add a test for a race between target_ids_read() and dbgfs_before_terminate() SeongJae Park
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Add a selftest for DAMOS apply intervals.  It runs two schemes having
different apply interval agains an artificial memory access workload,
and check if the scheme with smaller apply interval was applied more
frequently.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/Makefile        |  2 +-
 .../selftests/damon/damos_apply_interval.py   | 67 +++++++++++++++++++
 2 files changed, 68 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/damon/damos_apply_interval.py

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 9c3783f1a39d..b545fedafb3b 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -12,7 +12,7 @@ TEST_PROGS += debugfs_rm_non_contexts.sh
 TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
 TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py
 TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
-TEST_PROGS += damos_quota.py
+TEST_PROGS += damos_quota.py damos_apply_interval.py
 TEST_PROGS += reclaim.sh lru_sort.sh
 
 include ../lib.mk
diff --git a/tools/testing/selftests/damon/damos_apply_interval.py b/tools/testing/selftests/damon/damos_apply_interval.py
new file mode 100755
index 000000000000..f04d43702481
--- /dev/null
+++ b/tools/testing/selftests/damon/damos_apply_interval.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+import subprocess
+import time
+
+import _damon_sysfs
+
+def main():
+    # access two 10 MiB memory regions, 2 second per each
+    sz_region = 10 * 1024 * 1024
+    proc = subprocess.Popen(['./access_memory', '2', '%d' % sz_region, '2000'])
+
+    # Set quota up to 1 MiB per 100 ms
+    kdamonds = _damon_sysfs.Kdamonds([_damon_sysfs.Kdamond(
+            contexts=[_damon_sysfs.DamonCtx(
+                ops='vaddr',
+                targets=[_damon_sysfs.DamonTarget(pid=proc.pid)],
+                schemes=[
+                    _damon_sysfs.Damos(
+                        access_pattern=_damon_sysfs.DamosAccessPattern(
+                            # >= 25% access rate, >= 200ms age
+                            nr_accesses=[5, 20], age=[2, 2**64 - 1]),
+                        # aggregation interval (100 ms) is used
+                        apply_interval_us=0),
+                    # use 10ms apply interval
+                    _damon_sysfs.Damos(
+                        access_pattern=_damon_sysfs.DamosAccessPattern(
+                            # >= 25% access rate, >= 200ms age
+                            nr_accesses=[5, 20], age=[2, 2**64 - 1]),
+                        # explicitly set 10 ms apply interval
+                        apply_interval_us=10 * 1000)
+                    ] # schemes
+                )] # contexts
+            )]) # kdamonds
+
+    err = kdamonds.start()
+    if err != None:
+        print('kdamond start failed: %s' % err)
+        exit(1)
+
+    wss_collected = []
+    nr_quota_exceeds = 0
+    while proc.poll() == None:
+        time.sleep(0.1)
+        err = kdamonds.kdamonds[0].update_schemes_stats()
+        if err != None:
+            print('stats update failed: %s' % err)
+            exit(1)
+    schemes = kdamonds.kdamonds[0].contexts[0].schemes
+    nr_tried_stats = [s.stats.nr_tried for s in schemes]
+    if nr_tried_stats[0] == 0 or nr_tried_stats[1] == 0:
+        print('scheme(s) are not tried')
+        exit(1)
+
+    # Because the second scheme was having the apply interval that is ten times
+    # lower than that of the first scheme, the second scheme should be tried
+    # about ten times more frequently than the first scheme.  For possible
+    # timing errors, check if it was at least nine times more freuqnetly tried.
+    ratio = nr_tried_stats[1] / nr_tried_stats[0]
+    if ratio < 9:
+        print('%d / %d = %f (< 9)' %
+              (nr_tried_stats[1], nr_tried_stats[0], ratio))
+        exit(1)
+
+if __name__ == '__main__':
+    main()
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 6/8] selftests/damon: add a test for a race between target_ids_read() and dbgfs_before_terminate()
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
                   ` (4 preceding siblings ...)
  2024-02-07 20:31 ` [PATCH 5/8] selftests/damon: add a test for DAMOS apply intervals SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 7/8] selftests/damon: add a test for the pid leak of dbgfs_target_ids_write() SeongJae Park
  2024-02-07 20:31 ` [PATCH 8/8] selftests/damon/_chk_dependency: get debugfs mount point from /proc/mounts SeongJae Park
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

commit 34796417964b ("mm/damon/dbgfs: protect targets destructions with
kdamond_lock") fixed a race of DAMON debugfs interface.  Specifically,
the race was happening between target_ids_read() and
dbgfs_before_terminate().  Add a test for the issue to prevent the
problem from accidentally recurring.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/.gitignore      |  1 +
 tools/testing/selftests/damon/Makefile        |  2 +
 ...fs_target_ids_read_before_terminate_race.c | 80 +++++++++++++++++++
 ...s_target_ids_read_before_terminate_race.sh | 14 ++++
 4 files changed, 97 insertions(+)
 create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c
 create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh

diff --git a/tools/testing/selftests/damon/.gitignore b/tools/testing/selftests/damon/.gitignore
index c6c2965a6607..7d6c6e062be7 100644
--- a/tools/testing/selftests/damon/.gitignore
+++ b/tools/testing/selftests/damon/.gitignore
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0-only
 huge_count_read_write
+debugfs_target_ids_read_before_terminate_race
diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index b545fedafb3b..8a3a8df003db 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -2,6 +2,7 @@
 # Makefile for damon selftests
 
 TEST_GEN_FILES += huge_count_read_write
+TEST_GEN_FILES += debugfs_target_ids_read_before_terminate_race
 TEST_GEN_FILES += access_memory
 
 TEST_FILES = _chk_dependency.sh _debugfs_common.sh
@@ -9,6 +10,7 @@ TEST_PROGS = debugfs_attrs.sh debugfs_schemes.sh debugfs_target_ids.sh
 TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh
 TEST_PROGS += debugfs_duplicate_context_creation.sh
 TEST_PROGS += debugfs_rm_non_contexts.sh
+TEST_PROGS += debugfs_target_ids_read_before_terminate_race.sh
 TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
 TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py
 TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
diff --git a/tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c b/tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c
new file mode 100644
index 000000000000..b06f52a8ce2d
--- /dev/null
+++ b/tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: SeongJae Park <sj@kernel.org>
+ */
+#define _GNU_SOURCE
+
+#include <fcntl.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <time.h>
+#include <unistd.h>
+
+#define DBGFS_MONITOR_ON "/sys/kernel/debug/damon/monitor_on_DEPRECATED"
+#define DBGFS_TARGET_IDS "/sys/kernel/debug/damon/target_ids"
+
+static void turn_damon_on_exit(void)
+{
+	int target_ids_fd = open(DBGFS_TARGET_IDS, O_RDWR);
+	int monitor_on_fd = open(DBGFS_MONITOR_ON, O_RDWR);
+	char pid_str[128];
+
+	snprintf(pid_str, sizeof(pid_str), "%d", getpid());
+	write(target_ids_fd, pid_str, sizeof(pid_str));
+	write(monitor_on_fd, "on\n", 3);
+	close(target_ids_fd);
+	close(monitor_on_fd);
+	usleep(1000);
+	exit(0);
+}
+
+static void try_race(void)
+{
+	int target_ids_fd = open(DBGFS_TARGET_IDS, O_RDWR);
+	int pid = fork();
+	int buf[256];
+
+	if (pid < 0) {
+		fprintf(stderr, "fork() failed\n");
+		exit(1);
+	}
+	if (pid == 0)
+		turn_damon_on_exit();
+	while (true) {
+		int status;
+
+		read(target_ids_fd, buf, sizeof(buf));
+		if (waitpid(-1, &status, WNOHANG) == pid)
+			break;
+	}
+	close(target_ids_fd);
+}
+
+static inline uint64_t ts_to_ms(struct timespec *ts)
+{
+	return (uint64_t)ts->tv_sec * 1000 + (uint64_t)ts->tv_nsec / 1000000;
+}
+
+int main(int argc, char *argv[])
+{
+	struct timespec start_time, now;
+	int runtime_ms;
+
+	if (argc != 2) {
+		fprintf(stderr, "Usage: %s <runtime in ms>\n", argv[0]);
+		exit(1);
+	}
+	runtime_ms = atoi(argv[1]);
+	clock_gettime(CLOCK_MONOTONIC, &start_time);
+	while (true) {
+		try_race();
+		clock_gettime(CLOCK_MONOTONIC, &now);
+		if (ts_to_ms(&now) - ts_to_ms(&start_time) > runtime_ms)
+			break;
+	}
+	return 0;
+}
diff --git a/tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh b/tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh
new file mode 100755
index 000000000000..fc793c4c9aea
--- /dev/null
+++ b/tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+dmesg -C
+
+./debugfs_target_ids_read_before_terminate_race 5000
+
+if dmesg | grep -q dbgfs_target_ids_read
+then
+	dmesg
+	exit 1
+else
+	exit 0
+fi
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 7/8] selftests/damon: add a test for the pid leak of dbgfs_target_ids_write()
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
                   ` (5 preceding siblings ...)
  2024-02-07 20:31 ` [PATCH 6/8] selftests/damon: add a test for a race between target_ids_read() and dbgfs_before_terminate() SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  2024-02-07 20:31 ` [PATCH 8/8] selftests/damon/_chk_dependency: get debugfs mount point from /proc/mounts SeongJae Park
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

Commit ebb3f994dd92 ("mm/damon/dbgfs: fix 'struct pid' leaks in
'dbgfs_target_ids_write()'") fixes a pid leak bug in DAMON debugfs
interface, namely dbgfs_target_ids_write() function.  Add a selftest for
the issue to prevent the problem from mistakenly recurring.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/.gitignore      |  1 +
 tools/testing/selftests/damon/Makefile        |  2 +
 .../damon/debugfs_target_ids_pid_leak.c       | 68 +++++++++++++++++++
 .../damon/debugfs_target_ids_pid_leak.sh      | 22 ++++++
 4 files changed, 93 insertions(+)
 create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c
 create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh

diff --git a/tools/testing/selftests/damon/.gitignore b/tools/testing/selftests/damon/.gitignore
index 7d6c6e062be7..d861701f0327 100644
--- a/tools/testing/selftests/damon/.gitignore
+++ b/tools/testing/selftests/damon/.gitignore
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0-only
 huge_count_read_write
 debugfs_target_ids_read_before_terminate_race
+debugfs_target_ids_pid_leak
diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 8a3a8df003db..789d6949c247 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -3,6 +3,7 @@
 
 TEST_GEN_FILES += huge_count_read_write
 TEST_GEN_FILES += debugfs_target_ids_read_before_terminate_race
+TEST_GEN_FILES += debugfs_target_ids_pid_leak
 TEST_GEN_FILES += access_memory
 
 TEST_FILES = _chk_dependency.sh _debugfs_common.sh
@@ -11,6 +12,7 @@ TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh
 TEST_PROGS += debugfs_duplicate_context_creation.sh
 TEST_PROGS += debugfs_rm_non_contexts.sh
 TEST_PROGS += debugfs_target_ids_read_before_terminate_race.sh
+TEST_PROGS += debugfs_target_ids_pid_leak.sh
 TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh
 TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py
 TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
diff --git a/tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c b/tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c
new file mode 100644
index 000000000000..0cc2eef7d142
--- /dev/null
+++ b/tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c
@@ -0,0 +1,68 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Author: SeongJae Park <sj@kernel.org>
+ */
+
+#define _GNU_SOURCE
+
+#include <fcntl.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/time.h>
+#include <unistd.h>
+
+#define DBGFS_TARGET_IDS "/sys/kernel/debug/damon/target_ids"
+
+static void write_targetid_exit(void)
+{
+	int target_ids_fd = open(DBGFS_TARGET_IDS, O_RDWR);
+	char pid_str[128];
+
+	snprintf(pid_str, sizeof(pid_str), "%d", getpid());
+	write(target_ids_fd, pid_str, sizeof(pid_str));
+	close(target_ids_fd);
+	exit(0);
+}
+
+unsigned long msec_timestamp(void)
+{
+	struct timeval tv;
+
+	gettimeofday(&tv, NULL);
+	return tv.tv_sec * 1000UL + tv.tv_usec / 1000;
+}
+
+int main(int argc, char *argv[])
+{
+	unsigned long start_ms;
+	int time_to_run, nr_forks = 0;
+
+	if (argc != 2) {
+		fprintf(stderr, "Usage: %s <msecs to run>\n", argv[0]);
+		exit(1);
+	}
+	time_to_run = atoi(argv[1]);
+
+	start_ms = msec_timestamp();
+	while (true) {
+		int pid = fork();
+
+		if (pid < 0) {
+			fprintf(stderr, "fork() failed\n");
+			exit(1);
+		}
+		if (pid == 0)
+			write_targetid_exit();
+		wait(NULL);
+		nr_forks++;
+
+		if (msec_timestamp() - start_ms > time_to_run)
+			break;
+	}
+	printf("%d\n", nr_forks);
+	return 0;
+}
diff --git a/tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh b/tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh
new file mode 100755
index 000000000000..31fe33c2b032
--- /dev/null
+++ b/tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+before=$(grep "^pid " /proc/slabinfo | awk '{print $2}')
+
+nr_leaks=$(./debugfs_target_ids_pid_leak 1000)
+expected_after_max=$((before + nr_leaks / 2))
+
+after=$(grep "^pid " /proc/slabinfo | awk '{print $2}')
+
+echo > /sys/kernel/debug/damon/target_ids
+
+echo "tried $nr_leaks pid leak"
+echo "number of active pid slabs: $before -> $after"
+echo "(up to $expected_after_max expected)"
+if [ $after -gt $expected_after_max ]
+then
+	echo "maybe pids are leaking"
+	exit 1
+else
+	exit 0
+fi
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 8/8] selftests/damon/_chk_dependency: get debugfs mount point from /proc/mounts
  2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
                   ` (6 preceding siblings ...)
  2024-02-07 20:31 ` [PATCH 7/8] selftests/damon: add a test for the pid leak of dbgfs_target_ids_write() SeongJae Park
@ 2024-02-07 20:31 ` SeongJae Park
  7 siblings, 0 replies; 9+ messages in thread
From: SeongJae Park @ 2024-02-07 20:31 UTC (permalink / raw)
  To: Andrew Morton
  Cc: SeongJae Park, Shuah Khan, damon, linux-mm, linux-kselftest,
	linux-kernel

DAMON debugfs selftests dependency checker assumes debugfs would be
mounted at /sys/kernel/debug.  That would be ok for many cases, but some
systems might mounted the file system on some different places.  Parse
the real mount point using /proc/mounts file.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 tools/testing/selftests/damon/_chk_dependency.sh | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/damon/_chk_dependency.sh b/tools/testing/selftests/damon/_chk_dependency.sh
index 350f8c2b071d..dda3a87dc00a 100644
--- a/tools/testing/selftests/damon/_chk_dependency.sh
+++ b/tools/testing/selftests/damon/_chk_dependency.sh
@@ -4,7 +4,14 @@
 # Kselftest framework requirement - SKIP code is 4.
 ksft_skip=4
 
-DBGFS=/sys/kernel/debug/damon
+DBGFS=$(grep debugfs /proc/mounts --max-count 1 | awk '{print $2}')
+if [ "$DBGFS" = "" ]
+then
+	echo "debugfs not mounted"
+	exit $ksft_skip
+fi
+
+DBGFS+="/damon"
 
 if [ $EUID -ne 0 ];
 then
-- 
2.39.2


^ permalink raw reply related	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2024-02-07 20:31 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-07 20:31 [PATCH 0/8] selftests/damon: add more tests for core functionalities and corner cases SeongJae Park
2024-02-07 20:31 ` [PATCH 1/8] selftests/damon/_damon_sysfs: support DAMOS quota SeongJae Park
2024-02-07 20:31 ` [PATCH 2/8] selftests/damon/_damon_sysfs: support DAMOS stats SeongJae Park
2024-02-07 20:31 ` [PATCH 3/8] selftests/damon/_damon_sysfs: support DAMOS apply interval SeongJae Park
2024-02-07 20:31 ` [PATCH 4/8] selftests/damon: add a test for DAMOS quota SeongJae Park
2024-02-07 20:31 ` [PATCH 5/8] selftests/damon: add a test for DAMOS apply intervals SeongJae Park
2024-02-07 20:31 ` [PATCH 6/8] selftests/damon: add a test for a race between target_ids_read() and dbgfs_before_terminate() SeongJae Park
2024-02-07 20:31 ` [PATCH 7/8] selftests/damon: add a test for the pid leak of dbgfs_target_ids_write() SeongJae Park
2024-02-07 20:31 ` [PATCH 8/8] selftests/damon/_chk_dependency: get debugfs mount point from /proc/mounts SeongJae Park

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).