[PATCH 0/3 RFC] fsperf: enhancements supporting read policy benchmark

[PATCH 0/3 RFC] fsperf: enhancements supporting read policy benchmark

[Anand Jain]

I am sending this early to seek feedback on the best way to benchmark 
more than one read policy.

Patch 1,2 adds helpers to support a new setup.
Patch 3 adds a generic readonly dio fio benchmark script.

How to run:
Manage the read policy in the config file as required.

For example:

$ cat local.cfg
    [main]
    directory=/mnt/test

    [btrfs-pid]
    device=/dev/vg/scratch0
    mkfs=mkfs.btrfs -f -draid1 -mraid1 /dev/vg/scratch1
    mount=mount -o noatime
    readpolicy=pid
    
    [btrfs-new-policy]
    device=/dev/vg/scratch0
    mkfs=mkfs.btrfs -f -draid1 -mraid1 /dev/vg/scratch1
    mount=mount -o noatime
    readpolicy=new-policy

$ ./fsperf -c btrfs-pid diorandread
snip

$ ./fsperf -c btrfs-new-policy -t -C btrfs-pid diorandread
::
btrfs-new-policy test results
diorandread results
     metric         baseline   current    stdev         diff      
==================================================================
read_lat_ns_max     1.36e+08   1.09e+08       0   -20.02%
write_iops                 0          0       0     0.00%
read_clat_ns_p50     2277376    2244608       0    -1.44%
write_io_kbytes            0          0       0     0.00%
read_clat_ns_p99    12910592   11862016       0    -8.12%
write_bw_bytes             0          0       0     0.00%
read_iops            5680.85    5848.49       0     2.95%
write_clat_ns_p50          0          0       0     0.00%
read_io_bytes       1.40e+09   1.44e+09       0     2.95%
read_io_kbytes       1363496    1403708       0     2.95%
write_clat_ns_p99          0          0       0     0.00%
elapsed                   61         61       0     0.00%
read_bw_bytes       23268780   23955418       0     2.95%
sys_cpu                12.42      13.41       0     7.93%
write_lat_ns_min           0          0       0     0.00%
read_lat_ns_min       164944     189882       0    15.12%
write_lat_ns_max           0          0       0     0.00%

Anand Jain (3):
  fsperf: add a few helper functions
  fsperf: get the running section in the setup
  fsperf: add a new test case diorandread

 local-cfg-example        |  6 ++++++
 src/fsperf.py            |  2 +-
 src/utils.py             | 45 ++++++++++++++++++++++++++++++++++++++++
 tests/dio-randread.py    | 22 ++++++++++++++++++++
 tests/randwrite-2xram.py |  2 +-
 tests/untar-firefox.py   |  2 +-
 6 files changed, 76 insertions(+), 3 deletions(-)
 create mode 100644 tests/dio-randread.py

-- 
2.33.1

[PATCH 1/3] fsperf: add a few helper functions

[Anand Jain]

This is in preparation to add read policy benchmarking add the following
helpes
	get_fstype
	get_fsid
	get_readpolicies
	get_active_readpolicy
	set_readpolicy

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 src/utils.py | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/src/utils.py b/src/utils.py
index 5629894ce1fb..a7f0f1baf448 100644
--- a/src/utils.py
+++ b/src/utils.py
@@ -8,6 +8,8 @@ import itertools
 import numbers
 import datetime
 import statistics
+import subprocess
+import re
 
 LOWER_IS_BETTER = 0
 HIGHER_IS_BETTER = 1
@@ -230,3 +232,46 @@ def print_comparison_table(baseline, results):
         table_rows.append(cur)
     table.add_rows(table_rows)
     print(table.draw())
+
+def get_fstype(device):
+    fstype = subprocess.check_output("blkid -s TYPE -o value "+device, shell=True)
+    # strip the output b'btrfs\n'
+    return (str(fstype).removesuffix("\\n'")).removeprefix("b'")
+
+def get_fsid(device):
+    fsid = subprocess.check_output("blkid -s UUID -o value "+device, shell=True)
+    # Raw output is something like this
+    #    b'abcf123f-7e95-40cd-8322-0d32773cb4ec\n'
+    # strip off extra characters.
+    return str(fsid)[2:38]
+
+def get_readpolicies(device):
+    fsid = get_fsid(device)
+    sysfs = open("/sys/fs/btrfs/"+fsid+"/read_policy", "r")
+    # Strip '[ ]' around the active policy
+    policies = (((sysfs.read()).strip()).strip("[")).strip("]")
+    sysfs.close()
+    return policies
+
+def get_active_readpolicy(device):
+    fsid = get_fsid(device)
+    sysfs = open("/sys/fs/btrfs/"+fsid+"/read_policy", "r")
+    policies = (sysfs.read()).strip()
+    # Output is as below, pick the policy within '[ ]'
+    #   device [pid] latency
+    active = re.search(r"\[([A-Za-z0-9_]+)\]", policies)
+    sysfs.close()
+    return active.group(1)
+
+def set_readpolicy(device, policy="pid"):
+    if not policy in get_readpolicies(device):
+        print("Policy '{}' is invalid".format(policy))
+        sys.exit(1)
+        return
+    fsid = get_fsid(device)
+    # Ran out of ideas why run_command fails.
+    # command = "echo "+policy+" > /sys/fs/btrfs/"+fsid+"/read_policy"
+    # run_command(command)
+    sysfs = open("/sys/fs/btrfs/"+fsid+"/read_policy", "w")
+    ret = sysfs.write(policy)
+    sysfs.close()
-- 
2.33.1

[PATCH 3/3] fsperf: add a new test case diorandread

[Anand Jain]

The test case 'diorandread' is a readonly direct io test case using fio.
The primary aim of this test case is to benchmark read policies.

An optional read policy parameter can be specified in the local.cfg file
as shown below
  readpolicy=pid

When the readpolicy specified in the local.cfg file, the setup() in the
test case using it shall validate if the fstype is btrfs.

To compare the readpolicies the developer can run './fsperf diorandread'
with the supported read policies under a section. As shown below.

For example:

$ cat local.fg
::
  [btrfs-pid]
  device=/dev/vg/scratch0
  mkfs=mkfs.btrfs -f -draid1 -mraid1 /dev/vg/scratch1
  mount=mount -o noatime
  readpolicy=pid

  [btrfs-latency]
  device=/dev/vg/scratch0
  mkfs=mkfs.btrfs -f -draid1 -mraid1 /dev/vg/scratch1
  mount=mount -o noatime
  readpolicy=latency

$ ./fsperf -c btrfs-pid diorandread
$ ./fsperf -c btrfs-latency -t -C btrfs-pid diorandread

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 local-cfg-example     |  6 ++++++
 tests/dio-randread.py | 22 ++++++++++++++++++++++
 2 files changed, 28 insertions(+)
 create mode 100644 tests/dio-randread.py

diff --git a/local-cfg-example b/local-cfg-example
index a9034f98fbee..4b7fd0703feb 100644
--- a/local-cfg-example
+++ b/local-cfg-example
@@ -6,6 +6,12 @@ device=/dev/nvme0n1
 mkfs=mkfs.btrfs -f
 mount=mount -o noatime
 
+[btrfs-raid1]
+device=/dev/vg/scratch0
+mkfs=mkfs.btrfs -f -draid1 -mraid1 /dev/vg/scratch1
+mount=mount -o noatime
+readpolicy=pid
+
 [xfs]
 device=/dev/nvme0n1
 iosched=none
diff --git a/tests/dio-randread.py b/tests/dio-randread.py
new file mode 100644
index 000000000000..05ced875bb6d
--- /dev/null
+++ b/tests/dio-randread.py
@@ -0,0 +1,22 @@
+from PerfTest import FioTest
+from utils import get_fstype
+from utils import set_readpolicy
+from utils import get_active_readpolicy
+
+class DioRandread(FioTest):
+    name = "diorandread"
+    command = ("--name diorandread --direct=1 --size=1g --rw=randread "
+               "--runtime=60 --iodepth=1024 --nrfiles=16 "
+               "--numjobs=16 --group_reporting")
+
+    def setup(self, config, section):
+        fstype = get_fstype(config.get(section, 'device'))
+
+        if not fstype == "btrfs":
+            return
+
+        if config.has_option(section, 'readpolicy'):
+            policy = config.get(section, 'readpolicy')
+            set_readpolicy(config.get(section, 'device'), policy)
+            policy = get_active_readpolicy(config.get(section, 'device'))
+            print("\tReadpolicy is set to '{}'".format(policy))
-- 
2.33.1

Re: [PATCH 0/3 RFC] fsperf: enhancements supporting read policy benchmark

[Josef Bacik]

On Fri, Nov 12, 2021 at 04:04:36PM +0800, Anand Jain wrote:
> I am sending this early to seek feedback on the best way to benchmark 
> more than one read policy.
> 
> Patch 1,2 adds helpers to support a new setup.
> Patch 3 adds a generic readonly dio fio benchmark script.
> 

This looks reasonable to me, the only thing is I'd want a way for the setup()
part of the test to see that we don't have readpolicy support and simply skip.
Right now there's no mechanism for skipping a test, so add that and it'll be
good.

Also you can just do PR's through github if you want.  Thanks,

Josef

Re: [PATCH 0/3 RFC] fsperf: enhancements supporting read policy benchmark

[Anand Jain]

On 12/11/2021 22:34, Josef Bacik wrote:
> On Fri, Nov 12, 2021 at 04:04:36PM +0800, Anand Jain wrote:
>> I am sending this early to seek feedback on the best way to benchmark
>> more than one read policy.
>>
>> Patch 1,2 adds helpers to support a new setup.
>> Patch 3 adds a generic readonly dio fio benchmark script.
>>
> 
> This looks reasonable to me, the only thing is I'd want a way for the setup()
> part of the test to see that we don't have readpolicy support and simply skip.
> Right now there's no mechanism for skipping a test, so add that and it'll be
> good.

  Right. I have added it.

> Also you can just do PR's through github if you want.  Thanks,

  I included the above changes and sent a PR for this.

Thanks, Anand

> 
> Josef
>