From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCB22C433EF for ; Sat, 9 Oct 2021 05:43:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2C3FA60F41 for ; Sat, 9 Oct 2021 05:43:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2C3FA60F41 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=edi.works Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 38B7E6B0071; Sat, 9 Oct 2021 01:43:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 33C156B0072; Sat, 9 Oct 2021 01:43:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2035D6B0073; Sat, 9 Oct 2021 01:43:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id 0D5C06B0071 for ; Sat, 9 Oct 2021 01:43:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9FC3239F32 for ; Sat, 9 Oct 2021 05:43:19 +0000 (UTC) X-FDA: 78675806118.22.5E45082 Received: from mail-qt1-f193.google.com (mail-qt1-f193.google.com [209.85.160.193]) by imf07.hostedemail.com (Postfix) with ESMTP id 3FEBD1002AD4 for ; Sat, 9 Oct 2021 05:43:19 +0000 (UTC) Received: by mail-qt1-f193.google.com with SMTP id i1so10416856qtr.6 for ; Fri, 08 Oct 2021 22:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=edi.works; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=R1b6LRnDgw4MXgnqL8ApMyFBNf+HWB+MOQp9ynyp86Q=; b=Xwz0/FiEnkJdoNRxntzjBQc0Gm3a1pMvmW4rJ/VSan9bUkANBa8mnhQrPpTuGc0Hae 8CKOaacVHDcYNUMJQFgQD534oCqNmai1pLLiTVPnZ4cWEVJwArNfkY73kb0XyZo41D+F Re7wlKk2DZd5JyjQS9qbCV0Zv0ZJs2U1OpAOJEX+Tpy1Qep/sHjRtrnhK+TAM2WxAPui f2/ZMviteTrkP6MkVvrBa2n9dgRQl9+kEC7v4zjfh253YkytvYNWFvblp3AzR5xxFxc1 vOxYItPXxuJe9RKIuMpMtPUX47N9auwSnFl1gCSrqIrTBrKHKiQpUxBZZRpsLKmKXoNT Ra1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=R1b6LRnDgw4MXgnqL8ApMyFBNf+HWB+MOQp9ynyp86Q=; b=aHfMAgW0DTvZhoQ/Z5hHVRNELHu+qECPLjSbvYOWxQ3RWLHNqSVGckIUfAB4HeUVhi ihlh8fcrPV9zrQSUI0b7huIH5rK3Fegri+poW9jAKxpFY1lEXaBXKGb0lUPunoQYQQY5 AjwZFKrLjb+naNZ5eoajst8aXFCCapHILxZ7IzJ28QObhxV6IjjMDAAFoJWpwZCcwBzv PhK56Dj4Gu5pAifGLZCK5ZmNQuER83TkW4MxHHg+PqpcwTgM44kJlgsH6LQM8TxD6Ffh ZZj8nTYM8eKtjaMb+SUQprpS1RILim2hZAMcby5YOxdf2R1TarUQ9b5kN3V4oNKKcfjJ 0qMg== X-Gm-Message-State: AOAM532vMJHlXV3N8JVrXT6ZiCkSirV/TR/zG2t3kt3wyrOOmn+m1uk2 Cwe4NxVu+wcd1Hx3SAATJ7vb4A== X-Google-Smtp-Source: ABdhPJyhJjm+JmkYPHf2Z7h4TM+aA7Ak9oB7tciu7UAheWRLg4EpDJMCE5h16KIxP85tql1QpPeCdA== X-Received: by 2002:ac8:4e92:: with SMTP id 18mr2318782qtp.323.1633758198507; Fri, 08 Oct 2021 22:43:18 -0700 (PDT) Received: from localhost.localdomain (c-67-169-44-201.hsd1.ca.comcast.net. [67.169.44.201]) by smtp.gmail.com with ESMTPSA id t64sm1063783qkd.71.2021.10.08.22.43.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Oct 2021 22:43:17 -0700 (PDT) From: bot@edi.works To: yuzhao@google.com Cc: hdanton@sina.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, corbet@lwn.net, michael@michaellarabel.com, sofia.trinh@edi.works Subject: Re: [PATCH v4 00/11] Multigenerational LRU Framework Date: Fri, 8 Oct 2021 22:43:15 -0700 Message-Id: <20211009054315.47073-1-bot@edi.works> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210818063107.2696454-1-yuzhao@google.com> References: <20210818063107.2696454-1-yuzhao@google.com> X-Rspamd-Queue-Id: 3FEBD1002AD4 X-Stat-Signature: 8fnt7tsz1tynznbcyn9pckx5fjeea6gr Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=edi.works header.s=google header.b="Xwz0/FiE"; spf=pass (imf07.hostedemail.com: domain of bot@edi.works designates 209.85.160.193 as permitted sender) smtp.mailfrom=bot@edi.works; dmarc=none X-Rspamd-Server: rspam06 X-HE-Tag: 1633758199-757476 X-Bogosity: Ham, tests=bogofilter, spamicity=0.001217, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Kernel / MariaDB benchmark with MGLRU TLDR ==== With the MGLRU, MariaDB achieved 95% CIs [5.24, 10.71]% and [20.22, 25.97]% more transactions per minute (TPM), respectively, under the medium- and high-concurrency conditions when slightly overcommitting memory. There were no statistically significant changes in TPM under other conditions. Rationale ========= Memory overcommit can improve utilization and, if not overdone, can also increase throughput. The challenges are estimating working sets and optimizing page reclaim. The risks are performance degradations and OOM kills. Unless overcoming the challenges, the only way to reduce the risks is to overprovision memory. MariaDB is one of the most popular open-source RDBMSs. HammerDB is the leading open-source benchmarking software derived from the TPC specifications. OLTP is the most important use case for RDBMSs. Matrix ====== Kernels: version [+ patchset] * Baseline: 5.14 * Patched: 5.14 + MGLRU Memory conditions: % of memory size * Underutilizing: ~10% on inactive file list * Overcommitting: ~10% swapped out Concurrency conditions: average # of users per CPU * Low: ~3 * Medium: ~13 * High: ~19 Total configurations: 12 Data points per configuration: 10 Total run duration (minutes) per data point: ~45 Procedure ========= The latest MGLRU patchset for the 5.14 kernel is available at git fetch https://linux-mm.googlesource.com/page-reclaim \ refs/changes/30/1430/1 Baseline and patched 5.14 kernel images are available at https://drive.google.com/drive/folders/1eMkQleAFGkP2vzM_JyRA21oKE0ESHBqp hammerdbcli auto prep_tpcc.tcl systemctl stop mariadb e2image grub2-set-default systemctl stop mariadb e2image reboot hammerdbcli auto run_tpcc.tcl Hardware ======== Memory (GB): 64 CPU (total #): 32 NVMe SSD (GB): 1024 OS == $ cat /etc/redhat-release Red Hat Enterprise Linux release 8.4 (Ootpa) $ cat /proc/swaps Filename Type Size Used Priority /dev/nvme0n1p3 partition 32970748 0 -2 $ mount | grep data /dev/nvme0n1p4 on /mnt/data type ext4 (rw,relatime,seclabel) $ cat /proc/cmdline systemd.unified_cgroup_hierarchy=1 $ cat /sys/fs/cgroup/user.slice/memory.min 4294967296 $ cat /proc/sys/vm/overcommit_memory 1 MariaDB ======= $ mysql --version mysql Ver 15.1 Distrib 10.3.28-MariaDB, for Linux (x86_64) using readline 5.1 $ cat /etc/my.cnf [mysqld] innodb_buffer_pool_size=<50G, 60G> innodb_doublewrite=0 innodb_flush_log_at_trx_commit=0 innodb_flush_method=O_DIRECT_NO_FSYNC innodb_flush_neighbors=0 innodb_io_capacity=4000 innodb_io_capacity_max=20000 innodb_log_buffer_size=1G innodb_log_file_size=20G innodb_max_dirty_pages_pct=90 innodb_max_dirty_pages_pct_lwm=10 max_connections=1000 datadir=/mnt/data HammerDB ======== $ hammerdbcli -h HammerDB CLI v4.2 Copyright (C) 2003-2021 Steve Shaw Type "help" for a list of commands Usage: hammerdbcli [ auto [ script_to_autoload.tcl ] ] $ cat prep_tpcc.tcl dbset db maria diset connection maria_socket /var/lib/mysql/mysql.sock diset tpcc maria_count_ware 1200 diset tpcc maria_num_vu 32 diset tpcc maria_partition true buildschema waittocomplete quit $ cat run_tpcc.tcl dbset db maria diset connection maria_socket /var/lib/mysql/mysql.sock diset tpcc maria_total_iterations 20000000 diset tpcc maria_driver timed diset tpcc maria_rampup 10 diset tpcc maria_duration 30 diset tpcc maria_allwarehouse true vuset logtotemp 1 vuset unique 1 loadscript vuset vu <100, 400, 600> vucreate vurun runtimer 3000 Vudestroy Results ======= Comparing the patched with the baseline kernel, MariaDB achieved 95% CIs [5.24, 10.71]% and [20.22, 25.97]% more TPM, respectively, under the medium- and high-concurrency conditions when slightly overcommitting memory. There were no statistically significant changes in TPM under other conditions. +--------------------+-----------------------+-----------------------+ | Mean TPM [95% CI] | Underutilizing memory | Overcommitting memory | +--------------------+-----------------------+-----------------------+ | Low concurrency | 270811.6 / 271522.7 | 447933.4 / 447283.3 | | | [-40.97, 1463.17] | [-1330.61, 30.41] | +--------------------+-----------------------+-----------------------+ | Medium concurrency | 240212.9 / 242846.7 | 327276.6 / 353372.7 | | | [-2611.38, 7878.98] | [17149.01, 35043.19] | +--------------------+-----------------------+-----------------------+ | High concurrency | 283897.8 / 283668.1 | 274069.7 / 337366.8 | | | [-11538.08, 11078.68] | [55417.42, 71176.78] | +--------------------+-----------------------+-----------------------+ Table 1. Comparison between the baseline and patched kernels Comparing overcommitting with underutilizing memory, MariaDB achieved 95% CIs [65.12, 65.68]% and [32.45, 40.04]% more TPM, respectively, under the low- and medium-concurrency conditions when using the baseline kernel; 95% CIs [64.48, 64.98]%, [43.53, 47.50]% and [16.48, 21.38]% more TPM, respectively, under the low-, medium- and high-concurrency conditions when using the patched kernel. There were no statistically significant changes in TPM under other conditions. +--------------------+------------------------+----------------------+ | Mean TPM [95% CI] | Baseline kernel | Patched kernel | +--------------------+------------------------+----------------------+ | Low concurrency | 270811.6 / 447933.4 | 271522.7 / 447283.3 | | | [176362.0, 177881.6] | [175089.3, 176431.9] | +--------------------+------------------------+----------------------+ | Medium concurrency | 240212.9 / 327276.6 | 242846.7 / 353372.7 | | | [77946.4, 96181.0] | [105707.7, 115344.3] | +--------------------+------------------------+----------------------+ | High concurrency | 283897.8 / 274069.7 | 283668.1 / 337366.8 | | | [-21605.703, 1949.503] | [46758.85, 60638.55] | +--------------------+------------------------+----------------------+ Table 2. Comparison between underutilizing and overcommitting memory Metrics collected during each run are available at https://github.com/ediworks/KernelPerf/tree/master/mglru/mariadb/5.14 References ========== HammerDB v4.2 New Features: https://www.hammerdb.com/blog/uncategorized/hammerdb-v4-2-new-features -pt1-mariadb-build-and-test-example-with-the-cli/ Appendix ======== $ cat raw_data.r v <- c( # baseline 50g 100vu 269531,270113,270256,270367,270393,270630,270707,271373,272291,272455, # baseline 50g 400vu 231856,234985,235144,235552,238551,239994,244413,245255,247997,248382, # baseline 50g 600vu 256365,271733,275966,280623,281014,283764,293327,296750,298728,300708, # baseline 60g 100vu 446973,447383,447412,447489,447874,448046,448123,448531,448739,448764, # baseline 60g 400vu 312427,312936,313780,321503,329554,330551,332377,333584,337105,348949, # baseline 60g 600vu 262338,262971,266242,266489,268036,272494,279045,281472,289942,291668, # patched 50g 100vu 270621,270913,271026,271137,271517,271616,271699,272117,272218,272363, # patched 50g 400vu 233314,238265,238722,240540,241676,245204,245688,247440,248417,249201, # patched 50g 600vu 271114,271928,277562,279455,282074,285515,287836,288508,289451,303238, # patched 60g 100vu 445923,446178,446837,446889,447331,447480,447823,447999,448145,448228, # patched 60g 400vu 345705,349373,350832,351229,351758,352520,355130,355247,357762,364171, # patched 60g 600vu 330860,334705,336001,337291,338326,338361,338970,339163,339784,340207 ) a <- array(v, dim = c(10, 3, 2, 2)) # baseline vs patched for (m in 1:2) { for (c in 1:3) { r <- t.test(a[, c, m, 1], a[, c, m, 2]) print(r) p <- r$conf.int * 100 / r$estimate[1] if ((p[1] > 0 && p[2] < 0) || (p[1] < 0 && p[2] > 0)) { s <- sprintf("m%d c%d: no significance", m, c) } else { s <- sprintf("m%d c%d: [%.2f, %.2f]%%", m, c, -p[2], -p[1]) } print(s) } } # 50g vs 60g for (k in 1:2) { for (c in 1:3) { r <- t.test(a[, c, 1, k], a[, c, 2, k]) print(r) p <- r$conf.int * 100 / r$estimate[1] if ((p[1] > 0 && p[2] < 0) || (p[1] < 0 && p[2] > 0)) { s <- sprintf("k%d c%d: no significance", k, c) } else { s <- sprintf("k%d c%d: [%.2f, %.2f]%%", k, c, -p[2], -p[1]) } print(s) } } $ R -q -s -f raw_data.r Welch Two Sample t-test data: a[, c, m, 1] and a[, c, m, 2] t = -2.0139, df = 15.122, p-value = 0.06217 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -1463.17673 40.97673 sample estimates: mean of x mean of y 270811.6 271522.7 [1] "50g 100vu: no significance" Welch Two Sample t-test data: a[, c, m, 1] and a[, c, m, 2] t = -1.0564, df = 17.673, p-value = 0.305 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -7878.98 2611.38 sample estimates: mean of x mean of y 240212.9 242846.7 [1] "50g 400vu: no significance" Welch Two Sample t-test data: a[, c, m, 1] and a[, c, m, 2] t = 0.043083, df = 15.895, p-value = 0.9662 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -11078.68 11538.08 sample estimates: mean of x mean of y 283897.8 283668.1 [1] "50g 600vu: no significance" Welch Two Sample t-test data: a[, c, m, 1] and a[, c, m, 2] t = 2.0171, df = 16.831, p-value = 0.05993 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -30.41577 1330.61577 sample estimates: mean of x mean of y 447933.4 447283.3 [1] "60g 100vu: no significance" Welch Two Sample t-test data: a[, c, m, 1] and a[, c, m, 2] t = -6.3473, df = 12.132, p-value = 3.499e-05 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -35043.19 -17149.01 sample estimates: mean of x mean of y 327276.6 353372.7 [1] "60g 400vu: [5.24, 10.71]%" Welch Two Sample t-test data: a[, c, m, 1] and a[, c, m, 2] t = -17.844, df = 10.233, p-value = 4.822e-09 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -71176.78 -55417.42 sample estimates: mean of x mean of y 274069.7 337366.8 [1] "60g 600vu: [20.22, 25.97]%" Welch Two Sample t-test data: a[, c, 1, k] and a[, c, 2, k] t = -495.48, df = 15.503, p-value < 2.2e-16 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -177881.6 -176362.0 sample estimates: mean of x mean of y 270811.6 447933.4 [1] "baseline 100vu: [65.12, 65.68]%" Welch Two Sample t-test data: a[, c, 1, k] and a[, c, 2, k] t = -20.601, df = 13.182, p-value = 2.062e-11 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -96181.0 -77946.4 sample estimates: mean of x mean of y 240212.9 327276.6 [1] "baseline 400vu: [32.45, 40.04]%" Welch Two Sample t-test data: a[, c, 1, k] and a[, c, 2, k] t = 1.7607, df = 16.986, p-value = 0.09628 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -1949.503 21605.703 sample estimates: mean of x mean of y 283897.8 274069.7 [1] "baseline 600vu: no significance" Welch Two Sample t-test data: a[, c, 1, k] and a[, c, 2, k] t = -553.68, df = 16.491, p-value < 2.2e-16 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -176431.9 -175089.3 sample estimates: mean of x mean of y 271522.7 447283.3 [1] "patched 100vu: [64.48, 64.98]%" Welch Two Sample t-test data: a[, c, 1, k] and a[, c, 2, k] t = -48.194, df = 17.992, p-value < 2.2e-16 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -115344.3 -105707.7 sample estimates: mean of x mean of y 242846.7 353372.7 [1] "patched 400vu: [43.53, 47.50]%" Welch Two Sample t-test data: a[, c, 1, k] and a[, c, 2, k] t = -17.109, df = 10.6, p-value = 4.629e-09 alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: -60638.55 -46758.85 sample estimates: mean of x mean of y 283668.1 337366.8 [1] "patched 600vu: [16.48, 21.38]%"