From: Xing Zhengjun <zhengjun.xing@linux.intel.com> To: Trond Myklebust <trondmy@hammerspace.com>, "rong.a.chen@intel.com" <rong.a.chen@intel.com> Cc: "torvalds@linux-foundation.org" <torvalds@linux-foundation.org>, "lkp@01.org" <lkp@01.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org> Subject: Re: [LKP] [SUNRPC] 0472e47660: fsmark.app_overhead 16.0% regression Date: Thu, 30 May 2019 15:20:28 +0800 [thread overview] Message-ID: <d796ac23-d5d6-cdfa-89c8-536e9496b551@linux.intel.com> (raw) In-Reply-To: <9a07c589f955e5af5acc0fa09a16a3256089e764.camel@hammerspace.com> On 5/30/2019 10:00 AM, Trond Myklebust wrote: > Hi Xing, > > On Thu, 2019-05-30 at 09:35 +0800, Xing Zhengjun wrote: >> Hi Trond, >> >> On 5/20/2019 1:54 PM, kernel test robot wrote: >>> Greeting, >>> >>> FYI, we noticed a 16.0% improvement of fsmark.app_overhead due to >>> commit: >>> >>> >>> commit: 0472e476604998c127f3c80d291113e77c5676ac ("SUNRPC: Convert >>> socket page send code to use iov_iter()") >>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git >>> master >>> >>> in testcase: fsmark >>> on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ >>> 3.00GHz with 384G memory >>> with following parameters: >>> >>> iterations: 1x >>> nr_threads: 64t >>> disk: 1BRD_48G >>> fs: xfs >>> fs2: nfsv4 >>> filesize: 4M >>> test_size: 40G >>> sync_method: fsyncBeforeClose >>> cpufreq_governor: performance >>> >>> test-description: The fsmark is a file system benchmark to test >>> synchronous write workloads, for example, mail servers workload. >>> test-url: https://sourceforge.net/projects/fsmark/ >>> >>> >>> >>> Details are as below: >>> ----------------------------------------------------------------- >>> ---------------------------------> >>> >>> >>> To reproduce: >>> >>> git clone https://github.com/intel/lkp-tests.git >>> cd lkp-tests >>> bin/lkp install job.yaml # job file is attached in this >>> email >>> bin/lkp run job.yaml >>> >>> =================================================================== >>> ====================== >>> compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/n >>> r_threads/rootfs/sync_method/tbox_group/test_size/testcase: >>> gcc-7/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel- >>> 7.6/64t/debian-x86_64-2018-04-03.cgz/fsyncBeforeClose/lkp-ivb- >>> ep01/40G/fsmark >>> >>> commit: >>> e791f8e938 ("SUNRPC: Convert xs_send_kvec() to use >>> iov_iter_kvec()") >>> 0472e47660 ("SUNRPC: Convert socket page send code to use >>> iov_iter()") >>> >>> e791f8e9380d945e 0472e476604998c127f3c80d291 >>> ---------------- --------------------------- >>> fail:runs %reproduction fail:runs >>> | | | >>> :4 50% 2:4 dmesg.WARNING:at#for >>> _ip_interrupt_entry/0x >>> %stddev %change %stddev >>> \ | \ >>> 15118573 ± 2% +16.0% 17538083 fsmark.app_overhead >>> 510.93 -22.7% 395.12 fsmark.files_per_sec >>> 24.90 +22.8% 30.57 fsmark.time.elapsed_ >>> time >>> 24.90 +22.8% 30.57 fsmark.time.elapsed_ >>> time.max >>> 288.00 ± 2% - >>> 27.8% 208.00 fsmark.time.percent_of_cpu_this_job_got >>> 70.03 ± 2% - >>> 11.3% 62.14 fsmark.time.system_time >>> >> >> Do you have time to take a look at this regression? > > From your stats, it looks to me as if the problem is increased NUMA > overhead. Pretty much everything else appears to be the same or > actually performing better than previously. Am I interpreting that > correctly? The real regression is the throughput(fsmark.files_per_sec) is decreased by 22.7%. > > If my interpretation above is correct, then I'm not seeing where this > patch would be introducing new NUMA regressions. It is just converting > from using one method of doing socket I/O to another. Could it perhaps > be a memory artefact due to your running the NFS client and server on > the same machine? > > Apologies for pushing back a little, but I just don't have the > hardware available to test NUMA configurations, so I'm relying on > external testing for the above kind of scenario. > Thanks for looking at this. If you need more information, please let me know. > Thanks > Trond > -- Zhengjun Xing
WARNING: multiple messages have this Message-ID (diff)
From: Xing Zhengjun <zhengjun.xing@linux.intel.com> To: lkp@lists.01.org Subject: Re: [SUNRPC] 0472e47660: fsmark.app_overhead 16.0% regression Date: Thu, 30 May 2019 15:20:28 +0800 [thread overview] Message-ID: <d796ac23-d5d6-cdfa-89c8-536e9496b551@linux.intel.com> (raw) In-Reply-To: <9a07c589f955e5af5acc0fa09a16a3256089e764.camel@hammerspace.com> [-- Attachment #1: Type: text/plain, Size: 4000 bytes --] On 5/30/2019 10:00 AM, Trond Myklebust wrote: > Hi Xing, > > On Thu, 2019-05-30 at 09:35 +0800, Xing Zhengjun wrote: >> Hi Trond, >> >> On 5/20/2019 1:54 PM, kernel test robot wrote: >>> Greeting, >>> >>> FYI, we noticed a 16.0% improvement of fsmark.app_overhead due to >>> commit: >>> >>> >>> commit: 0472e476604998c127f3c80d291113e77c5676ac ("SUNRPC: Convert >>> socket page send code to use iov_iter()") >>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git >>> master >>> >>> in testcase: fsmark >>> on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ >>> 3.00GHz with 384G memory >>> with following parameters: >>> >>> iterations: 1x >>> nr_threads: 64t >>> disk: 1BRD_48G >>> fs: xfs >>> fs2: nfsv4 >>> filesize: 4M >>> test_size: 40G >>> sync_method: fsyncBeforeClose >>> cpufreq_governor: performance >>> >>> test-description: The fsmark is a file system benchmark to test >>> synchronous write workloads, for example, mail servers workload. >>> test-url: https://sourceforge.net/projects/fsmark/ >>> >>> >>> >>> Details are as below: >>> ----------------------------------------------------------------- >>> ---------------------------------> >>> >>> >>> To reproduce: >>> >>> git clone https://github.com/intel/lkp-tests.git >>> cd lkp-tests >>> bin/lkp install job.yaml # job file is attached in this >>> email >>> bin/lkp run job.yaml >>> >>> =================================================================== >>> ====================== >>> compiler/cpufreq_governor/disk/filesize/fs2/fs/iterations/kconfig/n >>> r_threads/rootfs/sync_method/tbox_group/test_size/testcase: >>> gcc-7/performance/1BRD_48G/4M/nfsv4/xfs/1x/x86_64-rhel- >>> 7.6/64t/debian-x86_64-2018-04-03.cgz/fsyncBeforeClose/lkp-ivb- >>> ep01/40G/fsmark >>> >>> commit: >>> e791f8e938 ("SUNRPC: Convert xs_send_kvec() to use >>> iov_iter_kvec()") >>> 0472e47660 ("SUNRPC: Convert socket page send code to use >>> iov_iter()") >>> >>> e791f8e9380d945e 0472e476604998c127f3c80d291 >>> ---------------- --------------------------- >>> fail:runs %reproduction fail:runs >>> | | | >>> :4 50% 2:4 dmesg.WARNING:at#for >>> _ip_interrupt_entry/0x >>> %stddev %change %stddev >>> \ | \ >>> 15118573 ± 2% +16.0% 17538083 fsmark.app_overhead >>> 510.93 -22.7% 395.12 fsmark.files_per_sec >>> 24.90 +22.8% 30.57 fsmark.time.elapsed_ >>> time >>> 24.90 +22.8% 30.57 fsmark.time.elapsed_ >>> time.max >>> 288.00 ± 2% - >>> 27.8% 208.00 fsmark.time.percent_of_cpu_this_job_got >>> 70.03 ± 2% - >>> 11.3% 62.14 fsmark.time.system_time >>> >> >> Do you have time to take a look at this regression? > > From your stats, it looks to me as if the problem is increased NUMA > overhead. Pretty much everything else appears to be the same or > actually performing better than previously. Am I interpreting that > correctly? The real regression is the throughput(fsmark.files_per_sec) is decreased by 22.7%. > > If my interpretation above is correct, then I'm not seeing where this > patch would be introducing new NUMA regressions. It is just converting > from using one method of doing socket I/O to another. Could it perhaps > be a memory artefact due to your running the NFS client and server on > the same machine? > > Apologies for pushing back a little, but I just don't have the > hardware available to test NUMA configurations, so I'm relying on > external testing for the above kind of scenario. > Thanks for looking at this. If you need more information, please let me know. > Thanks > Trond > -- Zhengjun Xing
next prev parent reply other threads:[~2019-05-30 7:20 UTC|newest] Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-20 5:54 [SUNRPC] 0472e47660: fsmark.app_overhead 16.0% regression kernel test robot 2019-05-20 5:54 ` kernel test robot 2019-05-30 1:35 ` [LKP] " Xing Zhengjun 2019-05-30 1:35 ` Xing Zhengjun 2019-05-30 2:00 ` [LKP] " Trond Myklebust 2019-05-30 7:20 ` Xing Zhengjun [this message] 2019-05-30 7:20 ` Xing Zhengjun 2019-05-30 19:10 ` [LKP] " Trond Myklebust 2019-05-31 3:27 ` Xing Zhengjun 2019-05-31 3:27 ` Xing Zhengjun 2019-07-08 8:32 ` [LKP] " Xing Zhengjun 2019-07-08 8:32 ` Xing Zhengjun [not found] ` <DM5PR13MB1851813BBEA446E25C5001C2B8F60@DM5PR13MB1851.namprd13.prod.outlook.com> 2019-07-09 2:39 ` [LKP] " Xing Zhengjun 2019-07-09 2:39 ` Xing Zhengjun 2019-07-12 6:42 ` [LKP] " Xing Zhengjun 2019-07-12 6:42 ` Xing Zhengjun 2019-07-24 5:17 ` [LKP] " Xing Zhengjun 2019-07-24 5:17 ` Xing Zhengjun 2019-08-07 7:56 ` [LKP] " Xing Zhengjun 2019-08-07 7:56 ` Xing Zhengjun 2019-08-30 0:43 ` [LKP] " Xing Zhengjun 2019-08-30 0:43 ` Xing Zhengjun 2019-09-25 9:00 ` [LKP] " Xing Zhengjun 2019-09-25 9:00 ` Xing Zhengjun
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=d796ac23-d5d6-cdfa-89c8-536e9496b551@linux.intel.com \ --to=zhengjun.xing@linux.intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=lkp@01.org \ --cc=rong.a.chen@intel.com \ --cc=torvalds@linux-foundation.org \ --cc=trondmy@hammerspace.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.