From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1A7CC433E1 for ; Wed, 24 Jun 2020 03:31:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70AC820857 for ; Wed, 24 Jun 2020 03:31:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70AC820857 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ECA4A6B0007; Tue, 23 Jun 2020 23:31:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E52886B0008; Tue, 23 Jun 2020 23:31:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D19D46B000A; Tue, 23 Jun 2020 23:31:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0169.hostedemail.com [216.40.44.169]) by kanga.kvack.org (Postfix) with ESMTP id B41486B0007 for ; Tue, 23 Jun 2020 23:31:53 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9B8313499 for ; Wed, 24 Jun 2020 03:31:51 +0000 (UTC) X-FDA: 76962681222.29.metal50_580669b26e40 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 7436C18086E27 for ; Wed, 24 Jun 2020 03:31:51 +0000 (UTC) X-HE-Tag: metal50_580669b26e40 X-Filterd-Recvd-Size: 6588 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Wed, 24 Jun 2020 03:31:50 +0000 (UTC) IronPort-SDR: BR+iJ9uzFz5srvzsshtq9hF8rcsduc0STvg2JPB/au2xKjvVMhHWCSM2OfjGG1PnqCPQiwUDcZ WF2Brzkt029g== X-IronPort-AV: E=McAfee;i="6000,8403,9661"; a="124571212" X-IronPort-AV: E=Sophos;i="5.75,273,1589266800"; d="scan'208";a="124571212" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2020 20:31:48 -0700 IronPort-SDR: vqGtMTTjU0DkuCs2AfagtZX/vQkgtfO+0GXC40K4tsidAgz2ccmtwi1rYxoSV9yenUCgm8rcfK SwA3bszqLA8Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,273,1589266800"; d="scan'208";a="452488485" Received: from yhuang-dev.sh.intel.com (HELO yhuang-dev) ([10.239.159.23]) by orsmga005.jf.intel.com with ESMTP; 23 Jun 2020 20:31:46 -0700 From: "Huang\, Ying" To: Andrew Morton Cc: , , Daniel Jordan , Michal Hocko , Minchan Kim , Tim Chen , Hugh Dickins Subject: Re: [PATCH -V2] swap: Reduce lock contention on swap cache from swap slots allocation References: <20200520031502.175659-1-ying.huang@intel.com> <20200520195102.2343f746e88a2bec5c29ef5b@linux-foundation.org> <87o8qihsw7.fsf@yhuang-dev.intel.com> Date: Wed, 24 Jun 2020 11:31:45 +0800 In-Reply-To: <87o8qihsw7.fsf@yhuang-dev.intel.com> (Ying Huang's message of "Thu, 21 May 2020 11:24:40 +0800") Message-ID: <87wo3xxhpa.fsf@yhuang-dev.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 7436C18086E27 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: "Huang, Ying" writes: > Andrew Morton writes: > >> On Wed, 20 May 2020 11:15:02 +0800 Huang Ying wrote: >> >>> In some swap scalability test, it is found that there are heavy lock >>> contention on swap cache even if we have split one swap cache radix >>> tree per swap device to one swap cache radix tree every 64 MB trunk in >>> commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB trunks"). >>> >>> The reason is as follow. After the swap device becomes fragmented so >>> that there's no free swap cluster, the swap device will be scanned >>> linearly to find the free swap slots. swap_info_struct->cluster_next >>> is the next scanning base that is shared by all CPUs. So nearby free >>> swap slots will be allocated for different CPUs. The probability for >>> multiple CPUs to operate on the same 64 MB trunk is high. This causes >>> the lock contention on the swap cache. >>> >>> To solve the issue, in this patch, for SSD swap device, a percpu >>> version next scanning base (cluster_next_cpu) is added. Every CPU >>> will use its own per-cpu next scanning base. And after finishing >>> scanning a 64MB trunk, the per-cpu scanning base will be changed to >>> the beginning of another randomly selected 64MB trunk. In this way, >>> the probability for multiple CPUs to operate on the same 64 MB trunk >>> is reduced greatly. Thus the lock contention is reduced too. For >>> HDD, because sequential access is more important for IO performance, >>> the original shared next scanning base is used. >>> >>> To test the patch, we have run 16-process pmbench memory benchmark on >>> a 2-socket server machine with 48 cores. One ram disk is configured >> >> What does "ram disk" mean here? Which drivers(s) are in use and backed >> by what sort of memory? > > We use the following kernel command line > > memmap=48G!6G memmap=48G!68G > > to create 2 DRAM based /dev/pmem disks (48GB each). Then we use these > ram disks as swap devices. > >>> as the swap device per socket. The pmbench working-set size is much >>> larger than the available memory so that swapping is triggered. The >>> memory read/write ratio is 80/20 and the accessing pattern is random. >>> In the original implementation, the lock contention on the swap cache >>> is heavy. The perf profiling data of the lock contention code path is >>> as following, >>> >>> _raw_spin_lock_irq.add_to_swap_cache.add_to_swap.shrink_page_list: 7.91 >>> _raw_spin_lock_irqsave.__remove_mapping.shrink_page_list: 7.11 >>> _raw_spin_lock.swapcache_free_entries.free_swap_slot.__swap_entry_free: 2.51 >>> _raw_spin_lock_irqsave.swap_cgroup_record.mem_cgroup_uncharge_swap: 1.66 >>> _raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node: 1.29 >>> _raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages: 1.03 >>> _raw_spin_lock_irq.shrink_active_list.shrink_lruvec.shrink_node: 0.93 >>> >>> After applying this patch, it becomes, >>> >>> _raw_spin_lock.swapcache_free_entries.free_swap_slot.__swap_entry_free: 3.58 >>> _raw_spin_lock_irq.shrink_inactive_list.shrink_lruvec.shrink_node: 2.3 >>> _raw_spin_lock_irqsave.swap_cgroup_record.mem_cgroup_uncharge_swap: 2.26 >>> _raw_spin_lock_irq.shrink_active_list.shrink_lruvec.shrink_node: 1.8 >>> _raw_spin_lock.free_pcppages_bulk.drain_pages_zone.drain_pages: 1.19 >>> >>> The lock contention on the swap cache is almost eliminated. >>> >>> And the pmbench score increases 18.5%. The swapin throughput >>> increases 18.7% from 2.96 GB/s to 3.51 GB/s. While the swapout >>> throughput increases 18.5% from 2.99 GB/s to 3.54 GB/s. >> >> If this was backed by plain old RAM, can we assume that the performance >> improvement on SSD swap is still good? > > We need really fast disk to show the benefit. I have tried this on 2 > Intel P3600 NVMe disks. The performance improvement is only about 1%. > The improvement should be better on the faster disks, such as Intel > Optane disk. I will try to find some to test. I finally find 2 Intel Optane disks to test. The pmbench throughput (page accesses per second) increases ~1.7% with the patch. The swapin throughput increases 2% (~1.36 GB/s to ~1.39 GB/s), the swapout throughput increases 1.7% (~1.61 GB/s to 1.63 GB/s). Perf profile shows the CPU cycles on the swap cache radix tree spinlock is reduced from ~1.76% to nearly 0. So the performance difference is much smaller, but still measurable. Best Regards, Huang, Ying