From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EE95C433E7 for ; Fri, 16 Oct 2020 08:00:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C2420207F7 for ; Fri, 16 Oct 2020 08:00:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404956AbgJPIAx (ORCPT ); Fri, 16 Oct 2020 04:00:53 -0400 Received: from out30-130.freemail.mail.aliyun.com ([115.124.30.130]:49850 "EHLO out30-130.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404915AbgJPIAv (ORCPT ); Fri, 16 Oct 2020 04:00:51 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R201e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04357;MF=richard.weiyang@linux.alibaba.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---0UCAvZF5_1602835246; Received: from localhost(mailfrom:richard.weiyang@linux.alibaba.com fp:SMTPD_---0UCAvZF5_1602835246) by smtp.aliyun-inc.com(127.0.0.1); Fri, 16 Oct 2020 16:00:46 +0800 Date: Fri, 16 Oct 2020 16:00:46 +0800 From: Wei Yang To: Wei Yang Cc: David Hildenbrand , linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux-foundation.org, Andrew Morton , "Michael S . Tsirkin" , Jason Wang , Pankaj Gupta Subject: Re: [PATCH v1 13/29] virtio-mem: factor out handling of fake-offline pages in memory notifier Message-ID: <20201016080046.GA43862@L-31X9LVDL-1304.local> Reply-To: Wei Yang References: <20201012125323.17509-1-david@redhat.com> <20201012125323.17509-14-david@redhat.com> <20201016071502.GM86495@L-31X9LVDL-1304.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201016071502.GM86495@L-31X9LVDL-1304.local> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 16, 2020 at 03:15:03PM +0800, Wei Yang wrote: >On Mon, Oct 12, 2020 at 02:53:07PM +0200, David Hildenbrand wrote: >>Let's factor out the core pieces and place the implementation next to >>virtio_mem_fake_offline(). We'll reuse this functionality soon. >> >>Cc: "Michael S. Tsirkin" >>Cc: Jason Wang >>Cc: Pankaj Gupta >>Signed-off-by: David Hildenbrand >>--- >> drivers/virtio/virtio_mem.c | 73 +++++++++++++++++++++++++------------ >> 1 file changed, 50 insertions(+), 23 deletions(-) >> >>diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c >>index d132bc54ef57..a2124892e510 100644 >>--- a/drivers/virtio/virtio_mem.c >>+++ b/drivers/virtio/virtio_mem.c >>@@ -168,6 +168,10 @@ static LIST_HEAD(virtio_mem_devices); >> >> static void virtio_mem_online_page_cb(struct page *page, unsigned int order); >> static void virtio_mem_retry(struct virtio_mem *vm); >>+static void virtio_mem_fake_offline_going_offline(unsigned long pfn, >>+ unsigned long nr_pages); >>+static void virtio_mem_fake_offline_cancel_offline(unsigned long pfn, >>+ unsigned long nr_pages); >> >> /* >> * Register a virtio-mem device so it will be considered for the online_page >>@@ -604,27 +608,15 @@ static void virtio_mem_notify_going_offline(struct virtio_mem *vm, >> unsigned long mb_id) >> { >> const unsigned long nr_pages = PFN_DOWN(vm->subblock_size); >>- struct page *page; >> unsigned long pfn; >>- int sb_id, i; >>+ int sb_id; >> >> for (sb_id = 0; sb_id < vm->nb_sb_per_mb; sb_id++) { >> if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) >> continue; >>- /* >>- * Drop our reference to the pages so the memory can get >>- * offlined and add the unplugged pages to the managed >>- * page counters (so offlining code can correctly subtract >>- * them again). >>- */ >> pfn = PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + >> sb_id * vm->subblock_size); >>- adjust_managed_page_count(pfn_to_page(pfn), nr_pages); > >One question about the original code, why we want to adjust count here? > >The code flow is > > __offline_pages() > memory_notify(MEM_GOING_OFFLINE, &arg) > virtio_mem_notify_going_offline(vm, mb_id) > adjust_managed_page_count(pfn_to_page(pfn), nr_pages) > adjust_managed_page_count(pfn_to_page(start_pfn), -offlined_pages) > >Do we adjust the count twice? > Ah, I got the reason why we need to adjust count for *unplugged* sub-blocks. >>- for (i = 0; i < nr_pages; i++) { >>- page = pfn_to_page(pfn + i); >>- if (WARN_ON(!page_ref_dec_and_test(page))) Another question is when we grab a refcount for the unpluged pages? The one you mentioned in virtio_mem_set_fake_offline(). >>- dump_page(page, "unplugged page referenced"); >>- } >>+ virtio_mem_fake_offline_going_offline(pfn, nr_pages); >> } >> } >> >>@@ -633,21 +625,14 @@ static void virtio_mem_notify_cancel_offline(struct virtio_mem *vm, >> { >> const unsigned long nr_pages = PFN_DOWN(vm->subblock_size); >> unsigned long pfn; >>- int sb_id, i; >>+ int sb_id; >> >> for (sb_id = 0; sb_id < vm->nb_sb_per_mb; sb_id++) { >> if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) >> continue; >>- /* >>- * Get the reference we dropped when going offline and >>- * subtract the unplugged pages from the managed page >>- * counters. >>- */ >> pfn = PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + >> sb_id * vm->subblock_size); >>- adjust_managed_page_count(pfn_to_page(pfn), -nr_pages); >>- for (i = 0; i < nr_pages; i++) >>- page_ref_inc(pfn_to_page(pfn + i)); >>+ virtio_mem_fake_offline_cancel_offline(pfn, nr_pages); >> } >> } >> >>@@ -853,6 +838,48 @@ static int virtio_mem_fake_offline(unsigned long pfn, unsigned long nr_pages) >> return 0; >> } >> >>+/* >>+ * Handle fake-offline pages when memory is going offline - such that the >>+ * pages can be skipped by mm-core when offlining. >>+ */ >>+static void virtio_mem_fake_offline_going_offline(unsigned long pfn, >>+ unsigned long nr_pages) >>+{ >>+ struct page *page; >>+ unsigned long i; >>+ >>+ /* >>+ * Drop our reference to the pages so the memory can get offlined >>+ * and add the unplugged pages to the managed page counters (so >>+ * offlining code can correctly subtract them again). >>+ */ >>+ adjust_managed_page_count(pfn_to_page(pfn), nr_pages); >>+ /* Drop our reference to the pages so the memory can get offlined. */ >>+ for (i = 0; i < nr_pages; i++) { >>+ page = pfn_to_page(pfn + i); >>+ if (WARN_ON(!page_ref_dec_and_test(page))) >>+ dump_page(page, "fake-offline page referenced"); >>+ } >>+} >>+ >>+/* >>+ * Handle fake-offline pages when memory offlining is canceled - to undo >>+ * what we did in virtio_mem_fake_offline_going_offline(). >>+ */ >>+static void virtio_mem_fake_offline_cancel_offline(unsigned long pfn, >>+ unsigned long nr_pages) >>+{ >>+ unsigned long i; >>+ >>+ /* >>+ * Get the reference we dropped when going offline and subtract the >>+ * unplugged pages from the managed page counters. >>+ */ >>+ adjust_managed_page_count(pfn_to_page(pfn), -nr_pages); >>+ for (i = 0; i < nr_pages; i++) >>+ page_ref_inc(pfn_to_page(pfn + i)); >>+} >>+ >> static void virtio_mem_online_page_cb(struct page *page, unsigned int order) >> { >> const unsigned long addr = page_to_phys(page); >>-- >>2.26.2 > >-- >Wei Yang >Help you, Help me -- Wei Yang Help you, Help me