From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C990DC433DF for ; Fri, 16 Oct 2020 04:03:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 43CC52076E for ; Fri, 16 Oct 2020 04:03:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43CC52076E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 58019940007; Fri, 16 Oct 2020 00:03:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50A4F900002; Fri, 16 Oct 2020 00:03:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F89E940007; Fri, 16 Oct 2020 00:03:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id 0A48A900002 for ; Fri, 16 Oct 2020 00:03:08 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8415C362D for ; Fri, 16 Oct 2020 04:03:08 +0000 (UTC) X-FDA: 77376443256.16.guide59_13027b82721a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 5E6DF1024CAAE for ; Fri, 16 Oct 2020 04:03:08 +0000 (UTC) X-HE-Tag: guide59_13027b82721a X-Filterd-Recvd-Size: 6245 Received: from out30-43.freemail.mail.aliyun.com (out30-43.freemail.mail.aliyun.com [115.124.30.43]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Fri, 16 Oct 2020 04:03:06 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=richard.weiyang@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0UC9wxDT_1602820981; Received: from localhost(mailfrom:richard.weiyang@linux.alibaba.com fp:SMTPD_---0UC9wxDT_1602820981) by smtp.aliyun-inc.com(127.0.0.1); Fri, 16 Oct 2020 12:03:02 +0800 Date: Fri, 16 Oct 2020 12:03:01 +0800 From: Wei Yang To: David Hildenbrand Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux-foundation.org, Andrew Morton , "Michael S . Tsirkin" , Jason Wang , Pankaj Gupta Subject: Re: [PATCH v1 09/29] virtio-mem: don't always trigger the workqueue when offlining memory Message-ID: <20201016040301.GJ86495@L-31X9LVDL-1304.local> Reply-To: Wei Yang References: <20201012125323.17509-1-david@redhat.com> <20201012125323.17509-10-david@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201012125323.17509-10-david@redhat.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Oct 12, 2020 at 02:53:03PM +0200, David Hildenbrand wrote: >Let's trigger from offlining code when we're not allowed to touch online >memory. This describes the change in virtio_mem_memory_notifier_cb()? > >Handle the other case (memmap possibly freeing up another memory block) >when actually removing memory. When removing via virtio_mem_remove(), >virtio_mem_retry() is a NOP and safe to use. > >While at it, move retry handling when offlining out of >virtio_mem_notify_offline(), to share it with Device Block Mode (DBM) >soon. I may not understand the logic fully. Here is my understanding of current logic: virtio_mem_run_wq() virtio_mem_unplug_request() virtio_mem_mb_unplug_any_sb_offline() virtio_mem_mb_remove() --- 1 virtio_mem_mb_unplug_any_sb_online() virtio_mem_mb_offline_and_remove() --- 2 This patch tries to trigger the wq at 1 and 2. And these two functions are only valid during this code flow. These two functions actually remove some memory from the system. So I am not sure where extra unplug-able memory comes from. I guess those memory is from memory block device and mem_sectioin, memmap? While those memory is still marked as online, right? In case we can gather extra memory at 1 and form a whole memory block. So that we can unplug an online memory block (by moving data to a new place), this just affect the process at 2. This means there is no need to trigger the wq at 1, and we can leave it at 2. > >Cc: "Michael S. Tsirkin" >Cc: Jason Wang >Cc: Pankaj Gupta >Signed-off-by: David Hildenbrand >--- > drivers/virtio/virtio_mem.c | 40 ++++++++++++++++++++++++++----------- > 1 file changed, 28 insertions(+), 12 deletions(-) > >diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c >index 5c93f8a65eba..8ea00f0b2ecd 100644 >--- a/drivers/virtio/virtio_mem.c >+++ b/drivers/virtio/virtio_mem.c >@@ -158,6 +158,7 @@ static DEFINE_MUTEX(virtio_mem_mutex); > static LIST_HEAD(virtio_mem_devices); > > static void virtio_mem_online_page_cb(struct page *page, unsigned int order); >+static void virtio_mem_retry(struct virtio_mem *vm); > > /* > * Register a virtio-mem device so it will be considered for the online_page >@@ -435,9 +436,17 @@ static int virtio_mem_mb_add(struct virtio_mem *vm, unsigned long mb_id) > static int virtio_mem_mb_remove(struct virtio_mem *vm, unsigned long mb_id) > { > const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id); >+ int rc; > > dev_dbg(&vm->vdev->dev, "removing memory block: %lu\n", mb_id); >- return remove_memory(vm->nid, addr, memory_block_size_bytes()); >+ rc = remove_memory(vm->nid, addr, memory_block_size_bytes()); >+ if (!rc) >+ /* >+ * We might have freed up memory we can now unplug, retry >+ * immediately instead of waiting. >+ */ >+ virtio_mem_retry(vm); >+ return rc; > } > > /* >@@ -452,11 +461,19 @@ static int virtio_mem_mb_offline_and_remove(struct virtio_mem *vm, > unsigned long mb_id) > { > const uint64_t addr = virtio_mem_mb_id_to_phys(mb_id); >+ int rc; > > dev_dbg(&vm->vdev->dev, "offlining and removing memory block: %lu\n", > mb_id); >- return offline_and_remove_memory(vm->nid, addr, >- memory_block_size_bytes()); >+ rc = offline_and_remove_memory(vm->nid, addr, >+ memory_block_size_bytes()); >+ if (!rc) >+ /* >+ * We might have freed up memory we can now unplug, retry >+ * immediately instead of waiting. >+ */ >+ virtio_mem_retry(vm); >+ return rc; > } > > /* >@@ -534,15 +551,6 @@ static void virtio_mem_notify_offline(struct virtio_mem *vm, > BUG(); > break; > } >- >- /* >- * Trigger the workqueue, maybe we can now unplug memory. Also, >- * when we offline and remove a memory block, this will re-trigger >- * us immediately - which is often nice because the removal of >- * the memory block (e.g., memmap) might have freed up memory >- * on other memory blocks we manage. >- */ >- virtio_mem_retry(vm); > } > > static void virtio_mem_notify_online(struct virtio_mem *vm, unsigned long mb_id) >@@ -679,6 +687,14 @@ static int virtio_mem_memory_notifier_cb(struct notifier_block *nb, > break; > case MEM_OFFLINE: > virtio_mem_notify_offline(vm, mb_id); >+ >+ /* >+ * Trigger the workqueue. Now that we have some offline memory, >+ * maybe we can handle pending unplug requests. >+ */ >+ if (!unplug_online) >+ virtio_mem_retry(vm); >+ > vm->hotplug_active = false; > mutex_unlock(&vm->hotplug_mutex); > break; >-- >2.26.2 -- Wei Yang Help you, Help me