From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 115B0C433DF for ; Mon, 12 Oct 2020 12:55:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 74D23221EB for ; Mon, 12 Oct 2020 12:55:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NWVh630n" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 74D23221EB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 00AA2940014; Mon, 12 Oct 2020 08:55:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED568940007; Mon, 12 Oct 2020 08:55:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9E6F940014; Mon, 12 Oct 2020 08:55:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id AAB00940007 for ; Mon, 12 Oct 2020 08:55:53 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 40B303622 for ; Mon, 12 Oct 2020 12:55:53 +0000 (UTC) X-FDA: 77363270586.01.value38_0610cbf271fa Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 0FD4510046477 for ; Mon, 12 Oct 2020 12:55:53 +0000 (UTC) X-HE-Tag: value38_0610cbf271fa X-Filterd-Recvd-Size: 14490 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Mon, 12 Oct 2020 12:55:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602507351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AaYZLte8fjkRY7vPK3h+9E7Lu8JJkuUbZB+FmZcPFLg=; b=NWVh630nmkFuQpckU/rZFdjL8j7oU2n+FVBtRi4gEMafUe7eVro7/aVVIY0fC2CFYd8/ku /pPa/waRoFAFYqGptcfQuTnMsJaRTwHwg6ik/7EyNgKQUuiMdAzxpmp+cBnLIBTIeJS0Xw ZfoCudYymgxk68zUZKrM2e4xRBp2zvY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-22-NmJKTitpPiKFqGufjuNfkQ-1; Mon, 12 Oct 2020 08:55:50 -0400 X-MC-Unique: NmJKTitpPiKFqGufjuNfkQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B143C8018A1; Mon, 12 Oct 2020 12:55:48 +0000 (UTC) Received: from t480s.redhat.com (ovpn-113-251.ams2.redhat.com [10.36.113.251]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0647D60C07; Mon, 12 Oct 2020 12:55:40 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtualization@lists.linux-foundation.org, Andrew Morton , "Michael S . Tsirkin" , David Hildenbrand , Jason Wang , Pankaj Gupta Subject: [PATCH v1 17/29] virito-mem: subblock states are specific to Sub Block Mode (SBM) Date: Mon, 12 Oct 2020 14:53:11 +0200 Message-Id: <20201012125323.17509-18-david@redhat.com> In-Reply-To: <20201012125323.17509-1-david@redhat.com> References: <20201012125323.17509-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Let's rename and move accordingly. While at it, rename sb_bitmap to "sb_states". Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 118 +++++++++++++++++++----------------- 1 file changed, 62 insertions(+), 56 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index e76d6f769aa5..2cc497ad8298 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -137,17 +137,23 @@ struct virtio_mem { * memory in one 4 KiB page. */ uint8_t *mb_states; - } sbm; =20 - /* - * $nb_sb_per_mb bit per memory block. Handled similar to sbm.mb_states= . - * - * With 4MB subblocks, we manage 128GB of memory in one page. - */ - unsigned long *sb_bitmap; + /* + * Bitmap: one bit per subblock. Allocated similar to + * sbm.mb_states. + * + * A set bit means the corresponding subblock is plugged, + * otherwise it's unblocked. + * + * With 4 MiB subblocks, we manage 128 GiB of memory in one + * 4 KiB page. + */ + unsigned long *sb_states; + } sbm; =20 /* - * Mutex that protects the sbm.mb_count, sbm.mb_states, and sb_bitmap. + * Mutex that protects the sbm.mb_count, sbm.mb_states, and + * sbm.sb_states. * * When this lock is held the pointers can't change, ONLINE and * OFFLINE blocks can't change the state and no subblocks will get @@ -326,13 +332,13 @@ static int virtio_mem_sbm_mb_states_prepare_next_mb= (struct virtio_mem *vm) * * Will not modify the state of the memory block. */ -static void virtio_mem_mb_set_sb_plugged(struct virtio_mem *vm, - unsigned long mb_id, int sb_id, - int count) +static void virtio_mem_sbm_set_sb_plugged(struct virtio_mem *vm, + unsigned long mb_id, int sb_id, + int count) { const int bit =3D (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb + sb_id; =20 - __bitmap_set(vm->sb_bitmap, bit, count); + __bitmap_set(vm->sbm.sb_states, bit, count); } =20 /* @@ -340,86 +346,87 @@ static void virtio_mem_mb_set_sb_plugged(struct vir= tio_mem *vm, * * Will not modify the state of the memory block. */ -static void virtio_mem_mb_set_sb_unplugged(struct virtio_mem *vm, - unsigned long mb_id, int sb_id, - int count) +static void virtio_mem_sbm_set_sb_unplugged(struct virtio_mem *vm, + unsigned long mb_id, int sb_id, + int count) { const int bit =3D (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb + sb_id; =20 - __bitmap_clear(vm->sb_bitmap, bit, count); + __bitmap_clear(vm->sbm.sb_states, bit, count); } =20 /* * Test if all selected subblocks are plugged. */ -static bool virtio_mem_mb_test_sb_plugged(struct virtio_mem *vm, - unsigned long mb_id, int sb_id, - int count) +static bool virtio_mem_sbm_test_sb_plugged(struct virtio_mem *vm, + unsigned long mb_id, int sb_id, + int count) { const int bit =3D (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb + sb_id; =20 if (count =3D=3D 1) - return test_bit(bit, vm->sb_bitmap); + return test_bit(bit, vm->sbm.sb_states); =20 /* TODO: Helper similar to bitmap_set() */ - return find_next_zero_bit(vm->sb_bitmap, bit + count, bit) >=3D + return find_next_zero_bit(vm->sbm.sb_states, bit + count, bit) >=3D bit + count; } =20 /* * Test if all selected subblocks are unplugged. */ -static bool virtio_mem_mb_test_sb_unplugged(struct virtio_mem *vm, - unsigned long mb_id, int sb_id, - int count) +static bool virtio_mem_sbm_test_sb_unplugged(struct virtio_mem *vm, + unsigned long mb_id, int sb_id, + int count) { const int bit =3D (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb + sb_id; =20 /* TODO: Helper similar to bitmap_set() */ - return find_next_bit(vm->sb_bitmap, bit + count, bit) >=3D bit + count; + return find_next_bit(vm->sbm.sb_states, bit + count, bit) >=3D + bit + count; } =20 /* * Find the first unplugged subblock. Returns vm->nb_sb_per_mb in case t= here is * none. */ -static int virtio_mem_mb_first_unplugged_sb(struct virtio_mem *vm, +static int virtio_mem_sbm_first_unplugged_sb(struct virtio_mem *vm, unsigned long mb_id) { const int bit =3D (mb_id - vm->first_mb_id) * vm->nb_sb_per_mb; =20 - return find_next_zero_bit(vm->sb_bitmap, bit + vm->nb_sb_per_mb, bit) - - bit; + return find_next_zero_bit(vm->sbm.sb_states, + bit + vm->nb_sb_per_mb, bit) - bit; } =20 /* * Prepare the subblock bitmap for the next memory block. */ -static int virtio_mem_sb_bitmap_prepare_next_mb(struct virtio_mem *vm) +static int virtio_mem_sbm_sb_states_prepare_next_mb(struct virtio_mem *v= m) { const unsigned long old_nb_mb =3D vm->next_mb_id - vm->first_mb_id; const unsigned long old_nb_bits =3D old_nb_mb * vm->nb_sb_per_mb; const unsigned long new_nb_bits =3D (old_nb_mb + 1) * vm->nb_sb_per_mb; int old_pages =3D PFN_UP(BITS_TO_LONGS(old_nb_bits) * sizeof(long)); int new_pages =3D PFN_UP(BITS_TO_LONGS(new_nb_bits) * sizeof(long)); - unsigned long *new_sb_bitmap, *old_sb_bitmap; + unsigned long *new_bitmap, *old_bitmap; =20 - if (vm->sb_bitmap && old_pages =3D=3D new_pages) + if (vm->sbm.sb_states && old_pages =3D=3D new_pages) return 0; =20 - new_sb_bitmap =3D vzalloc(new_pages * PAGE_SIZE); - if (!new_sb_bitmap) + new_bitmap =3D vzalloc(new_pages * PAGE_SIZE); + if (!new_bitmap) return -ENOMEM; =20 mutex_lock(&vm->hotplug_mutex); - if (new_sb_bitmap) - memcpy(new_sb_bitmap, vm->sb_bitmap, old_pages * PAGE_SIZE); + if (new_bitmap) + memcpy(new_bitmap, vm->sbm.sb_states, old_pages * PAGE_SIZE); =20 - old_sb_bitmap =3D vm->sb_bitmap; - vm->sb_bitmap =3D new_sb_bitmap; + old_bitmap =3D vm->sbm.sb_states; + vm->sbm.sb_states =3D new_bitmap; mutex_unlock(&vm->hotplug_mutex); =20 - vfree(old_sb_bitmap); + vfree(old_bitmap); return 0; } =20 @@ -630,7 +637,7 @@ static void virtio_mem_notify_going_offline(struct vi= rtio_mem *vm, int sb_id; =20 for (sb_id =3D 0; sb_id < vm->nb_sb_per_mb; sb_id++) { - if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) + if (virtio_mem_sbm_test_sb_plugged(vm, mb_id, sb_id, 1)) continue; pfn =3D PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + sb_id * vm->subblock_size); @@ -646,7 +653,7 @@ static void virtio_mem_notify_cancel_offline(struct v= irtio_mem *vm, int sb_id; =20 for (sb_id =3D 0; sb_id < vm->nb_sb_per_mb; sb_id++) { - if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) + if (virtio_mem_sbm_test_sb_plugged(vm, mb_id, sb_id, 1)) continue; pfn =3D PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + sb_id * vm->subblock_size); @@ -936,7 +943,7 @@ static void virtio_mem_online_page_cb(struct page *pa= ge, unsigned int order) * If plugged, online the pages, otherwise, set them fake * offline (PageOffline). */ - if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) + if (virtio_mem_sbm_test_sb_plugged(vm, mb_id, sb_id, 1)) generic_online_page(page, order); else virtio_mem_set_fake_offline(PFN_DOWN(addr), 1 << order, @@ -1071,7 +1078,7 @@ static int virtio_mem_mb_plug_sb(struct virtio_mem = *vm, unsigned long mb_id, =20 rc =3D virtio_mem_send_plug_request(vm, addr, size); if (!rc) - virtio_mem_mb_set_sb_plugged(vm, mb_id, sb_id, count); + virtio_mem_sbm_set_sb_plugged(vm, mb_id, sb_id, count); return rc; } =20 @@ -1092,7 +1099,7 @@ static int virtio_mem_mb_unplug_sb(struct virtio_me= m *vm, unsigned long mb_id, =20 rc =3D virtio_mem_send_unplug_request(vm, addr, size); if (!rc) - virtio_mem_mb_set_sb_unplugged(vm, mb_id, sb_id, count); + virtio_mem_sbm_set_sb_unplugged(vm, mb_id, sb_id, count); return rc; } =20 @@ -1115,14 +1122,14 @@ static int virtio_mem_mb_unplug_any_sb(struct vir= tio_mem *vm, while (*nb_sb) { /* Find the next candidate subblock */ while (sb_id >=3D 0 && - virtio_mem_mb_test_sb_unplugged(vm, mb_id, sb_id, 1)) + virtio_mem_sbm_test_sb_unplugged(vm, mb_id, sb_id, 1)) sb_id--; if (sb_id < 0) break; /* Try to unplug multiple subblocks at a time */ count =3D 1; while (count < *nb_sb && sb_id > 0 && - virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id - 1, 1)) { + virtio_mem_sbm_test_sb_plugged(vm, mb_id, sb_id - 1, 1)) { count++; sb_id--; } @@ -1168,7 +1175,7 @@ static int virtio_mem_prepare_next_mb(struct virtio= _mem *vm, return rc; =20 /* Resize the subblock bitmap if required. */ - rc =3D virtio_mem_sb_bitmap_prepare_next_mb(vm); + rc =3D virtio_mem_sbm_sb_states_prepare_next_mb(vm); if (rc) return rc; =20 @@ -1253,14 +1260,13 @@ static int virtio_mem_mb_plug_any_sb(struct virti= o_mem *vm, unsigned long mb_id, return -EINVAL; =20 while (*nb_sb) { - sb_id =3D virtio_mem_mb_first_unplugged_sb(vm, mb_id); + sb_id =3D virtio_mem_sbm_first_unplugged_sb(vm, mb_id); if (sb_id >=3D vm->nb_sb_per_mb) break; count =3D 1; while (count < *nb_sb && sb_id + count < vm->nb_sb_per_mb && - !virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id + count, - 1)) + !virtio_mem_sbm_test_sb_plugged(vm, mb_id, sb_id + count, 1)) count++; =20 rc =3D virtio_mem_mb_plug_sb(vm, mb_id, sb_id, count); @@ -1277,7 +1283,7 @@ static int virtio_mem_mb_plug_any_sb(struct virtio_= mem *vm, unsigned long mb_id, virtio_mem_fake_online(pfn, nr_pages); } =20 - if (virtio_mem_mb_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { + if (virtio_mem_sbm_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { if (online) virtio_mem_sbm_set_mb_state(vm, mb_id, VIRTIO_MEM_SBM_MB_ONLINE); @@ -1377,13 +1383,13 @@ static int virtio_mem_mb_unplug_any_sb_offline(st= ruct virtio_mem *vm, rc =3D virtio_mem_mb_unplug_any_sb(vm, mb_id, nb_sb); =20 /* some subblocks might have been unplugged even on failure */ - if (!virtio_mem_mb_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb)) + if (!virtio_mem_sbm_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb)) virtio_mem_sbm_set_mb_state(vm, mb_id, VIRTIO_MEM_SBM_MB_OFFLINE_PARTIAL); if (rc) return rc; =20 - if (virtio_mem_mb_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { + if (virtio_mem_sbm_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { /* * Remove the block from Linux - this should never fail. * Hinder the block from getting onlined by marking it @@ -1452,7 +1458,7 @@ static int virtio_mem_mb_unplug_any_sb_online(struc= t virtio_mem *vm, =20 /* If possible, try to unplug the complete block in one shot. */ if (*nb_sb >=3D vm->nb_sb_per_mb && - virtio_mem_mb_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { + virtio_mem_sbm_test_sb_plugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { rc =3D virtio_mem_mb_unplug_sb_online(vm, mb_id, 0, vm->nb_sb_per_mb); if (!rc) { @@ -1466,7 +1472,7 @@ static int virtio_mem_mb_unplug_any_sb_online(struc= t virtio_mem *vm, for (sb_id =3D vm->nb_sb_per_mb - 1; sb_id >=3D 0 && *nb_sb; sb_id--) { /* Find the next candidate subblock */ while (sb_id >=3D 0 && - !virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) + !virtio_mem_sbm_test_sb_plugged(vm, mb_id, sb_id, 1)) sb_id--; if (sb_id < 0) break; @@ -1485,7 +1491,7 @@ static int virtio_mem_mb_unplug_any_sb_online(struc= t virtio_mem *vm, * remove it. This will usually not fail, as no memory is in use * anymore - however some other notifiers might NACK the request. */ - if (virtio_mem_mb_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { + if (virtio_mem_sbm_test_sb_unplugged(vm, mb_id, 0, vm->nb_sb_per_mb)) { mutex_unlock(&vm->hotplug_mutex); rc =3D virtio_mem_mb_offline_and_remove(vm, mb_id); mutex_lock(&vm->hotplug_mutex); @@ -2007,7 +2013,7 @@ static void virtio_mem_remove(struct virtio_device = *vdev) =20 /* remove all tracking data - no locking needed */ vfree(vm->sbm.mb_states); - vfree(vm->sb_bitmap); + vfree(vm->sbm.sb_states); =20 /* reset the device and cleanup the queues */ vdev->config->reset(vdev); --=20 2.26.2