From: Gavin Shan <gshan@redhat.com> To: Oliver Upton <oliver.upton@linux.dev>, Peter Xu <peterx@redhat.com> Cc: kvm@vger.kernel.org, catalin.marinas@arm.com, andrew.jones@linux.dev, dmatlack@google.com, will@kernel.org, shan.gavin@gmail.com, bgardon@google.com, kvmarm@lists.linux.dev, pbonzini@redhat.com, zhenyzha@redhat.com, shuah@kernel.org, kvmarm@lists.cs.columbia.edu Subject: Re: [PATCH v5 3/7] KVM: x86: Allow to use bitmap in ring-based dirty page tracking Date: Tue, 11 Oct 2022 14:31:17 +0800 [thread overview] Message-ID: <53ac3666-46b3-8134-2e23-2840a16d333e@redhat.com> (raw) In-Reply-To: <6440b74c-9ebc-12f4-dd4e-469376a434f2@redhat.com> Hi Peter/Oliver, On 10/11/22 11:56 AM, Gavin Shan wrote: > On 10/11/22 9:12 AM, Oliver Upton wrote: >> On Mon, Oct 10, 2022 at 08:20:29PM -0400, Peter Xu wrote: >>> On Mon, Oct 10, 2022 at 11:58:22PM +0000, Oliver Upton wrote: >>>> I think this further drives the point home -- there's zero need for the >>>> bitmap with dirty ring on x86, so why even support it? The proposal of >>>> ALLOW_BITMAP && DIRTY_RING should be arm64-specific. Any other arch that >>>> needs to dirty memory outside of a vCPU context can opt-in to the >>>> behavior. >>> >>> Yeah that sounds working too, but it'll be slightly hackish as then the >>> user app will need some "#ifdef ARM64" blocks for e.g. sync dirty bitmap. >>> With the new cap the user app can implement the whole ring with generic >>> code. >> >> Isn't the current route of exposing ALLOW_BITMAP on other arches for no >> reason headed in exactly that direction? Userspace would need to know if >> it _really_ needs the dirty bitmap in addition to the dirty ring, which >> could take the form of architecture ifdeffery. >> >> OTOH, if the cap is only exposed when it is absolutely necessary, an >> arch-generic live migration implementation could enable the cap whenever >> it is advertized and scan the bitmap accordingly. >> >> The VMM must know something about the architecture it is running on, as >> it calls KVM_DEV_ARM_ITS_SAVE_TABLES after all... >> > > It looks good to me by using CONFIG_HAVE_KVM_DIRTY_RING_USE_BITMAP to > opt-in KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP. The most important point is > to ensure 'kvm->dirty_ring_with_bitmap == true' when dirty ring capability > is enabled. In this way, we can fail early when KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP > isn't enabled on attempt to enable dirty ring capability. > > If both of you agree, I will integrate the suggested code changes in > next respin, with necessary tweak. > > - In kvm_vm_ioctl_enable_cap_generic(), 'kvm->dirty_ring_with_bitmap' is > updated to 'true' unconditionally. > > static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm, > struct kvm_enable_cap *cap) > { > : > case KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP: > kvm->dirty_ring_with_bitmap = true; > return 0; > } > > - In mark_page_dirty_in_slot(), we need comprehensive checks like below. > > void mark_page_dirty_in_slot(struct kvm *kvm, > const struct kvm_memory_slot *memslot, > gfn_t gfn) > { > #ifdef CONFIG_HAVE_KVM_DIRTY_RING > if (WARN_ON_ONCE(vcpu && vcpu->kvm != kvm)) > return; > > #ifndef CONFIG_HAVE_KVM_DIRTY_RING_WITH_BITMAP > if (WARN_ON_ONCE(!vcpu)) > return; > #endif > #endif > > } > > - Use kvm_dirty_ring_exclusive(), which was suggested by Peter before. > The function is used in various spots to allow the dirty bitmap is > created and accessed. > > bool kvm_dirty_ring_exclusive(struct kvm *kvm) > { > return kvm->dirty_ring_size && !kvm->dirty_ring_with_bitmap; > } > > I've included Oliver's suggested changes into v6, which was just posted: https://lore.kernel.org/kvmarm/3123a04f-a674-782b-9e9b-0baf3db49ebc@redhat.com/ Please find your time to review v6 directly, thanks! >>> Also more flexible to expose it as generic cap? E.g., one day x86 can >>> enable this too for whatever reason (even though I don't think so..). >> >> I had imagined something like this patch where the arch opts-in to some >> generic construct if it *requires* the use of both the ring and bitmap >> (very rough sketch). >> Thanks, Gavin _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
WARNING: multiple messages have this Message-ID (diff)
From: Gavin Shan <gshan@redhat.com> To: Oliver Upton <oliver.upton@linux.dev>, Peter Xu <peterx@redhat.com> Cc: kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, catalin.marinas@arm.com, bgardon@google.com, shuah@kernel.org, andrew.jones@linux.dev, will@kernel.org, dmatlack@google.com, pbonzini@redhat.com, zhenyzha@redhat.com, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, seanjc@google.com, shan.gavin@gmail.com Subject: Re: [PATCH v5 3/7] KVM: x86: Allow to use bitmap in ring-based dirty page tracking Date: Tue, 11 Oct 2022 14:31:17 +0800 [thread overview] Message-ID: <53ac3666-46b3-8134-2e23-2840a16d333e@redhat.com> (raw) Message-ID: <20221011063117.SdnC-SQ5NtrZ24ilGXfAzgp7iRPuypiC_mZNxjTjgnQ@z> (raw) In-Reply-To: <6440b74c-9ebc-12f4-dd4e-469376a434f2@redhat.com> Hi Peter/Oliver, On 10/11/22 11:56 AM, Gavin Shan wrote: > On 10/11/22 9:12 AM, Oliver Upton wrote: >> On Mon, Oct 10, 2022 at 08:20:29PM -0400, Peter Xu wrote: >>> On Mon, Oct 10, 2022 at 11:58:22PM +0000, Oliver Upton wrote: >>>> I think this further drives the point home -- there's zero need for the >>>> bitmap with dirty ring on x86, so why even support it? The proposal of >>>> ALLOW_BITMAP && DIRTY_RING should be arm64-specific. Any other arch that >>>> needs to dirty memory outside of a vCPU context can opt-in to the >>>> behavior. >>> >>> Yeah that sounds working too, but it'll be slightly hackish as then the >>> user app will need some "#ifdef ARM64" blocks for e.g. sync dirty bitmap. >>> With the new cap the user app can implement the whole ring with generic >>> code. >> >> Isn't the current route of exposing ALLOW_BITMAP on other arches for no >> reason headed in exactly that direction? Userspace would need to know if >> it _really_ needs the dirty bitmap in addition to the dirty ring, which >> could take the form of architecture ifdeffery. >> >> OTOH, if the cap is only exposed when it is absolutely necessary, an >> arch-generic live migration implementation could enable the cap whenever >> it is advertized and scan the bitmap accordingly. >> >> The VMM must know something about the architecture it is running on, as >> it calls KVM_DEV_ARM_ITS_SAVE_TABLES after all... >> > > It looks good to me by using CONFIG_HAVE_KVM_DIRTY_RING_USE_BITMAP to > opt-in KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP. The most important point is > to ensure 'kvm->dirty_ring_with_bitmap == true' when dirty ring capability > is enabled. In this way, we can fail early when KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP > isn't enabled on attempt to enable dirty ring capability. > > If both of you agree, I will integrate the suggested code changes in > next respin, with necessary tweak. > > - In kvm_vm_ioctl_enable_cap_generic(), 'kvm->dirty_ring_with_bitmap' is > updated to 'true' unconditionally. > > static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm, > struct kvm_enable_cap *cap) > { > : > case KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP: > kvm->dirty_ring_with_bitmap = true; > return 0; > } > > - In mark_page_dirty_in_slot(), we need comprehensive checks like below. > > void mark_page_dirty_in_slot(struct kvm *kvm, > const struct kvm_memory_slot *memslot, > gfn_t gfn) > { > #ifdef CONFIG_HAVE_KVM_DIRTY_RING > if (WARN_ON_ONCE(vcpu && vcpu->kvm != kvm)) > return; > > #ifndef CONFIG_HAVE_KVM_DIRTY_RING_WITH_BITMAP > if (WARN_ON_ONCE(!vcpu)) > return; > #endif > #endif > > } > > - Use kvm_dirty_ring_exclusive(), which was suggested by Peter before. > The function is used in various spots to allow the dirty bitmap is > created and accessed. > > bool kvm_dirty_ring_exclusive(struct kvm *kvm) > { > return kvm->dirty_ring_size && !kvm->dirty_ring_with_bitmap; > } > > I've included Oliver's suggested changes into v6, which was just posted: https://lore.kernel.org/kvmarm/3123a04f-a674-782b-9e9b-0baf3db49ebc@redhat.com/ Please find your time to review v6 directly, thanks! >>> Also more flexible to expose it as generic cap? E.g., one day x86 can >>> enable this too for whatever reason (even though I don't think so..). >> >> I had imagined something like this patch where the arch opts-in to some >> generic construct if it *requires* the use of both the ring and bitmap >> (very rough sketch). >> Thanks, Gavin
next prev parent reply other threads:[~2022-10-11 6:31 UTC|newest] Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-10-05 0:41 [PATCH v5 0/7] KVM: arm64: Enable ring-based dirty memory tracking Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-05 0:41 ` [PATCH v5 1/7] KVM: x86: Introduce KVM_REQ_RING_SOFT_FULL Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-05 0:41 ` [PATCH v5 2/7] KVM: x86: Move declaration of kvm_cpu_dirty_log_size() to kvm_dirty_ring.h Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-05 0:41 ` [PATCH v5 3/7] KVM: x86: Allow to use bitmap in ring-based dirty page tracking Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-06 20:28 ` Peter Xu 2022-10-06 20:28 ` Peter Xu 2022-10-06 23:38 ` Gavin Shan 2022-10-06 23:38 ` Gavin Shan 2022-10-07 14:31 ` Peter Xu 2022-10-07 14:31 ` Peter Xu 2022-10-10 23:18 ` Oliver Upton 2022-10-10 23:18 ` Oliver Upton 2022-10-10 23:43 ` Oliver Upton 2022-10-10 23:43 ` Oliver Upton 2022-10-10 23:49 ` Peter Xu 2022-10-10 23:49 ` Peter Xu 2022-10-10 23:58 ` Gavin Shan 2022-10-10 23:58 ` Gavin Shan 2022-10-10 23:58 ` Oliver Upton 2022-10-10 23:58 ` Oliver Upton 2022-10-11 0:20 ` Peter Xu 2022-10-11 0:20 ` Peter Xu 2022-10-11 1:12 ` Oliver Upton 2022-10-11 1:12 ` Oliver Upton 2022-10-11 3:56 ` Gavin Shan 2022-10-11 3:56 ` Gavin Shan 2022-10-11 6:31 ` Gavin Shan [this message] 2022-10-11 6:31 ` Gavin Shan 2022-10-14 16:55 ` Peter Xu 2022-10-14 16:55 ` Peter Xu 2022-10-18 7:38 ` Oliver Upton 2022-10-18 7:38 ` Oliver Upton 2022-10-18 7:40 ` Oliver Upton 2022-10-18 7:40 ` Oliver Upton 2022-10-18 15:50 ` Peter Xu 2022-10-18 15:50 ` Peter Xu 2022-10-05 0:41 ` [PATCH v5 4/7] KVM: arm64: Enable ring-based dirty memory tracking Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-05 0:41 ` [PATCH v5 5/7] KVM: selftests: Use host page size to map ring buffer in dirty_log_test Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-05 0:41 ` [PATCH v5 6/7] KVM: selftests: Clear dirty ring states between two modes " Gavin Shan 2022-10-05 0:41 ` Gavin Shan 2022-10-05 0:41 ` [PATCH v5 7/7] KVM: selftests: Automate choosing dirty ring size " Gavin Shan 2022-10-05 0:41 ` Gavin Shan
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=53ac3666-46b3-8134-2e23-2840a16d333e@redhat.com \ --to=gshan@redhat.com \ --cc=andrew.jones@linux.dev \ --cc=bgardon@google.com \ --cc=catalin.marinas@arm.com \ --cc=dmatlack@google.com \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=kvmarm@lists.linux.dev \ --cc=oliver.upton@linux.dev \ --cc=pbonzini@redhat.com \ --cc=peterx@redhat.com \ --cc=shan.gavin@gmail.com \ --cc=shuah@kernel.org \ --cc=will@kernel.org \ --cc=zhenyzha@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).