From: Gavin Shan <gshan@redhat.com>
To: Oliver Upton <oliver.upton@linux.dev>, Peter Xu <peterx@redhat.com>
Cc: kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu,
kvm@vger.kernel.org, catalin.marinas@arm.com, bgardon@google.com,
shuah@kernel.org, andrew.jones@linux.dev, will@kernel.org,
dmatlack@google.com, pbonzini@redhat.com, zhenyzha@redhat.com,
james.morse@arm.com, suzuki.poulose@arm.com,
alexandru.elisei@arm.com, seanjc@google.com,
shan.gavin@gmail.com
Subject: Re: [PATCH v5 3/7] KVM: x86: Allow to use bitmap in ring-based dirty page tracking
Date: Tue, 11 Oct 2022 11:56:09 +0800 [thread overview]
Message-ID: <6440b74c-9ebc-12f4-dd4e-469376a434f2@redhat.com> (raw)
In-Reply-To: <Y0TDCxfVVme8uPGU@google.com>
On 10/11/22 9:12 AM, Oliver Upton wrote:
> On Mon, Oct 10, 2022 at 08:20:29PM -0400, Peter Xu wrote:
>> On Mon, Oct 10, 2022 at 11:58:22PM +0000, Oliver Upton wrote:
>>> I think this further drives the point home -- there's zero need for the
>>> bitmap with dirty ring on x86, so why even support it? The proposal of
>>> ALLOW_BITMAP && DIRTY_RING should be arm64-specific. Any other arch that
>>> needs to dirty memory outside of a vCPU context can opt-in to the
>>> behavior.
>>
>> Yeah that sounds working too, but it'll be slightly hackish as then the
>> user app will need some "#ifdef ARM64" blocks for e.g. sync dirty bitmap.
>> With the new cap the user app can implement the whole ring with generic
>> code.
>
> Isn't the current route of exposing ALLOW_BITMAP on other arches for no
> reason headed in exactly that direction? Userspace would need to know if
> it _really_ needs the dirty bitmap in addition to the dirty ring, which
> could take the form of architecture ifdeffery.
>
> OTOH, if the cap is only exposed when it is absolutely necessary, an
> arch-generic live migration implementation could enable the cap whenever
> it is advertized and scan the bitmap accordingly.
>
> The VMM must know something about the architecture it is running on, as
> it calls KVM_DEV_ARM_ITS_SAVE_TABLES after all...
>
It looks good to me by using CONFIG_HAVE_KVM_DIRTY_RING_USE_BITMAP to
opt-in KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP. The most important point is
to ensure 'kvm->dirty_ring_with_bitmap == true' when dirty ring capability
is enabled. In this way, we can fail early when KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP
isn't enabled on attempt to enable dirty ring capability.
If both of you agree, I will integrate the suggested code changes in
next respin, with necessary tweak.
- In kvm_vm_ioctl_enable_cap_generic(), 'kvm->dirty_ring_with_bitmap' is
updated to 'true' unconditionally.
static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm,
struct kvm_enable_cap *cap)
{
:
case KVM_CAP_DIRTY_LOG_RING_WITH_BITMAP:
kvm->dirty_ring_with_bitmap = true;
return 0;
}
- In mark_page_dirty_in_slot(), we need comprehensive checks like below.
void mark_page_dirty_in_slot(struct kvm *kvm,
const struct kvm_memory_slot *memslot,
gfn_t gfn)
{
#ifdef CONFIG_HAVE_KVM_DIRTY_RING
if (WARN_ON_ONCE(vcpu && vcpu->kvm != kvm))
return;
#ifndef CONFIG_HAVE_KVM_DIRTY_RING_WITH_BITMAP
if (WARN_ON_ONCE(!vcpu))
return;
#endif
#endif
}
- Use kvm_dirty_ring_exclusive(), which was suggested by Peter before.
The function is used in various spots to allow the dirty bitmap is
created and accessed.
bool kvm_dirty_ring_exclusive(struct kvm *kvm)
{
return kvm->dirty_ring_size && !kvm->dirty_ring_with_bitmap;
}
>> Also more flexible to expose it as generic cap? E.g., one day x86 can
>> enable this too for whatever reason (even though I don't think so..).
>
> I had imagined something like this patch where the arch opts-in to some
> generic construct if it *requires* the use of both the ring and bitmap
> (very rough sketch).
>
Thanks,
Gavin
next prev parent reply other threads:[~2022-10-11 3:56 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-05 0:41 [PATCH v5 0/7] KVM: arm64: Enable ring-based dirty memory tracking Gavin Shan
2022-10-05 0:41 ` [PATCH v5 1/7] KVM: x86: Introduce KVM_REQ_RING_SOFT_FULL Gavin Shan
2022-10-05 0:41 ` [PATCH v5 2/7] KVM: x86: Move declaration of kvm_cpu_dirty_log_size() to kvm_dirty_ring.h Gavin Shan
2022-10-05 0:41 ` [PATCH v5 3/7] KVM: x86: Allow to use bitmap in ring-based dirty page tracking Gavin Shan
2022-10-06 20:28 ` Peter Xu
2022-10-06 23:38 ` Gavin Shan
2022-10-07 14:31 ` Peter Xu
2022-10-10 23:18 ` Oliver Upton
2022-10-10 23:43 ` Oliver Upton
2022-10-10 23:49 ` Peter Xu
2022-10-10 23:58 ` Gavin Shan
2022-10-10 23:58 ` Oliver Upton
2022-10-11 0:20 ` Peter Xu
2022-10-11 1:12 ` Oliver Upton
2022-10-11 3:56 ` Gavin Shan [this message]
2022-10-11 6:31 ` Gavin Shan
2022-10-14 16:55 ` Peter Xu
2022-10-18 7:38 ` Oliver Upton
2022-10-18 7:40 ` Oliver Upton
2022-10-18 15:50 ` Peter Xu
2022-10-05 0:41 ` [PATCH v5 4/7] KVM: arm64: Enable ring-based dirty memory tracking Gavin Shan
2022-10-05 0:41 ` [PATCH v5 5/7] KVM: selftests: Use host page size to map ring buffer in dirty_log_test Gavin Shan
2022-10-05 0:41 ` [PATCH v5 6/7] KVM: selftests: Clear dirty ring states between two modes " Gavin Shan
2022-10-05 0:41 ` [PATCH v5 7/7] KVM: selftests: Automate choosing dirty ring size " Gavin Shan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6440b74c-9ebc-12f4-dd4e-469376a434f2@redhat.com \
--to=gshan@redhat.com \
--cc=alexandru.elisei@arm.com \
--cc=andrew.jones@linux.dev \
--cc=bgardon@google.com \
--cc=catalin.marinas@arm.com \
--cc=dmatlack@google.com \
--cc=james.morse@arm.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=kvmarm@lists.linux.dev \
--cc=oliver.upton@linux.dev \
--cc=pbonzini@redhat.com \
--cc=peterx@redhat.com \
--cc=seanjc@google.com \
--cc=shan.gavin@gmail.com \
--cc=shuah@kernel.org \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
--cc=zhenyzha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).