KVM Archive on lore.kernel.org
 help / color / Atom feed
From: Yan Zhao <yan.y.zhao@intel.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: 'Paolo Bonzini' <pbonzini@redhat.com>,
	Peter Xu <peterx@redhat.com>,
	"Christopherson, Sean J" <sean.j.christopherson@intel.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	"Wang, Zhenyu Z" <zhenyu.z.wang@intel.com>
Subject: Re: [PATCH RFC 04/15] KVM: Implement ring-based dirty memory tracking
Date: Tue, 17 Dec 2019 00:25:02 -0500
Message-ID: <20191217052502.GF21868@joy-OptiPlex-7040> (raw)
In-Reply-To: <AADFC41AFE54684AB9EE6CBC0274A5D19D646148@SHSMSX104.ccr.corp.intel.com>

On Tue, Dec 17, 2019 at 01:17:29PM +0800, Tian, Kevin wrote:
> > From: Tian, Kevin
> > Sent: Tuesday, December 17, 2019 10:29 AM
> > 
> > > From: Paolo Bonzini
> > > Sent: Monday, December 16, 2019 6:08 PM
> > >
> > > [Alex and Kevin: there are doubts below regarding dirty page tracking
> > > from VFIO and mdev devices, which perhaps you can help with]
> > >
> > > On 15/12/19 18:21, Peter Xu wrote:
> > > >                 init_rmode_tss
> > > >                     vmx_set_tss_addr
> > > >                         kvm_vm_ioctl_set_tss_addr [*]
> > > >                 init_rmode_identity_map
> > > >                     vmx_create_vcpu [*]
> > >
> > > These don't matter because their content is not visible to userspace
> > > (the backing storage is mmap-ed by __x86_set_memory_region).  In fact, d
> > >
> > > >                 vmx_write_pml_buffer
> > > >                     kvm_arch_write_log_dirty [&]
> > > >                 kvm_write_guest
> > > >                     kvm_hv_setup_tsc_page
> > > >                         kvm_guest_time_update [&]
> > > >                     nested_flush_cached_shadow_vmcs12 [&]
> > > >                     kvm_write_wall_clock [&]
> > > >                     kvm_pv_clock_pairing [&]
> > > >                     kvmgt_rw_gpa [?]
> > >
> > > This then expands (partially) to
> > >
> > > intel_gvt_hypervisor_write_gpa
> > >     emulate_csb_update
> > >         emulate_execlist_ctx_schedule_out
> > >             complete_execlist_workload
> > >                 complete_current_workload
> > >                      workload_thread
> > >         emulate_execlist_ctx_schedule_in
> > >             prepare_execlist_workload
> > >                 prepare_workload
> > >                     dispatch_workload
> > >                         workload_thread
> > >
> > > So KVMGT is always writing to GPAs instead of IOVAs and basically
> > > bypassing a guest IOMMU.  So here it would be better if kvmgt was
> > > changed not use kvm_write_guest (also because I'd probably have nacked
> > > that if I had known :)).
> > 
> > I agree.
> > 
> > >
> > > As far as I know, there is some work on live migration with both VFIO
> > > and mdev, and that probably includes some dirty page tracking API.
> > > kvmgt could switch to that API, or there could be VFIO APIs similar to
> > > kvm_write_guest but taking IOVAs instead of GPAs.  Advantage: this would
> > > fix the GPA/IOVA confusion.  Disadvantage: userspace would lose the
> > > tracking of writes from mdev devices.  Kevin, are these writes used in
> > > any way?  Do the calls to intel_gvt_hypervisor_write_gpa covers all
> > > writes from kvmgt vGPUs, or can the hardware write to memory as well
> > > (which would be my guess if I didn't know anything about kvmgt, which I
> > > pretty much don't)?
> > 
> > intel_gvt_hypervisor_write_gpa covers all writes due to software mediation.
> > 
> > for hardware updates, it needs be mapped in IOMMU through
> > vfio_pin_pages
> > before any DMA happens. The ongoing dirty tracking effort in VFIO will take
> > every pinned page through that API as dirtied.
> > 
> > However, currently VFIO doesn't implement any vfio_read/write_guest
> > interface yet. and it doesn't make sense to use vfio_pin_pages for software
> > dirtied pages, as pin is unnecessary and heavy involving iommu invalidation.
> 
> One correction. vfio_pin_pages doesn't involve iommu invalidation. I should
> just mean that pinning the page is not necessary. We just need a kvm-like
> interface based on hva to access.
>
And can we propose to differentiate read and write when calling vfio_pin_pages, e.g.
vfio_pin_pages_read, vfio_pin_pages_write? Otherwise, calling to
vfio_pin_pages will unnecessarily cause read pages to be dirty and
sometimes reading guest pages is a way for device model to track dirty
pages.

> > 
> > Alex, if you are OK we'll work on such interface and move kvmgt to use it.
> > After it's accepted, we can also mark pages dirty through this new interface
> > in Kirti's dirty page tracking series.
> > 

  reply index

Thread overview: 121+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-29 21:34 [PATCH RFC 00/15] KVM: Dirty ring interface Peter Xu
2019-11-29 21:34 ` [PATCH RFC 01/15] KVM: Move running VCPU from ARM to common code Peter Xu
2019-12-03 19:01   ` Sean Christopherson
2019-12-04  9:42     ` Paolo Bonzini
2019-12-09 22:05       ` Peter Xu
2019-11-29 21:34 ` [PATCH RFC 02/15] KVM: Add kvm/vcpu argument to mark_dirty_page_in_slot Peter Xu
2019-12-02 19:32   ` Sean Christopherson
2019-12-02 20:49     ` Peter Xu
2019-11-29 21:34 ` [PATCH RFC 03/15] KVM: Add build-time error check on kvm_run size Peter Xu
2019-12-02 19:30   ` Sean Christopherson
2019-12-02 20:53     ` Peter Xu
2019-12-02 22:19       ` Sean Christopherson
2019-12-02 22:40         ` Peter Xu
2019-12-03  5:50           ` Sean Christopherson
2019-12-03 13:41         ` Paolo Bonzini
2019-12-03 17:04           ` Peter Xu
2019-11-29 21:34 ` [PATCH RFC 04/15] KVM: Implement ring-based dirty memory tracking Peter Xu
2019-12-02 20:10   ` Sean Christopherson
2019-12-02 21:16     ` Peter Xu
2019-12-02 21:50       ` Sean Christopherson
2019-12-02 23:09         ` Peter Xu
2019-12-03 13:48         ` Paolo Bonzini
2019-12-03 18:46           ` Sean Christopherson
2019-12-04 10:05             ` Paolo Bonzini
2019-12-07  0:29               ` Sean Christopherson
2019-12-09  9:37                 ` Paolo Bonzini
2019-12-09 21:54               ` Peter Xu
2019-12-10 10:07                 ` Paolo Bonzini
2019-12-10 15:52                   ` Peter Xu
2019-12-10 17:09                     ` Paolo Bonzini
2019-12-15 17:21                       ` Peter Xu
2019-12-16 10:08                         ` Paolo Bonzini
2019-12-16 18:54                           ` Peter Xu
2019-12-17  9:01                             ` Paolo Bonzini
2019-12-17 16:24                               ` Peter Xu
2019-12-17 16:28                                 ` Paolo Bonzini
2019-12-18 21:58                                   ` Peter Xu
2019-12-18 22:24                                     ` Sean Christopherson
2019-12-18 22:37                                       ` Paolo Bonzini
2019-12-18 22:49                                         ` Peter Xu
2019-12-17  2:28                           ` Tian, Kevin
2019-12-17 16:18                             ` Alex Williamson
2019-12-17 16:30                               ` Paolo Bonzini
2019-12-18  0:29                                 ` Tian, Kevin
     [not found]                           ` <AADFC41AFE54684AB9EE6CBC0274A5D19D645E5F@SHSMSX104.ccr.corp.intel.com>
2019-12-17  5:17                             ` Tian, Kevin
2019-12-17  5:25                               ` Yan Zhao [this message]
2019-12-17 16:24                                 ` Alex Williamson
2019-12-03 19:13   ` Sean Christopherson
2019-12-04 10:14     ` Paolo Bonzini
2019-12-04 14:33       ` Sean Christopherson
2019-12-04 10:38   ` Jason Wang
2019-12-04 11:04     ` Paolo Bonzini
2019-12-04 19:52       ` Peter Xu
2019-12-05  6:51         ` Jason Wang
2019-12-05 12:08           ` Peter Xu
2019-12-05 13:12             ` Jason Wang
2019-12-10 13:25       ` Michael S. Tsirkin
2019-12-10 13:31         ` Paolo Bonzini
2019-12-10 16:02           ` Peter Xu
2019-12-10 21:53             ` Michael S. Tsirkin
2019-12-11  9:05               ` Paolo Bonzini
2019-12-11 13:04                 ` Michael S. Tsirkin
2019-12-11 14:54                   ` Peter Xu
2019-12-10 21:48           ` Michael S. Tsirkin
2019-12-11 12:53   ` Michael S. Tsirkin
2019-12-11 14:14     ` Paolo Bonzini
2019-12-11 20:59     ` Peter Xu
2019-12-11 22:57       ` Michael S. Tsirkin
2019-12-12  0:08         ` Paolo Bonzini
2019-12-12  7:36           ` Michael S. Tsirkin
2019-12-12  8:12             ` Paolo Bonzini
2019-12-12 10:38               ` Michael S. Tsirkin
2019-12-15 17:33           ` Peter Xu
2019-12-16  9:47             ` Michael S. Tsirkin
2019-12-16 15:07               ` Peter Xu
2019-12-16 15:33                 ` Michael S. Tsirkin
2019-12-16 15:47                   ` Peter Xu
2019-12-11 17:24   ` Christophe de Dinechin
2019-12-13 20:23     ` Peter Xu
2019-12-14  7:57       ` Paolo Bonzini
2019-12-14 16:26         ` Peter Xu
2019-12-16  9:29           ` Paolo Bonzini
2019-12-16 15:26             ` Peter Xu
2019-12-16 15:31               ` Paolo Bonzini
2019-12-16 15:43                 ` Peter Xu
2019-12-17 12:16         ` Christophe de Dinechin
2019-12-17 12:19           ` Paolo Bonzini
2019-12-17 15:38             ` Peter Xu
2019-12-17 16:31               ` Paolo Bonzini
2019-12-17 16:42                 ` Peter Xu
2019-12-17 16:48                   ` Paolo Bonzini
2019-12-17 19:41                     ` Peter Xu
2019-12-18  0:33                       ` Paolo Bonzini
2019-12-18 16:32                         ` Peter Xu
2019-12-18 16:41                           ` Paolo Bonzini
2019-12-20 18:19       ` Peter Xu
2019-11-29 21:34 ` [PATCH RFC 05/15] KVM: Make dirty ring exclusive to dirty bitmap log Peter Xu
2019-11-29 21:34 ` [PATCH RFC 06/15] KVM: Introduce dirty ring wait queue Peter Xu
2019-11-29 21:34 ` [PATCH RFC 07/15] KVM: X86: Implement ring-based dirty memory tracking Peter Xu
2019-11-29 21:34 ` [PATCH RFC 08/15] KVM: selftests: Always clear dirty bitmap after iteration Peter Xu
2019-11-29 21:34 ` [PATCH RFC 09/15] KVM: selftests: Sync uapi/linux/kvm.h to tools/ Peter Xu
2019-11-29 21:35 ` [PATCH RFC 10/15] KVM: selftests: Use a single binary for dirty/clear log test Peter Xu
2019-11-29 21:35 ` [PATCH RFC 11/15] KVM: selftests: Introduce after_vcpu_run hook for dirty " Peter Xu
2019-11-29 21:35 ` [PATCH RFC 12/15] KVM: selftests: Add dirty ring buffer test Peter Xu
2019-11-29 21:35 ` [PATCH RFC 13/15] KVM: selftests: Let dirty_log_test async for dirty ring test Peter Xu
2019-11-29 21:35 ` [PATCH RFC 14/15] KVM: selftests: Add "-c" parameter to dirty log test Peter Xu
2019-11-29 21:35 ` [PATCH RFC 15/15] KVM: selftests: Test dirty ring waitqueue Peter Xu
2019-11-30  8:29 ` [PATCH RFC 00/15] KVM: Dirty ring interface Paolo Bonzini
2019-12-02  2:13   ` Peter Xu
2019-12-03 13:59     ` Paolo Bonzini
2019-12-05 19:30       ` Peter Xu
2019-12-05 19:59         ` Paolo Bonzini
2019-12-05 20:52           ` Peter Xu
2019-12-02 20:21   ` Sean Christopherson
2019-12-02 20:43     ` Peter Xu
2019-12-04 10:39 ` Jason Wang
2019-12-04 19:33   ` Peter Xu
2019-12-05  6:49     ` Jason Wang
2019-12-11 13:41 ` Christophe de Dinechin
2019-12-11 14:16   ` Paolo Bonzini
2019-12-11 17:15     ` Peter Xu

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191217052502.GF21868@joy-OptiPlex-7040 \
    --to=yan.y.zhao@intel.com \
    --cc=alex.williamson@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=peterx@redhat.com \
    --cc=sean.j.christopherson@intel.com \
    --cc=vkuznets@redhat.com \
    --cc=zhenyu.z.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
		kvm@vger.kernel.org
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.kvm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git