KVM Archive on lore.kernel.org
 help / color / Atom feed
From: David Hildenbrand <david@redhat.com>
To: Christian Borntraeger <borntraeger@de.ibm.com>,
	Janosch Frank <frankja@linux.vnet.ibm.com>
Cc: KVM <kvm@vger.kernel.org>, Cornelia Huck <cohuck@redhat.com>,
	Thomas Huth <thuth@redhat.com>,
	Ulrich Weigand <Ulrich.Weigand@de.ibm.com>,
	Claudio Imbrenda <imbrenda@linux.ibm.com>,
	linux-s390 <linux-s390@vger.kernel.org>,
	Michael Mueller <mimu@linux.ibm.com>,
	Vasily Gorbik <gor@linux.ibm.com>
Subject: Re: [PATCH v2 02/42] KVM: s390/interrupt: do not pin adapter interrupt pages
Date: Mon, 17 Feb 2020 10:43:48 +0100
Message-ID: <073d3666-480e-5ba5-a46b-4cbd615f4174@redhat.com> (raw)
In-Reply-To: <20200214222658.12946-3-borntraeger@de.ibm.com>

On 14.02.20 23:26, Christian Borntraeger wrote:
> From: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
> 
> The adapter interrupt page containing the indicator bits is currently
> pinned. That means that a guest with many devices can pin a lot of
> memory pages in the host. This also complicates the reference tracking
> which is needed for memory management handling of protected virtual
> machines. It might also have some strange side effects for madvise
> MADV_DONTNEED and other things.
> 
> We can simply try to get the userspace page set the bits and free the
> page. By storing the userspace address in the irq routing entry instead
> of the guest address we can actually avoid many lookups and list walks
> so that this variant is very likely not slower.
> 
> If userspace messes around with the memory slots the worst thing that
> can happen is that we write to some other memory within that process.
> As we get the the page with FOLL_WRITE this can also not be used to
> write to shared read-only pages.
> 
> Signed-off-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
> [borntraeger@de.ibm.com: patch simplification]
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  Documentation/virt/kvm/devices/s390_flic.rst |  11 +-
>  arch/s390/include/asm/kvm_host.h             |   3 -
>  arch/s390/kvm/interrupt.c                    | 170 ++++++-------------
>  3 files changed, 53 insertions(+), 131 deletions(-)
> 
> diff --git a/Documentation/virt/kvm/devices/s390_flic.rst b/Documentation/virt/kvm/devices/s390_flic.rst
> index 954190da7d04..ea96559ba501 100644
> --- a/Documentation/virt/kvm/devices/s390_flic.rst
> +++ b/Documentation/virt/kvm/devices/s390_flic.rst
> @@ -108,16 +108,9 @@ Groups:
>        mask or unmask the adapter, as specified in mask
>  
>      KVM_S390_IO_ADAPTER_MAP
> -      perform a gmap translation for the guest address provided in addr,
> -      pin a userspace page for the translated address and add it to the
> -      list of mappings
> -
> -      .. note:: A new mapping will be created unconditionally; therefore,
> -	        the calling code should avoid making duplicate mappings.
> -
> +      This is now a no-op. The mapping is purely done by the irq route.
>      KVM_S390_IO_ADAPTER_UNMAP
> -      release a userspace page for the translated address specified in addr
> -      from the list of mappings
> +      This is now a no-op. The mapping is purely done by the irq route.
>  

The interface should have accepted a hva from the very start and not
guest addresses ...

[...]

>  
>  static int modify_io_adapter(struct kvm_device *dev,
> @@ -2456,12 +2378,13 @@ static int modify_io_adapter(struct kvm_device *dev,
>  		if (ret > 0)
>  			ret = 0;
>  		break;
> +	/*
> +	 * We resolve the gpa to hva when setting the IRQ routing. the set_irq
> +	 * code uses get_user_pages_remote to do the actual write.

nit: "get_user_pages_remote()"

> +	 */
>  	case KVM_S390_IO_ADAPTER_MAP:
> -		ret = kvm_s390_adapter_map(dev->kvm, req.id, req.addr);
> -		break;
>  	case KVM_S390_IO_ADAPTER_UNMAP:
> -		ret = kvm_s390_adapter_unmap(dev->kvm, req.id, req.addr);
> -		break;
> +		return 0;
>  	default:
>  		ret = -EINVAL;
>  	}
> @@ -2699,19 +2622,21 @@ static unsigned long get_ind_bit(__u64 addr, unsigned long bit_nr, bool swap)
>  	return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit;
>  }
>  
> -static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter,
> -					  u64 addr)
> +static struct page *get_map_page(struct kvm *kvm,
> +				 struct s390_io_adapter *adapter,
> +				 u64 uaddr)
>  {
> -	struct s390_map_info *map;
> +	struct page *page = NULL;
>  
>  	if (!adapter)
>  		return NULL;

AFAIKs, this check is not necessary.

> -
> -	list_for_each_entry(map, &adapter->maps, list) {
> -		if (map->guest_addr == addr)
> -			return map;
> -	}
> -	return NULL;
> +	if (!uaddr)
> +		return NULL;

I do wonder if that check is necessary. I don't think so but might be
missing something.

> +	down_read(&kvm->mm->mmap_sem);
> +	get_user_pages_remote(NULL, kvm->mm, uaddr, 1, FOLL_WRITE,
> +			      &page, NULL, NULL);
> +	up_read(&kvm->mm->mmap_sem);
> +	return page;
>  }
>  
>  static int adapter_indicators_set(struct kvm *kvm,
> @@ -2720,30 +2645,35 @@ static int adapter_indicators_set(struct kvm *kvm,
>  {
>  	unsigned long bit;
>  	int summary_set, idx;
> -	struct s390_map_info *info;
> +	struct page *ind_page, *summary_page;
>  	void *map;
>  
> -	info = get_map_info(adapter, adapter_int->ind_addr);
> -	if (!info)
> +	ind_page = get_map_page(kvm, adapter, adapter_int->ind_addr);
> +	if (!ind_page)
>  		return -1;
> -	map = page_address(info->page);
> -	bit = get_ind_bit(info->addr, adapter_int->ind_offset, adapter->swap);
> -	set_bit(bit, map);
> -	idx = srcu_read_lock(&kvm->srcu);
> -	mark_page_dirty(kvm, info->guest_addr >> PAGE_SHIFT);
> -	set_page_dirty_lock(info->page);
> -	info = get_map_info(adapter, adapter_int->summary_addr);
> -	if (!info) {
> -		srcu_read_unlock(&kvm->srcu, idx);
> +	summary_page = get_map_page(kvm, adapter, adapter_int->summary_addr);
> +	if (!summary_page) {
> +		put_page(ind_page);
>  		return -1;
>  	}
> -	map = page_address(info->page);
> -	bit = get_ind_bit(info->addr, adapter_int->summary_offset,
> -			  adapter->swap);
> +
> +	idx = srcu_read_lock(&kvm->srcu);
> +	map = page_address(ind_page);
> +	bit = get_ind_bit(adapter_int->ind_addr,
> +			  adapter_int->ind_offset, adapter->swap);
> +	set_bit(bit, map);
> +	mark_page_dirty(kvm, adapter_int->ind_addr >> PAGE_SHIFT);
> +	set_page_dirty_lock(ind_page);
> +	map = page_address(summary_page);
> +	bit = get_ind_bit(adapter_int->summary_addr,
> +			  adapter_int->summary_offset, adapter->swap);
>  	summary_set = test_and_set_bit(bit, map);
> -	mark_page_dirty(kvm, info->guest_addr >> PAGE_SHIFT);
> -	set_page_dirty_lock(info->page);
> +	mark_page_dirty(kvm, adapter_int->summary_addr >> PAGE_SHIFT);
> +	set_page_dirty_lock(summary_page);
>  	srcu_read_unlock(&kvm->srcu, idx);
> +
> +	put_page(ind_page);
> +	put_page(summary_page);
>  	return summary_set ? 0 : 1;
>  }
>  
> @@ -2765,9 +2695,7 @@ static int set_adapter_int(struct kvm_kernel_irq_routing_entry *e,
>  	adapter = get_io_adapter(kvm, e->adapter.adapter_id);
>  	if (!adapter)
>  		return -1;
> -	down_read(&adapter->maps_lock);
>  	ret = adapter_indicators_set(kvm, adapter, &e->adapter);
> -	up_read(&adapter->maps_lock);
>  	if ((ret > 0) && !adapter->masked) {
>  		ret = kvm_s390_inject_airq(kvm, adapter);
>  		if (ret == 0)
> @@ -2818,23 +2746,27 @@ int kvm_set_routing_entry(struct kvm *kvm,
>  			  struct kvm_kernel_irq_routing_entry *e,
>  			  const struct kvm_irq_routing_entry *ue)
>  {
> -	int ret;
> +	u64 uaddr;
>  
>  	switch (ue->type) {
> +	/* we store the userspace addresses instead of the guest addresses */
>  	case KVM_IRQ_ROUTING_S390_ADAPTER:
>  		e->set = set_adapter_int;
> -		e->adapter.summary_addr = ue->u.adapter.summary_addr;
> -		e->adapter.ind_addr = ue->u.adapter.ind_addr;
> +		uaddr =  gmap_translate(kvm->arch.gmap, ue->u.adapter.summary_addr);
> +		if (uaddr == -EFAULT)
> +			return -EFAULT;
> +		e->adapter.summary_addr = uaddr;
> +		uaddr =  gmap_translate(kvm->arch.gmap, ue->u.adapter.ind_addr);
> +		if (uaddr == -EFAULT)
> +			return -EFAULT;

AFAIK, leaving e->adapter.summary_addr set is not an issue.

Interesting, in kvm_s390_adapter_map(), we didn't synchronize again slot
updates when doing the gmap_translate(), which looks wrong to me ...

It seems to be the same thing here. I do wonder if it is safe to do a
gmap_translate() here, looks like this can race with
kvm_arch_commit_memory_region().

I would have assumed we need e.g., the slots_lock while doing the
gmap_translate() - or a srcu_read_lock(&vcpu->kvm->srcu) or similar ...


Apart from that, looks good to me.

-- 
Thanks,

David / dhildenb


  reply index

Thread overview: 132+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-14 22:26 [PATCH v2 00/42] KVM: s390: Add support for protected VMs Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 01/42] mm:gup/writeback: add callbacks for inaccessible pages Christian Borntraeger
2020-02-17  9:14   ` David Hildenbrand
2020-02-17 11:10     ` Christian Borntraeger
2020-02-18  8:27       ` David Hildenbrand
2020-02-18 15:46         ` Sean Christopherson
2020-02-18 16:02           ` Will Deacon
2020-02-18 16:15             ` Christian Borntraeger
2020-02-18 21:35               ` Sean Christopherson
2020-02-19  8:31                 ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 02/42] KVM: s390/interrupt: do not pin adapter interrupt pages Christian Borntraeger
2020-02-17  9:43   ` David Hildenbrand [this message]
2020-02-20 12:18     ` David Hildenbrand
2020-02-20 13:31     ` Christian Borntraeger
2020-02-20 13:34       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 03/42] s390/protvirt: introduce host side setup Christian Borntraeger
2020-02-17  9:53   ` David Hildenbrand
2020-02-17 11:11     ` Christian Borntraeger
2020-02-17 11:13       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 04/42] s390/protvirt: add ultravisor initialization Christian Borntraeger
2020-02-17  9:57   ` David Hildenbrand
2020-02-17 11:13     ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 05/42] s390/mm: provide memory management functions for protected KVM guests Christian Borntraeger
2020-02-17 10:21   ` David Hildenbrand
2020-02-17 11:28     ` Christian Borntraeger
2020-02-17 12:07       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 06/42] s390/mm: add (non)secure page access exceptions handlers Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 07/42] KVM: s390: protvirt: Add UV debug trace Christian Borntraeger
2020-02-17 10:41   ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 08/42] KVM: s390: add new variants of UV CALL Christian Borntraeger
2020-02-17 10:42   ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 09/42] KVM: s390: protvirt: Add initial vm and cpu lifecycle handling Christian Borntraeger
2020-02-17 10:56   ` David Hildenbrand
2020-02-17 12:04     ` Christian Borntraeger
2020-02-17 12:09       ` David Hildenbrand
2020-02-17 14:53         ` [PATCH 0/2] example changes Christian Borntraeger
2020-02-17 14:53           ` [PATCH 1/2] lock changes Christian Borntraeger
2020-02-17 14:53           ` [PATCH 2/2] merge vm/cpu create Christian Borntraeger
2020-02-17 15:00             ` Janosch Frank
2020-02-17 15:02               ` Christian Borntraeger
2020-02-19 11:02               ` Christian Borntraeger
2020-02-17 19:18           ` [PATCH 0/2] example changes David Hildenbrand
2020-02-18  8:09     ` [PATCH v2 09/42] KVM: s390: protvirt: Add initial vm and cpu lifecycle handling Christian Borntraeger
2020-02-18  8:39   ` [PATCH v2.1] " Christian Borntraeger
2020-02-18  9:12     ` David Hildenbrand
2020-02-18 21:18       ` Christian Borntraeger
2020-02-19  8:32         ` David Hildenbrand
2020-02-19 11:01       ` Christian Borntraeger
2020-02-18  9:56     ` David Hildenbrand
2020-02-18 20:26       ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 10/42] KVM: s390: protvirt: Add KVM api documentation Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 11/42] KVM: s390: protvirt: Secure memory is not mergeable Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 12/42] KVM: s390/mm: Make pages accessible before destroying the guest Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 13/42] KVM: s390: protvirt: Handle SE notification interceptions Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 14/42] KVM: s390: protvirt: Instruction emulation Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 15/42] KVM: s390: protvirt: Add interruption injection controls Christian Borntraeger
2020-02-17 10:59   ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 16/42] KVM: s390: protvirt: Implement interruption injection Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 17/42] KVM: s390: protvirt: Add SCLP interrupt handling Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 18/42] KVM: s390: protvirt: Handle spec exception loops Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 19/42] KVM: s390: protvirt: Add new gprs location handling Christian Borntraeger
2020-02-17 11:01   ` David Hildenbrand
2020-02-17 11:33     ` Christian Borntraeger
2020-02-17 14:37     ` Janosch Frank
2020-02-14 22:26 ` [PATCH v2 20/42] KVM: S390: protvirt: Introduce instruction data area bounce buffer Christian Borntraeger
2020-02-17 11:08   ` David Hildenbrand
2020-02-17 14:47     ` Janosch Frank
2020-02-17 15:00       ` Christian Borntraeger
2020-02-17 15:38         ` Janosch Frank
2020-02-17 16:58           ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 21/42] KVM: s390: protvirt: handle secure guest prefix pages Christian Borntraeger
2020-02-17 11:11   ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 22/42] KVM: s390/mm: handle guest unpin events Christian Borntraeger
2020-02-17 14:23   ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 23/42] KVM: s390: protvirt: Write sthyi data to instruction data area Christian Borntraeger
2020-02-17 14:24   ` David Hildenbrand
2020-02-17 18:40     ` Christian Borntraeger
2020-02-17 19:16       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 24/42] KVM: s390: protvirt: STSI handling Christian Borntraeger
2020-02-18  8:35   ` David Hildenbrand
2020-02-18  8:44     ` Christian Borntraeger
2020-02-18  9:08       ` David Hildenbrand
2020-02-18  9:11         ` Christian Borntraeger
2020-02-18  9:13           ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 25/42] KVM: s390: protvirt: disallow one_reg Christian Borntraeger
2020-02-18  8:40   ` David Hildenbrand
2020-02-18  8:57     ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 26/42] KVM: s390: protvirt: Do only reset registers that are accessible Christian Borntraeger
2020-02-18  8:42   ` David Hildenbrand
2020-02-18  9:20     ` Christian Borntraeger
2020-02-18  9:28       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 27/42] KVM: s390: protvirt: Only sync fmt4 registers Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 28/42] KVM: s390: protvirt: Add program exception injection Christian Borntraeger
2020-02-18  9:33   ` David Hildenbrand
2020-02-18  9:37     ` Christian Borntraeger
2020-02-18  9:39       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 29/42] KVM: s390: protvirt: Add diag 308 subcode 8 - 10 handling Christian Borntraeger
2020-02-18  9:38   ` David Hildenbrand
2020-02-19 12:45     ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 30/42] KVM: s390: protvirt: UV calls in support of diag308 0, 1 Christian Borntraeger
2020-02-18  9:44   ` David Hildenbrand
2020-02-19 11:53     ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 31/42] KVM: s390: protvirt: Report CPU state to Ultravisor Christian Borntraeger
2020-02-18  9:48   ` David Hildenbrand
2020-02-19 19:36     ` Christian Borntraeger
2020-02-19 19:46       ` Christian Borntraeger
2020-02-20 10:52         ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 32/42] KVM: s390: protvirt: Support cmd 5 operation state Christian Borntraeger
2020-02-18  9:50   ` David Hildenbrand
2020-02-19 11:06     ` Christian Borntraeger
2020-02-19 11:08       ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 33/42] KVM: s390: protvirt: Mask PSW interrupt bits for interception 104 and 112 Christian Borntraeger
2020-02-18  9:53   ` David Hildenbrand
2020-02-18 10:02     ` David Hildenbrand
2020-02-18 10:05     ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 34/42] KVM: s390: protvirt: do not inject interrupts after start Christian Borntraeger
2020-02-18  9:53   ` David Hildenbrand
2020-02-18 10:02     ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 35/42] KVM: s390: protvirt: Add UV cpu reset calls Christian Borntraeger
2020-02-18  9:54   ` David Hildenbrand
2020-02-14 22:26 ` [PATCH v2 36/42] DOCUMENTATION: Protected virtual machine introduction and IPL Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 37/42] s390/uv: Fix handling of length extensions (already in s390 tree) Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 38/42] s390: protvirt: Add sysfs firmware interface for Ultravisor information Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 39/42] example for future extension: mm:gup/writeback: add callbacks for inaccessible pages: error cases Christian Borntraeger
2020-02-18 16:25   ` Will Deacon
2020-02-18 16:30     ` Christian Borntraeger
2020-02-18 16:33       ` Will Deacon
2020-02-14 22:26 ` [PATCH v2 40/42] example for future extension: mm:gup/writeback: add callbacks for inaccessible pages: source indication Christian Borntraeger
2020-02-17 14:15   ` Ulrich Weigand
2020-02-17 14:38     ` Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 41/42] potential fixup for "s390/mm: provide memory management functions for protected KVM guests" Christian Borntraeger
2020-02-14 22:26 ` [PATCH v2 42/42] KVM: s390: rstify new ioctls in api.rst Christian Borntraeger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=073d3666-480e-5ba5-a46b-4cbd615f4174@redhat.com \
    --to=david@redhat.com \
    --cc=Ulrich.Weigand@de.ibm.com \
    --cc=borntraeger@de.ibm.com \
    --cc=cohuck@redhat.com \
    --cc=frankja@linux.vnet.ibm.com \
    --cc=gor@linux.ibm.com \
    --cc=imbrenda@linux.ibm.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=mimu@linux.ibm.com \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

KVM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/kvm/0 kvm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 kvm kvm/ https://lore.kernel.org/kvm \
		kvm@vger.kernel.org
	public-inbox-index kvm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.kvm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git