From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751477AbcFNKlC (ORCPT ); Tue, 14 Jun 2016 06:41:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43011 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750806AbcFNKk7 (ORCPT ); Tue, 14 Jun 2016 06:40:59 -0400 Subject: Re: [RFC PATCH V3 0/3] basic device IOTLB support To: mst@redhat.com, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org References: <1464082585-13049-1-git-send-email-jasowang@redhat.com> Cc: wexu@redhat.com, peterx@redhat.com, vkaplans@redhat.com From: Jason Wang Message-ID: <575FDF35.40201@redhat.com> Date: Tue, 14 Jun 2016 18:40:53 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 MIME-Version: 1.0 In-Reply-To: <1464082585-13049-1-git-send-email-jasowang@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 14 Jun 2016 10:40:59 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2016年05月24日 17:36, Jason Wang wrote: > This patch tries to implement an device IOTLB for vhost. This could be > used with for co-operation with userspace IOMMU implementation (qemu) > for a secure DMA environment (DMAR) in guest. > > The idea is simple. When vhost meets an IOTLB miss, it will request > the assistance of userspace to do the translation, this is done > through: > > - when there's a IOTLB miss, it will notify userspace through > vhost_net fd and then userspace read the fault address, size and > access from vhost fd. > - userspace write the translation result back to vhost fd, vhost can > then update its IOTLB. > > The codes were optimized for fixed mapping users e.g dpdk in guest. It > will be slow if dynamic mappings were used in guest. We could do > optimizations on top. > > The codes were designed to be architecture independent. It should be > easily ported to any architecture. > > Stress tested with l2fwd/vfio in guest with 4K/2M/1G page size. On 1G > hugepage case, 100% TLB hit rate were noticed. > > Changes from V2: > - introduce memory accessors for vhost > - switch from ioctls to oridinary file read/write for iotlb miss and > updating > - do not assume virtqueue were virtually mapped contiguously, all > virtqueue access were done throug IOTLB > - verify memory access during IOTLB update and fail early > - introduce a module parameter for the size of IOTLB > > Changes from V1: > - support any size/range of updating and invalidation through > introducing the interval tree. > - convert from per device iotlb request to per virtqueue iotlb > request, this solves the possible deadlock in V1. > - read/write permission check support. > > Please review. Have a benchmark on this. Test was done with l2fwd in guest. For 2MB page, no difference in 64B performance and I notice a 4%-5% drop for 1500B performance compare to UIO in guest. We can add some shortcut to bypass the IOTLB for virtqueue accessing, but I think it's better to be done on top. > > Jason Wang (3): > vhost: introduce vhost memory accessors > vhost: convert pre sorted vhost memory array to interval tree > vhost: device IOTLB API > > drivers/vhost/net.c | 63 +++- > drivers/vhost/vhost.c | 760 ++++++++++++++++++++++++++++++++++++++------- > drivers/vhost/vhost.h | 60 +++- > include/uapi/linux/vhost.h | 28 ++ > 4 files changed, 790 insertions(+), 121 deletions(-) >