From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45A16C47097 for ; Fri, 4 Jun 2021 01:11:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 27314613F6 for ; Fri, 4 Jun 2021 01:11:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229854AbhFDBND (ORCPT ); Thu, 3 Jun 2021 21:13:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:56119 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229754AbhFDBNB (ORCPT ); Thu, 3 Jun 2021 21:13:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622769076; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74wa3D7052FGdwsOb19pvsgbuYGA5vLypJgGwvpg9kA=; b=a37wmN21NMiNqjs6CSHLYuHZjqb6wCLH+LZfaXhCht2TsQxTxs6l+n0xHrtrQ4F8j+lSAB eF0uY1mVHBer36oeFvnSDcASegOzFoJxD3FRjO7xf/IW7lZCPecPzXfzHCyeMcoUbBz68G M2URL10SkPoKdMHoX10Eh8hftny+ZJw= Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-239-wtnnjxZjOvONSl9khzuPxA-1; Thu, 03 Jun 2021 21:11:14 -0400 X-MC-Unique: wtnnjxZjOvONSl9khzuPxA-1 Received: by mail-pg1-f199.google.com with SMTP id k9-20020a63d1090000b029021091ebb84cso4971503pgg.3 for ; Thu, 03 Jun 2021 18:11:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=74wa3D7052FGdwsOb19pvsgbuYGA5vLypJgGwvpg9kA=; b=j6SvCf9u8v8dB31I3amrANPEnXILk8qCetfhs5ic672vROni5mIXVqAi9HmQ1Fu8fX A0629mHFPRK3WzfOS+mmR6722PFBxrYGHJq+9ymGXxkp56u4+6RgzxDQBPbuotMMlf8d dEJVG+DRiMiOhbUXmyMGARsfoyR3HYrN01pldIweXR96fH63555lbq/Z753Lm2DRIgTf GPESaoypkiL3UHB4COQHYhbhoxV7Hi67RaM/Tbc1x4quruVK9t65YmNK3nvTlumlGgMd qaIiA8ksQQ1ow+v2AYeq6RD1OmrzJ+QZJRiJ1fVB2hdvFyDu17QE5XCWEcCsd0vJ954g qiqw== X-Gm-Message-State: AOAM53077qM/E/Wedfx15Pe/s7miEjtTvMeFbi4748Sazf3b6abjfNkm f2B9+2A803XtBW7DJd7DQfTC3DMm6zMWbH8ohpVJuxfmA7L0GZbheq5yH9qD5IHgOqAPMexRj2a /cyFtooKxd0FHwKAHiHsN1WNF X-Received: by 2002:a63:370b:: with SMTP id e11mr2255877pga.356.1622769073851; Thu, 03 Jun 2021 18:11:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw19wf1A8+65Q36xarUBfhf8NPj5f2hsFRG2/Iexl5uV4W9R5ePiMPVn1Z2bNWIP1oHtD7E3w== X-Received: by 2002:a63:370b:: with SMTP id e11mr2255853pga.356.1622769073569; Thu, 03 Jun 2021 18:11:13 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id h18sm248362pgl.87.2021.06.03.18.11.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Jun 2021 18:11:13 -0700 (PDT) Subject: Re: [RFC] /dev/ioasid uAPI proposal To: Jason Gunthorpe Cc: Alex Williamson , "Tian, Kevin" , Jean-Philippe Brucker , "Jiang, Dave" , "Raj, Ashok" , "kvm@vger.kernel.org" , Jonathan Corbet , Robin Murphy , LKML , "iommu@lists.linux-foundation.org" , David Gibson , Kirti Wankhede , David Woodhouse References: <20210602160140.GV1002214@nvidia.com> <20210602111117.026d4a26.alex.williamson@redhat.com> <20210602173510.GE1002214@nvidia.com> <20210602120111.5e5bcf93.alex.williamson@redhat.com> <20210602180925.GH1002214@nvidia.com> <20210602130053.615db578.alex.williamson@redhat.com> <20210602195404.GI1002214@nvidia.com> <20210602143734.72fb4fa4.alex.williamson@redhat.com> <6a9426d7-ed55-e006-9c4c-6b7c78142e39@redhat.com> <20210603130927.GZ1002214@nvidia.com> From: Jason Wang Message-ID: <65614634-1db4-7119-1a90-64ba5c6e9042@redhat.com> Date: Fri, 4 Jun 2021 09:11:03 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <20210603130927.GZ1002214@nvidia.com> Content-Type: text/plain; charset=gbk; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org ÔÚ 2021/6/3 ÏÂÎç9:09, Jason Gunthorpe дµÀ: > On Thu, Jun 03, 2021 at 10:52:51AM +0800, Jason Wang wrote: > >> Basically, we don't want to bother with pseudo KVM device like what VFIO >> did. So for simplicity, we rules out the IOMMU that can't enforce coherency >> in vhost-vDPA if the parent purely depends on the platform IOMMU: > VDPA HW cannot issue no-snoop TLPs in the first place. Note that virtio/vDPA is not necessarily a PCI device. > > virtio does not define a protocol to discover such a functionality, Actually we had: VIRTIO_F_ACCESS_PLATFORM(33) This feature indicates that the device can be used on a platform where device access to data in memory is limited and/or translated. E.g. this is the case if the device can be located behind an IOMMU that translates bus addresses from the device into physical addresses in memory, if the device can be limited to only access certain memory addresses or if special commands such as a cache flush can be needed to synchronise data in memory with the device. > nor do any virtio drivers implement the required platform specific > cache flushing to make no-snoop TLPs work. I don't get why virtio drivers needs to do that. I think DMA API should hide those arch/platform specific stuffs from us. > > It is fundamentally part of the virtio HW PCI API that a device vendor > cannot alter. The spec doesn't forbid this, and it just leave the detection and action to the driver in a platform specific way. Thanks > > Basically since we already know that the virtio kernel drivers do not > call the cache flush instruction we don't need the weird KVM logic to > turn it on at all. > > Enforcing no-snoop at the IOMMU here is redundant/confusing. > > Jason > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93B63C47099 for ; Fri, 4 Jun 2021 01:11:21 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B1516140D for ; Fri, 4 Jun 2021 01:11:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B1516140D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id F154A84036; Fri, 4 Jun 2021 01:11:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tIHg2DCVMLCT; Fri, 4 Jun 2021 01:11:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTP id CEEB783DEB; Fri, 4 Jun 2021 01:11:19 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B2369C000E; Fri, 4 Jun 2021 01:11:19 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 47E25C0001 for ; Fri, 4 Jun 2021 01:11:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3BF536F931 for ; Fri, 4 Jun 2021 01:11:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vuBWdG39m9DB for ; Fri, 4 Jun 2021 01:11:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id E14FB60AF2 for ; Fri, 4 Jun 2021 01:11:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622769076; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74wa3D7052FGdwsOb19pvsgbuYGA5vLypJgGwvpg9kA=; b=a37wmN21NMiNqjs6CSHLYuHZjqb6wCLH+LZfaXhCht2TsQxTxs6l+n0xHrtrQ4F8j+lSAB eF0uY1mVHBer36oeFvnSDcASegOzFoJxD3FRjO7xf/IW7lZCPecPzXfzHCyeMcoUbBz68G M2URL10SkPoKdMHoX10Eh8hftny+ZJw= Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-561-CiP0Pt11NaSj2spCZ61Icg-1; Thu, 03 Jun 2021 21:11:14 -0400 X-MC-Unique: CiP0Pt11NaSj2spCZ61Icg-1 Received: by mail-pl1-f199.google.com with SMTP id u1-20020a1709027141b02901040cac08fdso3387082plm.12 for ; Thu, 03 Jun 2021 18:11:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=74wa3D7052FGdwsOb19pvsgbuYGA5vLypJgGwvpg9kA=; b=ChbN5QTZ052+mBXRHR8nL9gWHPhRQBIV4NdImQV9y1pTNnAn/jL3wW66YiT/TmYa4w 6J8F+nncMo655cGl1ejifdHje/Mi20CXPpif2O+kE2Ivvamnn3kuqBzpgtTd9cCG0X/D NGm8bgrw/VE7j2MzE0U0awqpgFClQuV7azw0V/KsvxFmR2nQDP7J452sdLyMDuDsS/wY TOLWOf517uaBQ7w1+I42JMEptYLh5VBAIWLT88vfMYjcrjZnE7Jt0zT4ThtvRwIrClax oO7xrgSqfssdnC1ge69lUOAZcz3tfrt0bPyi4tjg7ZJO1TP6euYvLnsXzcRJkEpIBmZB 5X5g== X-Gm-Message-State: AOAM531DEvr6jmOLa776RkewOMxj2LCm7KygKCCAlJQVgzypg5gw0zqv Kuxoz6IbLAXgT4EVQ5MiW4KK7iZSd2TJPPKaChTSSyh6gVzKcDQRLeL21QjQ2nJnCY80qQDUedK w3i8elhDnLzzBM3BAEHfp1cGg3xzaGQ== X-Received: by 2002:a63:370b:: with SMTP id e11mr2255873pga.356.1622769073849; Thu, 03 Jun 2021 18:11:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw19wf1A8+65Q36xarUBfhf8NPj5f2hsFRG2/Iexl5uV4W9R5ePiMPVn1Z2bNWIP1oHtD7E3w== X-Received: by 2002:a63:370b:: with SMTP id e11mr2255853pga.356.1622769073569; Thu, 03 Jun 2021 18:11:13 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id h18sm248362pgl.87.2021.06.03.18.11.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Jun 2021 18:11:13 -0700 (PDT) Subject: Re: [RFC] /dev/ioasid uAPI proposal To: Jason Gunthorpe References: <20210602160140.GV1002214@nvidia.com> <20210602111117.026d4a26.alex.williamson@redhat.com> <20210602173510.GE1002214@nvidia.com> <20210602120111.5e5bcf93.alex.williamson@redhat.com> <20210602180925.GH1002214@nvidia.com> <20210602130053.615db578.alex.williamson@redhat.com> <20210602195404.GI1002214@nvidia.com> <20210602143734.72fb4fa4.alex.williamson@redhat.com> <6a9426d7-ed55-e006-9c4c-6b7c78142e39@redhat.com> <20210603130927.GZ1002214@nvidia.com> From: Jason Wang Message-ID: <65614634-1db4-7119-1a90-64ba5c6e9042@redhat.com> Date: Fri, 4 Jun 2021 09:11:03 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.10.2 MIME-Version: 1.0 In-Reply-To: <20210603130927.GZ1002214@nvidia.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Jean-Philippe Brucker , "Tian, Kevin" , "Jiang, Dave" , "Raj, Ashok" , "kvm@vger.kernel.org" , Jonathan Corbet , David Woodhouse , LKML , "iommu@lists.linux-foundation.org" , Alex Williamson , Kirti Wankhede , Robin Murphy , David Gibson X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="gbk"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" CtTaIDIwMjEvNi8zIM/Czuc5OjA5LCBKYXNvbiBHdW50aG9ycGUg0LS1wDoKPiBPbiBUaHUsIEp1 biAwMywgMjAyMSBhdCAxMDo1Mjo1MUFNICswODAwLCBKYXNvbiBXYW5nIHdyb3RlOgo+Cj4+IEJh c2ljYWxseSwgd2UgZG9uJ3Qgd2FudCB0byBib3RoZXIgd2l0aCBwc2V1ZG8gS1ZNIGRldmljZSBs aWtlIHdoYXQgVkZJTwo+PiBkaWQuIFNvIGZvciBzaW1wbGljaXR5LCB3ZSBydWxlcyBvdXQgdGhl IElPTU1VIHRoYXQgY2FuJ3QgZW5mb3JjZSBjb2hlcmVuY3kKPj4gaW4gdmhvc3QtdkRQQSBpZiB0 aGUgcGFyZW50IHB1cmVseSBkZXBlbmRzIG9uIHRoZSBwbGF0Zm9ybSBJT01NVToKPiBWRFBBIEhX IGNhbm5vdCBpc3N1ZSBuby1zbm9vcCBUTFBzIGluIHRoZSBmaXJzdCBwbGFjZS4KCgpOb3RlIHRo YXQgdmlydGlvL3ZEUEEgaXMgbm90IG5lY2Vzc2FyaWx5IGEgUENJIGRldmljZS4KCgo+Cj4gdmly dGlvIGRvZXMgbm90IGRlZmluZSBhIHByb3RvY29sIHRvIGRpc2NvdmVyIHN1Y2ggYSBmdW5jdGlv bmFsaXR5LAoKCkFjdHVhbGx5IHdlIGhhZDoKClZJUlRJT19GX0FDQ0VTU19QTEFURk9STSgzMykK VGhpcyBmZWF0dXJlIGluZGljYXRlcyB0aGF0IHRoZSBkZXZpY2UgY2FuIGJlIHVzZWQgb24gYSBw bGF0Zm9ybSB3aGVyZSAKZGV2aWNlIGFjY2VzcyB0byBkYXRhIGluIG1lbW9yeSBpcyBsaW1pdGVk IGFuZC9vciB0cmFuc2xhdGVkLiBFLmcuIHRoaXMgCmlzIHRoZSBjYXNlIGlmIHRoZSBkZXZpY2Ug Y2FuIGJlIGxvY2F0ZWQgYmVoaW5kIGFuIElPTU1VIHRoYXQgdHJhbnNsYXRlcyAKYnVzIGFkZHJl c3NlcyBmcm9tIHRoZSBkZXZpY2UgaW50byBwaHlzaWNhbCBhZGRyZXNzZXMgaW4gbWVtb3J5LCBp ZiB0aGUgCmRldmljZSBjYW4gYmUgbGltaXRlZCB0byBvbmx5IGFjY2VzcyBjZXJ0YWluIG1lbW9y eSBhZGRyZXNzZXMgb3IgaWYgCnNwZWNpYWwgY29tbWFuZHMgc3VjaCBhcyBhIGNhY2hlIGZsdXNo IGNhbiBiZSBuZWVkZWQgdG8gc3luY2hyb25pc2UgZGF0YSAKaW4gbWVtb3J5IHdpdGggdGhlIGRl dmljZS4KCgo+IG5vciBkbyBhbnkgdmlydGlvIGRyaXZlcnMgaW1wbGVtZW50IHRoZSByZXF1aXJl ZCBwbGF0Zm9ybSBzcGVjaWZpYwo+IGNhY2hlIGZsdXNoaW5nIHRvIG1ha2Ugbm8tc25vb3AgVExQ cyB3b3JrLgoKCkkgZG9uJ3QgZ2V0IHdoeSB2aXJ0aW8gZHJpdmVycyBuZWVkcyB0byBkbyB0aGF0 LiBJIHRoaW5rIERNQSBBUEkgc2hvdWxkIApoaWRlIHRob3NlIGFyY2gvcGxhdGZvcm0gc3BlY2lm aWMgc3R1ZmZzIGZyb20gdXMuCgoKPgo+IEl0IGlzIGZ1bmRhbWVudGFsbHkgcGFydCBvZiB0aGUg dmlydGlvIEhXIFBDSSBBUEkgdGhhdCBhIGRldmljZSB2ZW5kb3IKPiBjYW5ub3QgYWx0ZXIuCgoK VGhlIHNwZWMgZG9lc24ndCBmb3JiaWQgdGhpcywgYW5kIGl0IGp1c3QgbGVhdmUgdGhlIGRldGVj dGlvbiBhbmQgYWN0aW9uIAp0byB0aGUgZHJpdmVyIGluIGEgcGxhdGZvcm0gc3BlY2lmaWMgd2F5 LgoKVGhhbmtzCgoKPgo+IEJhc2ljYWxseSBzaW5jZSB3ZSBhbHJlYWR5IGtub3cgdGhhdCB0aGUg dmlydGlvIGtlcm5lbCBkcml2ZXJzIGRvIG5vdAo+IGNhbGwgdGhlIGNhY2hlIGZsdXNoIGluc3Ry dWN0aW9uIHdlIGRvbid0IG5lZWQgdGhlIHdlaXJkIEtWTSBsb2dpYyB0bwo+IHR1cm4gaXQgb24g YXQgYWxsLgo+Cj4gRW5mb3JjaW5nIG5vLXNub29wIGF0IHRoZSBJT01NVSBoZXJlIGlzIHJlZHVu ZGFudC9jb25mdXNpbmcuCj4KPiBKYXNvbgo+CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fXwppb21tdSBtYWlsaW5nIGxpc3QKaW9tbXVAbGlzdHMubGludXgt Zm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4v bGlzdGluZm8vaW9tbXU=