From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5456C433F5 for ; Fri, 22 Oct 2021 07:49:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B52D360EB8 for ; Fri, 22 Oct 2021 07:49:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232142AbhJVHvq (ORCPT ); Fri, 22 Oct 2021 03:51:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231872AbhJVHvp (ORCPT ); Fri, 22 Oct 2021 03:51:45 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 180CDC061764 for ; Fri, 22 Oct 2021 00:49:28 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id d13so3335644wrf.11 for ; Fri, 22 Oct 2021 00:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=l4ig/cSrg8mQN6aKAdfHWEATu5gWqjzgX7zon10Q9+E=; b=XNhjtbDMlD0fLOkQBRfHMcymLt/KXEWZO0gzlFJJkY7tBULpUdJRS/3zpdmfMBLV4H mDyJzKhY7XbOJxQruVq5kcUcL2jZDv1gnvQ9jlhLZbg5KSBTSxZeZvrngodKSZNSEl8T Pf8RChum9PCyRSCLuC4ch1v42ocyebTEN4wJh8w08Ks8CqkjNEC8HhO8Cuuz4OWdpkPv Ndv3qIL1U/ZUkmavejbqkHMkEQvYMfDd9YjvNpLjTz2JkhjTIni3XGjzgxXkPKZKg+Qy kQmnSmqvnDz0Wh6heFLPsNne7vzaPPNcev9WX9RnAprj7p04Nm7y9wLDcgRl/bor6Ngj L0nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=l4ig/cSrg8mQN6aKAdfHWEATu5gWqjzgX7zon10Q9+E=; b=xSTitikSipkoXdR/QBJEIbZoofM+HkPKKIAKnrleQzQ/sWDCF72O/BEKDuWvd56ZnD bIQ61X+yl+q2vVg4xNJaEwNCZ8W3vXe9dKl4QRcnI49R4qNDJUh/kTZV9gBEXDxb8y6k O21wReD6YVgVyn+kViUH2ptMwj+MgpWnEtpc4ZIcGk54GFygMnvpg8vDHwIb8Jk9vb6J GgapaZ8NxHGdUk0FIEHNBhqkjL/GG2glVLKpsmqt9IlTjaEg3d3rTUid2/8xckc2yicO +rBXMt/d8oBxOAvrIPrkAaY+fug4M58FKPlo7pJX57oyyEPZMKwFtkeSaFKSqOviIPz5 MlAA== X-Gm-Message-State: AOAM533x9sA+BZeFBgNn3I/imOolXq9iGXwL86m2bMtDxvwr+kr7/1vw L81bHaBUzITVbMJy8r+f1/7YJQ== X-Google-Smtp-Source: ABdhPJxpO/NacPoc1tCOWgCvmhl6vzuBziA4+8Ev+IfkXnOna1GvDwP4ExJS99BeJ3odnzDfvvbRCQ== X-Received: by 2002:a5d:6d81:: with SMTP id l1mr13601357wrs.110.1634888966529; Fri, 22 Oct 2021 00:49:26 -0700 (PDT) Received: from myrica (cpc92880-cmbg19-2-0-cust679.5-4.cable.virginm.net. [82.27.106.168]) by smtp.gmail.com with ESMTPSA id l2sm8802620wrs.90.2021.10.22.00.49.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Oct 2021 00:49:25 -0700 (PDT) Date: Fri, 22 Oct 2021 08:49:03 +0100 From: Jean-Philippe Brucker To: Jason Gunthorpe Cc: "Tian, Kevin" , Alex Williamson , "Liu, Yi L" , "hch@lst.de" , "jasowang@redhat.com" , "joro@8bytes.org" , "parav@mellanox.com" , "lkml@metux.net" , "pbonzini@redhat.com" , "lushenming@huawei.com" , "eric.auger@redhat.com" , "corbet@lwn.net" , "Raj, Ashok" , "yi.l.liu@linux.intel.com" , "Tian, Jun J" , "Wu, Hao" , "Jiang, Dave" , "jacob.jun.pan@linux.intel.com" , "kwankhede@nvidia.com" , "robin.murphy@arm.com" , "kvm@vger.kernel.org" , "iommu@lists.linux-foundation.org" , "dwmw2@infradead.org" , "linux-kernel@vger.kernel.org" , "baolu.lu@linux.intel.com" , "david@gibson.dropbear.id.au" , "nicolinc@nvidia.com" Subject: Re: [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO Message-ID: References: <20210923114219.GG964074@nvidia.com> <20210930222355.GH964074@nvidia.com> <20211014154259.GT2744544@nvidia.com> <20211021232223.GM2744544@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20211021232223.GM2744544@nvidia.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Oct 21, 2021 at 08:22:23PM -0300, Jason Gunthorpe wrote: > On Thu, Oct 21, 2021 at 03:58:02PM +0100, Jean-Philippe Brucker wrote: > > On Thu, Oct 21, 2021 at 02:26:00AM +0000, Tian, Kevin wrote: > > > > I'll leave it to Jean to confirm. If only coherent DMA can be used in > > > > the guest on other platforms, suppose VFIO should not blindly set > > > > IOMMU_CACHE and in concept it should deny assigning a non-coherent > > > > device since no co-ordination with guest exists today. > > > > > > Jean, what's your opinion? > > > > Yes a sanity check to prevent assigning non-coherent devices would be > > good, though I'm not particularly worried about non-coherent devices. PCIe > > on Arm should be coherent (according to the Base System Architecture). So > > vfio-pci devices should be coherent, but vfio-platform and mdev are > > case-by-case (hopefully all coherent since it concerns newer platforms). > > > > More worrying, I thought we disabled No-Snoop for VFIO but I was wrong, > > it's left enabled. On Arm I don't think userspace can perform the right > > cache maintenance operations to maintain coherency with a device that > > issues No-Snoop writes. Userspace can issue clean+invalidate but not > > invalidate alone, so there is no equivalent to > > arch_sync_dma_for_cpu(). > > So what happens in a VM? Does a VM know that arch_sync_dma_for_cpu() > is not available? This would only affect userspace drivers, it's only host or guest userspace that cannot issue the maintenance operations. The VM can do arch_sync_dma_for_cpu() Thanks, Jean > > And how does this work with the nested IOMMU translation? I thought I > read in the SMMU spec that the io page table entries could control > cachability including in nesting cases? > > > I think the worse that can happen is the device owner shooting itself in > > the foot by using No-Snoop, but would it hurt to disable it? > > No, the worst is the same as Intel - a driver running in the guest VM > assumes it can use arch_sync_dma_for_cpu() and acts accordingly, > resulting in a broken VM. > > Jason