From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932111AbdCFMr0 (ORCPT ); Mon, 6 Mar 2017 07:47:26 -0500 Received: from foss.arm.com ([217.140.101.70]:34178 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753142AbdCFMrK (ORCPT ); Mon, 6 Mar 2017 07:47:10 -0500 Subject: Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults References: <1488538070-12549-1-git-send-email-sunil.kovvuri@gmail.com> <1488538070-12549-2-git-send-email-sunil.kovvuri@gmail.com> <20170303.095635.355325147613813698.davem@davemloft.net> To: Sunil Kovvuri , David Miller Cc: Linux Netdev List , Sunil Goutham , LKML , LAKML From: Robin Murphy Message-ID: Date: Mon, 6 Mar 2017 12:46:16 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/03/17 05:54, Sunil Kovvuri wrote: > On Fri, Mar 3, 2017 at 11:26 PM, David Miller wrote: >> From: sunil.kovvuri@gmail.com >> Date: Fri, 3 Mar 2017 16:17:47 +0530 >> >>> @@ -1643,6 +1650,9 @@ static int nicvf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) >>> if (!pass1_silicon(nic->pdev)) >>> nic->hw_tso = true; >>> >>> + /* Check if we are attached to IOMMU */ >>> + nic->iommu_domain = iommu_get_domain_for_dev(dev); >> >> This function is not universally available. > > Even if CONFIG_IOMMU_API is not enabled, it will return NULL and will be okay. > http://lxr.free-electrons.com/source/include/linux/iommu.h#L400 > >> >> This looks very hackish to me anyways, how all of this stuff is supposed >> to work is that you simply use the DMA interfaces unconditionally and >> whatever is behind the operations takes care of everything. >> >> Doing it conditionally in the driver with all of this special IOMMU >> domain et al. knowledge makes no sense to me at all. >> >> I don't see other drivers doing stuff like this at all, so if you're >> going to handle this in a unique way like this you better write >> several paragraphs in your commit message explaining why this weird >> crap is necessary. > > I already tried to explain in the commit message that HW anyway takes care > of data coherency, so calling DMA interfaces when there is no IOMMU will > only result in performance drop. > > We are seeing a 0.75Mpps drop with IP forwarding rate due to that. > Hence I have restricted calling DMA interfaces to only when IOMMU is enabled. What's 0.07Mpps as a percentage of baseline? On a correctly configured coherent arm64 system, in the absence of an IOMMU, dma_map_*() is essentially just virt_to_phys() behind a function call or two, so I'd be interested to know where any non-trivial overhead might be coming from. Robin. > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >