From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx2.parallels.com ([199.115.105.18]:33076 "EHLO mx2.parallels.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756929AbaEIUZn convert rfc822-to-8bit (ORCPT ); Fri, 9 May 2014 16:25:43 -0400 From: James Bottomley To: "bhelgaas@google.com" CC: "linux-kernel@vger.kernel.org" , "rdunlap@infradead.org" , "Liviu.Dudau@arm.com" , "joro@8bytes.org" , "gregkh@linuxfoundation.org" , "dwmw2@infradead.org" , "linux-pci@vger.kernel.org" , "arnd@arndb.de" , "linux-doc@vger.kernel.org" Subject: Re: [PATCH v2 4/5] iommu: Use dma_addr_t for IOVA arguments Date: Fri, 9 May 2014 20:25:40 +0000 Message-ID: <1399667139.2166.71.camel@dabdike.int.hansenpartnership.com> References: <20140506223250.17968.27054.stgit@bhelgaas-glaptop.roam.corp.google.com> <1399629530.879.21.camel@i7.infradead.org> <20140509153226.GA3571@google.com> <9450765.GaoRYW154J@wuerfel> In-Reply-To: Content-Type: text/plain; charset=US-ASCII MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org List-ID: On Fri, 2014-05-09 at 14:19 -0600, Bjorn Helgaas wrote: > On Fri, May 9, 2014 at 1:52 PM, Arnd Bergmann wrote: > > On Friday 09 May 2014 09:32:26 Bjorn Helgaas wrote: > >> > There are people who care deeply about the performance of IOMMU API > >> > map/unmap. It isn't used *just* for virtual machines any more. See > >> > drivers/infiniband/hw/usnic/usnic_uiom.c for example. > >> > >> Of course we should care about IOMMU API performance. We should also > >> care about interface consistency, and it seems there's a tradeoff in > >> this case. I said "relatively" because I expect map/unmap to be less > >> frequent than read/write operations that use the mapping. I don't > >> know anything about infiniband, so maybe that assumption is false > >> there. > > > > In most drivers using the streaming DMA API, every mapping is used > > exactly once. Think of network or block drivers: they rarely send > > the same data twice to the device, and it usually comes from or > > goes to some user space buffer. > > Oh, good point. I don't work that high up in the stack, so thanks for > reminding me of that. To round this out, for most devices we have two types of mappings: the mailbox ones, which designate regions of communication memory between the kernel and the device which are usually permanent mappings, and the transmission mappings: every bit of data we send to the device is mapped, sent/received and then unmapped. The setup and teardown costs factor into the throughput. Some high iops devices (like SSD or high speed net) are peculiarly sensitive to this. James