From mboxrd@z Thu Jan 1 00:00:00 1970 From: Santosh Shukla Subject: Re: [PATCH] eal: map io resources for non x86 architectures Date: Tue, 29 Dec 2015 20:21:48 +0530 Message-ID: References: <2241331.HNmyzf8foi@xps13> <2979402.yeVYlcCDUH@xps13> <20151218053053.GL29571@yliu-dev.sh.intel.com> <20151218082139.GC18863@yliu-dev.sh.intel.com> <1451397878.18084.4.camel@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: "dev@dpdk.org" To: Alex Williamson Return-path: Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) by dpdk.org (Postfix) with ESMTP id B194A8E96 for ; Tue, 29 Dec 2015 15:51:50 +0100 (CET) Received: by mail-pa0-f49.google.com with SMTP id cy9so125193305pac.0 for ; Tue, 29 Dec 2015 06:51:50 -0800 (PST) In-Reply-To: <1451397878.18084.4.camel@redhat.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Dec 29, 2015 at 7:34 PM, Alex Williamson wrote: > On Tue, 2015-12-29 at 16:17 +0530, Santosh Shukla wrote: >> On Tue, Dec 29, 2015 at 3:26 PM, Burakov, Anatoly >> wrote: >> > Hi Santosh, >> > >> > > On Fri, Dec 18, 2015 at 6:25 PM, Santosh Shukla > > > om> >> > > wrote: >> > > > On Fri, Dec 18, 2015 at 1:51 PM, Yuanhan Liu >> > > > wrote: >> > > > > On Fri, Dec 18, 2015 at 01:24:41PM +0530, Santosh Shukla >> > > > > wrote: >> > > > > > > > I guess we have done enough evaluation / investigation >> > > > > > > > that >> > > > > > > > suggest - so to map iopci region to userspace in arch >> > > > > > > > agnostic-way - >> > > > > > > > >> > > > > > > > # either we need to modify kernel >> > > > > > > > - Make sure all the non-x86 arch to >> > > > > > > > support >> > > > > > > > mapping for iopci region (i.e. pci_mmap_page_range). I >> > > > > > > > don;t >> > > > > > > > think its a correct approach though. >> > > > > > > > or >> > > > > > > > - include /dev/ioport char-mem device >> > > > > > > > file who >> > > > > > > > could do more than byte operation, Note that this >> > > > > > > > implementation >> > > > > > > > does not exist in kernel. I could send an RFC to lkml. >> > > > > > > >> > > > > > > Maybe you could propose the two to lkml, to get some >> > > > > > > feedbacks >> > > > > > > from those kernel/ARM gurus? Please cc me if you do so. >> > > > > > > >> > > > > > >> > > > > > The latter one I already shared old lkml thread, Pl. >> > > > > > revisit my v1 >> > > > > > 0/6 patch [1] and in that refer [2]. >> > > > > >> > > > > Oops, sorry, a bit busy, that I didn't look at it carefully. >> > > > > My bad, >> > > > > anyway. >> > > > > >> > > > > > Josh has already proposed to lkml but for some reason >> > > > > > thread didn't >> > > > > > went far. I can restart that discussion giving dpdk use- >> > > > > > case as an >> > > > > > example/ requirement. >> > > > > >> > > > > I had a quick go through of the discussion. Both hpa and Arnd >> > > > > seem to >> > > > > be fine with the ioctl interface on /dev/port. Have you tried >> > > > > that? >> > > > > And if you want to restart it, ioctl might be a better option >> > > > > than >> > > > > /dev/ioport, judging from the discussion. >> > > > > >> > > > >> > > > I tried legacy patch and re-writing with ioctl-way; doing >> > > > changes in >> > > > dpdk port as well in kernel, had to test on quite other hw not >> > > > only >> > > > arm64 though! so it will take time for me, I am travelling >> > > > tomorrow so >> > > > bit delayed, We'll post patch to lkml and share dpdk-virtio >> > > > feedback >> > > > perhaps by Monday. >> > > > >> > > >> > > I posted a query about /dev/ioports approach in lkml thread [1], >> > > and Arnd >> > > suggested to use vfio framework but it looks like vfio too does >> > > not map >> > > ioresource_io region. Same communicated to Arnd and waiting for >> > > his reply. >> > > >> > > In mean time I like to ask general question; >> > > - Has anyone tried vfio/non-iommu approach for virtio pmd driver? >> > > If not >> > > then Is there any plan? Someone planning to take up. >> > > [1] https://lkml.org/lkml/2015/12/23/145 >> > >> > I have submitted a patch to support no-iommu mode, but I'm not >> > aware of anyone trying VFIO-noiommu at all. That's probably >> > expected since it's Christmas/New Year in a lot of places, and >> > everyone is on a break. >> > >> > That said, I'm not sure I completely understand what is it that >> > you're asking about. The code you're referring to (in vfio_pci.c, >> > line 854 as of kernel 4.3) is checking if a PCI BAR is available >> > for IO (hence checking if the IORESOURCE_MEM >> >> >> Thanks for reply! You comment might help to move this discuss to next >> level. >> >> Look at kernel/resource.c, it exports two symbol ioport_resource and >> iomem_resource and sets appropriate flag type i.e.. IORESOURCE_IO and >> IORESOURCE_MEM. In virtio-net case; it creates both pci region i.e.. >> _io bar and _mem bar. dpdk virtio pmd driver (<= 0.95 virtio spec) >> uses pci _io bar region for device initialization as virtio headers >> are locate at pci _io bar region. Since it uses pci _iobar region so >> likely it update pci_resource.[index].flag = IORESOURCE_IO. and vfio >> mmap function wont handle ioresource_io (i guess). And that is why I >> asked same to lkml thread. >> >> >> bit is set). There isn't any "ioresource_mem" region as far as VFIO >> is >> concerned, there are only BARs, ROM, VGA and PCI >> config regions (linux/vfio.h, line 314 as of kernel 4.3). So if >> you're >> missing some PCI regions for VFIO to map, they would first need to be >> added to VFIO kernel implementation before they can be used by DPDK. >> That is, unless I'm misunderstanding something :) >> > >> >> Agree. As mentioned above, I guess ioresource_io pci bar >> implementation missing in vfio kernel? but before adding code support >> in kernel I would like to hear from experts example, You, Alex! >> (looping alex) > > MMIO and I/O port space are simply regions as far as VFIO is concerned. > The userspace API supports the concept of I/O port regions that can be > mmap'd by userspace, but the implementation does not since I/O port > regions cannot be mmap'd on x86. Someone needs to propose some code > for vfio-pci to support it. Thanks, > I will work on this and post an RFC soon to LKML, That RFC targetted to map IOport region for non-x86 platform in particular. Thanks for confirming the vfio behaviour. > Alex