From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Burakov, Anatoly" Subject: Re: [RFC] pci: force address of mappings in secondary process Date: Mon, 28 Jan 2019 09:59:56 +0000 Message-ID: References: <20170711011242.4606-1-stephen@networkplumber.org> <3f48cbd3-1e60-5a92-3929-7ba06e3cc69c@intel.com> <6f4d8c45-4215-e258-49a0-93235806d00d@intel.com> <20190124093734.0d3312c2@shemminger-XPS-13-9360> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: dpdk-dev , Thomas Monjalon To: Stephen Hemminger , Ferruh Yigit Return-path: Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by dpdk.org (Postfix) with ESMTP id C539954AE for ; Mon, 28 Jan 2019 11:00:02 +0100 (CET) In-Reply-To: <20190124093734.0d3312c2@shemminger-XPS-13-9360> Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 23-Jan-19 8:37 PM, Stephen Hemminger wrote: > On Wed, 23 Jan 2019 19:21:03 +0000 > Ferruh Yigit wrote: > >> On 7/12/2017 9:58 AM, jianfeng.tan at intel.com (Tan, Jianfeng) wrote: >>>> -----Original Message----- >>>> From: Gonzalez Monroy, Sergio >>>> Sent: Wednesday, July 12, 2017 3:32 PM >>>> To: Tan, Jianfeng; Stephen Hemminger; dev at dpdk.org >>>> Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary >>>> process >>>> >>>> On 12/07/2017 03:45, Tan, Jianfeng wrote: >>>>> >>>>>> -----Original Message----- >>>>>> From: Gonzalez Monroy, Sergio >>>>>> Sent: Tuesday, July 11, 2017 7:36 PM >>>>>> To: Tan, Jianfeng; Stephen Hemminger; dev at dpdk.org >>>>>> Subject: Re: [dpdk-dev] [RFC] pci: force address of mappings in secondary >>>>>> process >>>>>> >>>>>> On 11/07/2017 02:56, Tan, Jianfeng wrote: >>>>>>>> -----Original Message----- >>>>>>>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Stephen >>>>>>>> Hemminger >>>>>>>> Sent: Tuesday, July 11, 2017 9:13 AM >>>>>>>> To: dev at dpdk.org >>>>>>>> Cc: Stephen Hemminger >>>>>>>> Subject: [dpdk-dev] [RFC] pci: force address of mappings in secondary >>>>>>>> process >>>>>>>> >>>>>>>> The PCI memory resources in the secondary process should be in >>>>>>>> the exact same location as the primary process. Otherwise >>>>>>>> there is a risk of a stray pointer. >>>>>>>> >>>>>>>> Not sure if this is right, but it looks like a potential >>>>>>>> problem. >>>>>>>> >>>>>>>> --- >>>>>>>> lib/librte_eal/common/eal_common_pci_uio.c | 2 +- >>>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>>>> >>>>>>>> diff --git a/lib/librte_eal/common/eal_common_pci_uio.c >>>>>>>> b/lib/librte_eal/common/eal_common_pci_uio.c >>>>>>>> index 367a6816dcb8..2156b1a436c4 100644 >>>>>>>> --- a/lib/librte_eal/common/eal_common_pci_uio.c >>>>>>>> +++ b/lib/librte_eal/common/eal_common_pci_uio.c >>>>>>>> @@ -77,7 +77,7 @@ pci_uio_map_secondary(struct rte_pci_device >>>> *dev) >>>>>>>> >>>>>>>> void *mapaddr = pci_map_resource(uio_res- >>>>>>>>> maps[i].addr, >>>>>>>> fd, (off_t)uio_res->maps[i].offset, >>>>>>>> - (size_t)uio_res->maps[i].size, 0); >>>>>>>> + (size_t)uio_res->maps[i].size, >>>>>>>> MAP_FIXED); >>>>>>>> /* fd is not needed in slave process, close it */ >>>>>>>> close(fd); >>>>>>>> if (mapaddr != uio_res->maps[i].addr) { >>>>>>>> -- >>>>>>>> 2.11.0 >>>>>>> +1 for this RFC. I also once encounter such problem, and I use the same >>>>>> way to solve it. The addr parameter of mmap() syscall is only a hint instead >>>> of >>>>>> a must even the VMA is not occupied yet. >>>>>>> Thanks, >>>>>>> Jianfeng >>>>>> How do you know the VMA is not occupied? >>>>> I did by check /proc/self/maps. >>>>> >>>>>> I think the risk here is that the dynamic linker loaded some shared >>>>>> library in that VMA, and forcing MAP_FIXED is not a safe solution. >>>>>> What I have observed is that Linux will return a different VMA than the >>>>>> one hinted when there is already a mapping in the requested/hinted >>>> VMA. >>>>> IMO, that's not the target of this RFC. The target is to solve the situation (in >>>> current primary/secondary model) that kernel will not use the addr even >>>> there's no VMA on that addr. This is my understanding, Stephen, please >>>> correct me if I'm wrong. >>>> >>>> The point I was trying to make is, that you do not know if there is a >>>> mapping or not in that address, and by using MAP_FIXED it will unmap >>>> whatever was in there before. >>> >>> Oh, I missed that if there's conflict, the existing VMA will be unmapped. That's a bad effect. >>> >>>> >>>> So unless you parse /proc/self/maps and check that the VMA range is not >>>> being used, forcing MAP_FIXED is not safe. >>>> >>>>>> I reckon this is a similar issue as we have with the multi-process model >>>>>> when we do not get the VMA requested for the huge-pages. >>>>>> AFAIK we do not have a robust solution for this issue other than restart >>>>>> the program and hope the dynamic linker does not map anything in the >>>> VMA >>>>>> ranges that we need to map from the primary. This is also assuming that >>>>>> the application does not allocate memory and maps things before calling >>>>>> eal_init as it could potentially use VMA ranges that we need in the >>>>>> secondary process. >>>>> This is another problem. >>>> >>>> It is the same problem, VMA ranges that we need to map being already used. >>> >>> Still two problems from my side: >>> (1) A VMA already exists on that addr/len range; conflict happens. >>> (2) Kernel will not allocate the VMA to DPDK even there is no VMA on that ranges; there's no conflict. >> >> Hi Stephen, >> >> Both Sergio & Jianfeng are not active anymore in DPDK, which the discussion >> seems was going on with. cc'ed Anatoly. >> >> Is this RFC still valid? >> Should we expect an update on it? >> >> Thanks, >> ferruh >> >>> >>> Thanks, >>> Jianfeng >>> >>>> >>>> Thanks, >>>> Sergio >>>> >>>>>> The proposal for new secondary process model would solve these issues: >>>>>> http://dpdk.org/ml/archives/dev/2017-May/066147.html >>>>> And yes, this might happen to solve the targeted issue in this RFC. But >>>> before the new model is out, this patch seems a workable way for the >>>> original issue. >>>>> >>>>> Thanks, >>>>> Jianfeng >>>>> >>>>>> Thanks, >>>>>> Sergio >>>> >>> >>> >> > > MAP_FIXED is the wrong solution. If the secondary passes the address > it wants, and gets something else that means that is overlapping. > The current code returns an error which is the best response. > ...which is why this: http://patches.dpdk.org/bundle/aburakov/reliable_device_map/ is a better approach :) -- Thanks, Anatoly