From: "Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com> To: Jason Gunthorpe <jgg@ziepe.ca> Cc: "Wangzhou (B)" <wangzhou1@hisilicon.com>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Arnd Bergmann <arnd@arndb.de>, Zhangfei Gao <zhangfei.gao@linaro.org>, "linux-accelerators@lists.ozlabs.org" <linux-accelerators@lists.ozlabs.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "Liguozhu (Kenneth)" <liguozhu@hisilicon.com>, "chensihang (A)" <chensihang1@hisilicon.com> Subject: RE: [RFC PATCH v2] uacce: Add uacce_ctrl misc device Date: Tue, 26 Jan 2021 01:26:45 +0000 [thread overview] Message-ID: <d7fce136c3644755a7aea5794bddf453@hisilicon.com> (raw) In-Reply-To: <20210126011304.GZ4605@ziepe.ca> > -----Original Message----- > From: Jason Gunthorpe [mailto:jgg@ziepe.ca] > Sent: Tuesday, January 26, 2021 2:13 PM > To: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com> > Cc: Wangzhou (B) <wangzhou1@hisilicon.com>; Greg Kroah-Hartman > <gregkh@linuxfoundation.org>; Arnd Bergmann <arnd@arndb.de>; Zhangfei Gao > <zhangfei.gao@linaro.org>; linux-accelerators@lists.ozlabs.org; > linux-kernel@vger.kernel.org; iommu@lists.linux-foundation.org; > linux-mm@kvack.org; Liguozhu (Kenneth) <liguozhu@hisilicon.com>; chensihang > (A) <chensihang1@hisilicon.com> > Subject: Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device > > On Mon, Jan 25, 2021 at 11:35:22PM +0000, Song Bao Hua (Barry Song) wrote: > > > > On Mon, Jan 25, 2021 at 10:21:14PM +0000, Song Bao Hua (Barry Song) wrote: > > > > mlock, while certainly be able to prevent swapping out, it won't > > > > be able to stop page moving due to: > > > > * memory compaction in alloc_pages() > > > > * making huge pages > > > > * numa balance > > > > * memory compaction in CMA > > > > > > Enabling those things is a major reason to have SVA device in the > > > first place, providing a SW API to turn it all off seems like the > > > wrong direction. > > > > I wouldn't say this is a major reason to have SVA. If we read the > > history of SVA and papers, people would think easy programming due > > to data struct sharing between cpu and device, and process space > > isolation in device would be the major reasons for SVA. SVA also > > declares it supports zero-copy while zero-copy doesn't necessarily > > depend on SVA. > > Once you have to explicitly make system calls to declare memory under > IO, you loose all of that. > > Since you've asked the app to be explicit about the DMAs it intends to > do, there is not really much reason to use SVA for those DMAs anymore. Let's see a non-SVA case. We are not using SVA, we can have a memory pool by hugetlb or pin, and app can allocate memory from this pool, and get stable I/O performance on the memory from the pool. But device has its separate page table which is not bound with this process, thus lacking the protection of process space isolation. Plus, CPU and device are using different address. And then we move to SVA case, we can still have a memory pool by hugetlb or pin, and app can allocate memory from this pool since this pool is mapped to the address space of the process, and we are able to get stable I/O performance since it is always there. But in this case, device is using the page table of process with the full permission control. And they are using same address and can possibly enjoy the easy programming if HW supports. SVA is not doom to work with IO page fault only. If we have SVA+pin, we would get both sharing address and stable I/O latency. > > Jason Thanks Barry
WARNING: multiple messages have this Message-ID (diff)
From: "Song Bao Hua (Barry Song)" <song.bao.hua@hisilicon.com> To: Jason Gunthorpe <jgg@ziepe.ca> Cc: "chensihang \(A\)" <chensihang1@hisilicon.com>, Arnd Bergmann <arnd@arndb.de>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, Zhangfei Gao <zhangfei.gao@linaro.org>, "Liguozhu \(Kenneth\)" <liguozhu@hisilicon.com>, "linux-accelerators@lists.ozlabs.org" <linux-accelerators@lists.ozlabs.org> Subject: RE: [RFC PATCH v2] uacce: Add uacce_ctrl misc device Date: Tue, 26 Jan 2021 01:26:45 +0000 [thread overview] Message-ID: <d7fce136c3644755a7aea5794bddf453@hisilicon.com> (raw) In-Reply-To: <20210126011304.GZ4605@ziepe.ca> > -----Original Message----- > From: Jason Gunthorpe [mailto:jgg@ziepe.ca] > Sent: Tuesday, January 26, 2021 2:13 PM > To: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com> > Cc: Wangzhou (B) <wangzhou1@hisilicon.com>; Greg Kroah-Hartman > <gregkh@linuxfoundation.org>; Arnd Bergmann <arnd@arndb.de>; Zhangfei Gao > <zhangfei.gao@linaro.org>; linux-accelerators@lists.ozlabs.org; > linux-kernel@vger.kernel.org; iommu@lists.linux-foundation.org; > linux-mm@kvack.org; Liguozhu (Kenneth) <liguozhu@hisilicon.com>; chensihang > (A) <chensihang1@hisilicon.com> > Subject: Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device > > On Mon, Jan 25, 2021 at 11:35:22PM +0000, Song Bao Hua (Barry Song) wrote: > > > > On Mon, Jan 25, 2021 at 10:21:14PM +0000, Song Bao Hua (Barry Song) wrote: > > > > mlock, while certainly be able to prevent swapping out, it won't > > > > be able to stop page moving due to: > > > > * memory compaction in alloc_pages() > > > > * making huge pages > > > > * numa balance > > > > * memory compaction in CMA > > > > > > Enabling those things is a major reason to have SVA device in the > > > first place, providing a SW API to turn it all off seems like the > > > wrong direction. > > > > I wouldn't say this is a major reason to have SVA. If we read the > > history of SVA and papers, people would think easy programming due > > to data struct sharing between cpu and device, and process space > > isolation in device would be the major reasons for SVA. SVA also > > declares it supports zero-copy while zero-copy doesn't necessarily > > depend on SVA. > > Once you have to explicitly make system calls to declare memory under > IO, you loose all of that. > > Since you've asked the app to be explicit about the DMAs it intends to > do, there is not really much reason to use SVA for those DMAs anymore. Let's see a non-SVA case. We are not using SVA, we can have a memory pool by hugetlb or pin, and app can allocate memory from this pool, and get stable I/O performance on the memory from the pool. But device has its separate page table which is not bound with this process, thus lacking the protection of process space isolation. Plus, CPU and device are using different address. And then we move to SVA case, we can still have a memory pool by hugetlb or pin, and app can allocate memory from this pool since this pool is mapped to the address space of the process, and we are able to get stable I/O performance since it is always there. But in this case, device is using the page table of process with the full permission control. And they are using same address and can possibly enjoy the easy programming if HW supports. SVA is not doom to work with IO page fault only. If we have SVA+pin, we would get both sharing address and stable I/O latency. > > Jason Thanks Barry _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-01-26 1:31 UTC|newest] Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-01-25 8:34 [RFC PATCH v2] uacce: Add uacce_ctrl misc device Zhou Wang 2021-01-25 8:34 ` Zhou Wang 2021-01-25 9:28 ` Greg Kroah-Hartman 2021-01-25 9:28 ` Greg Kroah-Hartman 2021-01-25 12:47 ` Zhou Wang 2021-01-25 12:47 ` Zhou Wang 2021-01-25 15:47 ` Jason Gunthorpe 2021-01-25 15:47 ` Jason Gunthorpe 2021-01-25 22:21 ` Song Bao Hua (Barry Song) 2021-01-25 22:21 ` Song Bao Hua (Barry Song) 2021-01-25 22:21 ` Song Bao Hua (Barry Song) 2021-01-25 23:16 ` Jason Gunthorpe 2021-01-25 23:16 ` Jason Gunthorpe 2021-01-25 23:16 ` Jason Gunthorpe 2021-01-25 23:35 ` Song Bao Hua (Barry Song) 2021-01-25 23:35 ` Song Bao Hua (Barry Song) 2021-01-25 23:35 ` Song Bao Hua (Barry Song) 2021-01-26 1:13 ` Jason Gunthorpe 2021-01-26 1:13 ` Jason Gunthorpe 2021-01-26 1:13 ` Jason Gunthorpe 2021-01-26 1:26 ` Song Bao Hua (Barry Song) [this message] 2021-01-26 1:26 ` Song Bao Hua (Barry Song) 2021-01-26 1:26 ` Song Bao Hua (Barry Song) 2021-01-26 18:20 ` Jason Gunthorpe 2021-01-26 18:20 ` Jason Gunthorpe 2021-01-26 18:20 ` Jason Gunthorpe 2021-01-28 1:28 ` Song Bao Hua (Barry Song) 2021-01-28 1:28 ` Song Bao Hua (Barry Song) 2021-01-28 1:28 ` Song Bao Hua (Barry Song) 2021-01-29 10:09 ` Tian, Kevin 2021-01-29 10:09 ` Tian, Kevin 2021-01-29 10:33 ` Song Bao Hua (Barry Song) 2021-01-29 10:33 ` Song Bao Hua (Barry Song) 2021-02-01 23:44 ` Jason Gunthorpe 2021-02-01 23:44 ` Jason Gunthorpe 2021-02-01 23:44 ` Jason Gunthorpe 2021-02-02 0:22 ` Song Bao Hua (Barry Song) 2021-02-02 0:22 ` Song Bao Hua (Barry Song) 2021-02-02 0:22 ` Song Bao Hua (Barry Song) 2021-02-02 2:51 ` Tian, Kevin 2021-02-02 2:51 ` Tian, Kevin 2021-02-02 2:51 ` Tian, Kevin 2021-02-02 3:47 ` Song Bao Hua (Barry Song) 2021-02-02 3:47 ` Song Bao Hua (Barry Song) 2021-02-02 3:47 ` Song Bao Hua (Barry Song) 2021-01-26 9:00 ` Zhou Wang 2021-01-26 9:00 ` Zhou Wang
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=d7fce136c3644755a7aea5794bddf453@hisilicon.com \ --to=song.bao.hua@hisilicon.com \ --cc=arnd@arndb.de \ --cc=chensihang1@hisilicon.com \ --cc=gregkh@linuxfoundation.org \ --cc=iommu@lists.linux-foundation.org \ --cc=jgg@ziepe.ca \ --cc=liguozhu@hisilicon.com \ --cc=linux-accelerators@lists.ozlabs.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=wangzhou1@hisilicon.com \ --cc=zhangfei.gao@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.