* Preventing zero GPU virtual address allocation @ 2015-03-05 2:56 Song, Ruiling 2015-03-05 12:52 ` [Intel-gfx] " Daniel Vetter 0 siblings, 1 reply; 31+ messages in thread From: Song, Ruiling @ 2015-03-05 2:56 UTC (permalink / raw) To: Vetter, Daniel Cc: Zou, Nanhai, intel-gfx, Yang, Rong R, beignet, Weinehall, David [-- Attachment #1: Type: text/plain, Size: 479 bytes --] Hi Daniel, OpenCL language support NULL pointer, using zero as the NULL pointer is the obvious way. That is zero will be treated as invalid address. Then it requires drm won't allocate zero to drm buffer. And David in CC list has help us make a patch, please see attached. The logic is only for ppgtt, and he said zero offset is used under ggtt. My question is what is offset zero used under ggtt? Will it make sure zero is not allocatable to drm buffer object? Ruiling [-- Attachment #2: nozerooffset2.patch --] [-- Type: application/octet-stream, Size: 583 bytes --] diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 996f60fae212..72657aa40319 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -3530,6 +3530,12 @@ i915_gem_object_bind_to_vm(struct drm_i915_gem_object *obj, struct i915_vma *vma; int ret; + /* We reserve address 0 when using ppgtt, + * since it's used by opencl as the NULL pointer + */ + if (start == 0 && !i915_is_ggtt(vm)) + start = 1; + fence_size = i915_gem_get_gtt_size(dev, obj->base.size, obj->tiling_mode); [-- Attachment #3: Type: text/plain, Size: 153 bytes --] _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-05 2:56 Preventing zero GPU virtual address allocation Song, Ruiling @ 2015-03-05 12:52 ` Daniel Vetter 2015-03-05 13:01 ` Chris Wilson 0 siblings, 1 reply; 31+ messages in thread From: Daniel Vetter @ 2015-03-05 12:52 UTC (permalink / raw) To: Song, Ruiling Cc: Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David On Thu, Mar 05, 2015 at 02:56:52AM +0000, Song, Ruiling wrote: > Hi Daniel, > > OpenCL language support NULL pointer, using zero as the NULL pointer is > the obvious way. That is zero will be treated as invalid address. Then > it requires drm won't allocate zero to drm buffer. And David in CC > list has help us make a patch, please see attached. The logic is only > for ppgtt, and he said zero offset is used under ggtt. My question is > what is offset zero used under ggtt? Will it make sure zero is not > allocatable to drm buffer object? The code in i915_gem_execbuf.c already supports an optional bias to avoid putting a buffer into the first few kb. See __EXEC_OBJECT_NEEDS_BIAS. I suggest you expose this to userspace, which also address your issue that you didn't add an abi revision flag. And since this is abi I need open-source userspace and all that. Thanks, Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Preventing zero GPU virtual address allocation 2015-03-05 12:52 ` [Intel-gfx] " Daniel Vetter @ 2015-03-05 13:01 ` Chris Wilson 2015-03-05 15:27 ` [Intel-gfx] " Daniel Vetter 0 siblings, 1 reply; 31+ messages in thread From: Chris Wilson @ 2015-03-05 13:01 UTC (permalink / raw) To: Daniel Vetter Cc: Yang, Rong R, intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel On Thu, Mar 05, 2015 at 01:52:51PM +0100, Daniel Vetter wrote: > On Thu, Mar 05, 2015 at 02:56:52AM +0000, Song, Ruiling wrote: > > Hi Daniel, > > > > OpenCL language support NULL pointer, using zero as the NULL pointer is > > the obvious way. That is zero will be treated as invalid address. Then > > it requires drm won't allocate zero to drm buffer. And David in CC > > list has help us make a patch, please see attached. The logic is only > > for ppgtt, and he said zero offset is used under ggtt. My question is > > what is offset zero used under ggtt? Will it make sure zero is not > > allocatable to drm buffer object? > > The code in i915_gem_execbuf.c already supports an optional bias to avoid > putting a buffer into the first few kb. See __EXEC_OBJECT_NEEDS_BIAS. I > suggest you expose this to userspace, which also address your issue that > you didn't add an abi revision flag. A better API would be to allow userspace to request a buffer to place at a specific point in the VM and fail if that is not possible aka soft-pinning. Then OCL could assign a bo to offset 0 and detect writes to the NULL address if it so desired. With full-ppgtt, userspace can be sure of being able to evict any location in its VM and so also allows graceful detection of scenarios under which it cannot provide the NULL address safety feature (and opt not to run, or just bury its head in the sand). -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-05 13:01 ` Chris Wilson @ 2015-03-05 15:27 ` Daniel Vetter 2015-03-05 21:07 ` Chris Wilson 2015-03-06 2:11 ` [Intel-gfx] " Zou, Nanhai 0 siblings, 2 replies; 31+ messages in thread From: Daniel Vetter @ 2015-03-05 15:27 UTC (permalink / raw) To: Chris Wilson, Daniel Vetter, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David On Thu, Mar 05, 2015 at 01:01:21PM +0000, Chris Wilson wrote: > On Thu, Mar 05, 2015 at 01:52:51PM +0100, Daniel Vetter wrote: > > On Thu, Mar 05, 2015 at 02:56:52AM +0000, Song, Ruiling wrote: > > > Hi Daniel, > > > > > > OpenCL language support NULL pointer, using zero as the NULL pointer is > > > the obvious way. That is zero will be treated as invalid address. Then > > > it requires drm won't allocate zero to drm buffer. And David in CC > > > list has help us make a patch, please see attached. The logic is only > > > for ppgtt, and he said zero offset is used under ggtt. My question is > > > what is offset zero used under ggtt? Will it make sure zero is not > > > allocatable to drm buffer object? > > > > The code in i915_gem_execbuf.c already supports an optional bias to avoid > > putting a buffer into the first few kb. See __EXEC_OBJECT_NEEDS_BIAS. I > > suggest you expose this to userspace, which also address your issue that > > you didn't add an abi revision flag. > > A better API would be to allow userspace to request a buffer to place at > a specific point in the VM and fail if that is not possible aka > soft-pinning. Then OCL could assign a bo to offset 0 and detect writes > to the NULL address if it so desired. With full-ppgtt, userspace can be > sure of being able to evict any location in its VM and so also allows > graceful detection of scenarios under which it cannot provide the NULL > address safety feature (and opt not to run, or just bury its head in the > sand). I recommended exposing the PIN_BIAS since that will work without full ppgtt too. And yeah for full ppgtt we could just use svm where userspace controls the address, but since that's still a bit out we might need a quick interim solution? -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-05 15:27 ` [Intel-gfx] " Daniel Vetter @ 2015-03-05 21:07 ` Chris Wilson 2015-03-09 15:46 ` Jesse Barnes 2015-03-06 2:11 ` [Intel-gfx] " Zou, Nanhai 1 sibling, 1 reply; 31+ messages in thread From: Chris Wilson @ 2015-03-05 21:07 UTC (permalink / raw) To: Daniel Vetter Cc: Yang, Rong R, intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel On Thu, Mar 05, 2015 at 04:27:59PM +0100, Daniel Vetter wrote: > I recommended exposing the PIN_BIAS since that will work without full > ppgtt too. And yeah for full ppgtt we could just use svm where userspace > controls the address, but since that's still a bit out we might need a > quick interim solution? Letting userspace control the address of bo used in a batch is about 2 patches each of ~100 lines. And it could be used will full-ppgtt before svm if mesa wants to take complete control of its layout. I think it is one of those useful tools that is likely to find uses far beyond the initial justification. -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Preventing zero GPU virtual address allocation 2015-03-05 21:07 ` Chris Wilson @ 2015-03-09 15:46 ` Jesse Barnes 2015-03-09 15:49 ` Jesse Barnes 0 siblings, 1 reply; 31+ messages in thread From: Jesse Barnes @ 2015-03-09 15:46 UTC (permalink / raw) To: Chris Wilson, Daniel Vetter, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David On 03/05/2015 01:07 PM, Chris Wilson wrote: > On Thu, Mar 05, 2015 at 04:27:59PM +0100, Daniel Vetter wrote: >> I recommended exposing the PIN_BIAS since that will work without full >> ppgtt too. And yeah for full ppgtt we could just use svm where userspace >> controls the address, but since that's still a bit out we might need a >> quick interim solution? > > Letting userspace control the address of bo used in a batch is about 2 > patches each of ~100 lines. And it could be used will full-ppgtt before > svm if mesa wants to take complete control of its layout. I think it is > one of those useful tools that is likely to find uses far beyond the > initial justification. Well we need someone to pick it up and do it; we've already shafted userspace for several years due to foot dragging on the command parser... I hope something as simple as this doesn't stall out. Jesse _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Preventing zero GPU virtual address allocation 2015-03-09 15:46 ` Jesse Barnes @ 2015-03-09 15:49 ` Jesse Barnes 0 siblings, 0 replies; 31+ messages in thread From: Jesse Barnes @ 2015-03-09 15:49 UTC (permalink / raw) To: Chris Wilson, Daniel Vetter, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David, Zou, Nanhai On 03/09/2015 08:46 AM, Jesse Barnes wrote: > On 03/05/2015 01:07 PM, Chris Wilson wrote: >> On Thu, Mar 05, 2015 at 04:27:59PM +0100, Daniel Vetter wrote: >>> I recommended exposing the PIN_BIAS since that will work without full >>> ppgtt too. And yeah for full ppgtt we could just use svm where userspace >>> controls the address, but since that's still a bit out we might need a >>> quick interim solution? >> >> Letting userspace control the address of bo used in a batch is about 2 >> patches each of ~100 lines. And it could be used will full-ppgtt before >> svm if mesa wants to take complete control of its layout. I think it is >> one of those useful tools that is likely to find uses far beyond the >> initial justification. > > Well we need someone to pick it up and do it; we've already shafted > userspace for several years due to foot dragging on the command > parser... I hope something as simple as this doesn't stall out. Nevermind, I see you already did it. Ruiling and Nanhai, can you make sure Chris's proposed interface will work for you and provide a review if so? Thanks, Jesse _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-05 15:27 ` [Intel-gfx] " Daniel Vetter 2015-03-05 21:07 ` Chris Wilson @ 2015-03-06 2:11 ` Zou, Nanhai 2015-03-06 8:39 ` [Beignet] " Chris Wilson 1 sibling, 1 reply; 31+ messages in thread From: Zou, Nanhai @ 2015-03-06 2:11 UTC (permalink / raw) To: Daniel Vetter, Chris Wilson, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David I don't understand why we need a complex solution when there is already a simple solution with patch. What is the drawback of reserving page 0? Before we going to that complex solution, could we just reserve page zero? It is simple and straight forward. Thanks Zou Nanhai > -----Original Message----- > From: Beignet [mailto:beignet-bounces@lists.freedesktop.org] On Behalf Of > Daniel Vetter > Sent: Thursday, March 05, 2015 11:28 PM > To: Chris Wilson; Daniel Vetter; Song, Ruiling; Vetter, Daniel; > intel-gfx@lists.freedesktop.org; Yang, Rong R; beignet@lists.freedesktop.org; > Weinehall, David > Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address allocation > > On Thu, Mar 05, 2015 at 01:01:21PM +0000, Chris Wilson wrote: > > On Thu, Mar 05, 2015 at 01:52:51PM +0100, Daniel Vetter wrote: > > > On Thu, Mar 05, 2015 at 02:56:52AM +0000, Song, Ruiling wrote: > > > > Hi Daniel, > > > > > > > > OpenCL language support NULL pointer, using zero as the NULL > > > > pointer is the obvious way. That is zero will be treated as > > > > invalid address. Then it requires drm won't allocate zero to drm > > > > buffer. And David in CC list has help us make a patch, please see > > > > attached. The logic is only for ppgtt, and he said zero offset is > > > > used under ggtt. My question is what is offset zero used under > > > > ggtt? Will it make sure zero is not allocatable to drm buffer object? > > > > > > The code in i915_gem_execbuf.c already supports an optional bias to > > > avoid putting a buffer into the first few kb. See > > > __EXEC_OBJECT_NEEDS_BIAS. I suggest you expose this to userspace, > > > which also address your issue that you didn't add an abi revision flag. > > > > A better API would be to allow userspace to request a buffer to place > > at a specific point in the VM and fail if that is not possible aka > > soft-pinning. Then OCL could assign a bo to offset 0 and detect writes > > to the NULL address if it so desired. With full-ppgtt, userspace can > > be sure of being able to evict any location in its VM and so also > > allows graceful detection of scenarios under which it cannot provide > > the NULL address safety feature (and opt not to run, or just bury its > > head in the sand). > > I recommended exposing the PIN_BIAS since that will work without full ppgtt > too. And yeah for full ppgtt we could just use svm where userspace controls the > address, but since that's still a bit out we might need a quick interim solution? > -Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch > _______________________________________________ > Beignet mailing list > Beignet@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/beignet _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-06 2:11 ` [Intel-gfx] " Zou, Nanhai @ 2015-03-06 8:39 ` Chris Wilson 2015-03-09 2:34 ` Zou, Nanhai 0 siblings, 1 reply; 31+ messages in thread From: Chris Wilson @ 2015-03-06 8:39 UTC (permalink / raw) To: Zou, Nanhai Cc: intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel, Yang, Rong R On Fri, Mar 06, 2015 at 02:11:18AM +0000, Zou, Nanhai wrote: > I don't understand why we need a complex solution when there is already a simple solution with patch. > What is the drawback of reserving page 0? > Before we going to that complex solution, could we just reserve page zero? > It is simple and straight forward. Because it is a nonsense ABI constraint. If you want the equivalent of MAP_FIXED, we should give you MAP_FIXED. -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-06 8:39 ` [Beignet] " Chris Wilson @ 2015-03-09 2:34 ` Zou, Nanhai 2015-03-09 12:02 ` [Intel-gfx] " Chris Wilson 0 siblings, 1 reply; 31+ messages in thread From: Zou, Nanhai @ 2015-03-09 2:34 UTC (permalink / raw) To: Chris Wilson Cc: intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel, Yang, Rong R We don't need MAP_FIXED, we just want to avoid address 0 to be allocated. Though I think using MAP_FIXED is overkill, will bring much unnecessary complexity on both kernel and beignet side. I don't mind if people can provide stable MAP_FIXED patches to resolve this problem a few months or years later. At that time, kernel driver can revert the reserve page 0 patch. Before that reserve page 0 can benefit all the Beignet user without breaking anything. As I know, on CPU side, many arches with flexible address space like IA64 have reserved virtual page 0 to address this problem, I don't see why this is non sense. Thanks Zou Nanhai > -----Original Message----- > From: Chris Wilson [mailto:chris@chris-wilson.co.uk] > Sent: Friday, March 06, 2015 4:39 PM > To: Zou, Nanhai > Cc: Daniel Vetter; Song, Ruiling; Vetter, Daniel; intel-gfx@lists.freedesktop.org; > Yang, Rong R; beignet@lists.freedesktop.org; Weinehall, David > Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address allocation > > On Fri, Mar 06, 2015 at 02:11:18AM +0000, Zou, Nanhai wrote: > > I don't understand why we need a complex solution when there is already a > simple solution with patch. > > What is the drawback of reserving page 0? > > Before we going to that complex solution, could we just reserve page zero? > > It is simple and straight forward. > > Because it is a nonsense ABI constraint. If you want the equivalent of > MAP_FIXED, we should give you MAP_FIXED. > -Chris > > -- > Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-09 2:34 ` Zou, Nanhai @ 2015-03-09 12:02 ` Chris Wilson 2015-03-10 1:57 ` Zou, Nanhai 2015-03-13 9:10 ` [Beignet] " David Weinehall 0 siblings, 2 replies; 31+ messages in thread From: Chris Wilson @ 2015-03-09 12:02 UTC (permalink / raw) To: Zou, Nanhai Cc: Daniel Vetter, intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel, Yang, Rong R On Mon, Mar 09, 2015 at 02:34:46AM +0000, Zou, Nanhai wrote: > We don't need MAP_FIXED, we just want to avoid address 0 to be allocated. > > Though I think using MAP_FIXED is overkill, will bring much unnecessary complexity on both kernel and beignet side. > I don't mind if people can provide stable MAP_FIXED patches to resolve this problem a few months or years later. > > At that time, kernel driver can revert the reserve page 0 patch. > Before that reserve page 0 can benefit all the Beignet user without breaking anything. The point is that is becomes ABI. So no the kernel can't just revert it. There is nothing special about address 0 in ether GTT or virtual memory. If you require a special object allocated at address 0, allocate a special object at address 0. -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-09 12:02 ` [Intel-gfx] " Chris Wilson @ 2015-03-10 1:57 ` Zou, Nanhai 2015-03-13 9:10 ` [Beignet] " David Weinehall 1 sibling, 0 replies; 31+ messages in thread From: Zou, Nanhai @ 2015-03-10 1:57 UTC (permalink / raw) To: Chris Wilson Cc: Daniel Vetter, intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel, Yang, Rong R > -----Original Message----- > From: Chris Wilson [mailto:chris@chris-wilson.co.uk] > Sent: Monday, March 09, 2015 8:02 PM > To: Zou, Nanhai > Cc: Daniel Vetter; Song, Ruiling; Vetter, Daniel; intel-gfx@lists.freedesktop.org; > Yang, Rong R; beignet@lists.freedesktop.org; Weinehall, David > Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address allocation > > On Mon, Mar 09, 2015 at 02:34:46AM +0000, Zou, Nanhai wrote: > > We don't need MAP_FIXED, we just want to avoid address 0 to be allocated. > > > > Though I think using MAP_FIXED is overkill, will bring much unnecessary > complexity on both kernel and beignet side. > > I don't mind if people can provide stable MAP_FIXED patches to resolve this > problem a few months or years later. > > > > At that time, kernel driver can revert the reserve page 0 patch. > > Before that reserve page 0 can benefit all the Beignet user without breaking > anything. > > The point is that is becomes ABI. So no the kernel can't just revert it. > There is nothing special about address 0 in ether GTT or virtual memory. > If you require a special object allocated at address 0, allocate a special object > at address 0. > -Chris Hi, Zero page is not an ABI, It is only a strategy of virtual space allocation. Nobody would need exactly a page of virtual address 0. So there is no dependency. Zero page is a very common used method to avoid the NULL pointer issue. See http://en.wikipedia.org/wiki/Zero_page I can see a lot of issue to do that with MAP_FIXED zero page in user space. Say if a program combine libva, beignet and mesa or other graphics components together, which component should be responsible of allocating this zero page? What if a component happened to allocate a page in offset 0 before beignet and release it later? Using a zero page just remove the unnecessary mess and resolve problem in a clean explicit way. Thanks Zou Nanhai > -- > Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-09 12:02 ` [Intel-gfx] " Chris Wilson 2015-03-10 1:57 ` Zou, Nanhai @ 2015-03-13 9:10 ` David Weinehall 2015-03-13 9:18 ` [Intel-gfx] " Chris Wilson 2015-03-13 9:27 ` Daniel Vetter 1 sibling, 2 replies; 31+ messages in thread From: David Weinehall @ 2015-03-13 9:10 UTC (permalink / raw) To: Chris Wilson, Zou, Nanhai, Daniel Vetter, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet On 2015-03-09 14:02, Chris Wilson wrote: > On Mon, Mar 09, 2015 at 02:34:46AM +0000, Zou, Nanhai wrote: >> We don't need MAP_FIXED, we just want to avoid address 0 to be allocated. >> >> Though I think using MAP_FIXED is overkill, will bring much unnecessary complexity on both kernel and beignet side. >> I don't mind if people can provide stable MAP_FIXED patches to resolve this problem a few months or years later. >> >> At that time, kernel driver can revert the reserve page 0 patch. >> Before that reserve page 0 can benefit all the Beignet user without breaking anything. > > The point is that is becomes ABI. So no the kernel can't just revert it. > There is nothing special about address 0 in ether GTT or virtual memory. > If you require a special object allocated at address 0, allocate a > special object at address 0. I've explained the ABI issue in a separate e-mail discussion, and I believe that they now fully understand what you meant. That said, their main chain of reasoning makes some sense -- there is a race condition if we rely on using MAP_FIXED, at least on systems that do not support ppgtt. Ending up in a situation where opencl applications work on other hw, but fails when run on an i915-system would, at least in my opinion, not be ideal, no matter if it's due to an unfortunate design. *If* a MAP_FIXED solution is decided upon, how can userland be sure that the GTT page mapped to 0 is indeed usable as the NULL pointer? ON a PPGTT system that would be easy enough -- it's per process, so we'll be the only process allocating a page at 0, but if allocations use a global address space that won't be possible to guarantee. I realise that the first submitted patch didn't cover the GTT case, since the first indication I got was that not only was there a special case for GTT to need page 0 for other things, but also that this was "good enough" for opencl, but it seems that a full solution would be needed. Since this is a memory area we're talking about it's not uncommon to have the 0th page represent the NULL pointer and impossible for applications to reserve, so it would hardly be an unusual and inexplicable solution. All this said, how do "the other two" (NVidia, ATI) deal with this? Implicit NULL-page, explicit MAP_FIXED allocation, or something else? Kind regards, David --------------------------------------------------------------------- Intel Finland Oy Registered Address: PL 281, 00181 Helsinki Business Identity Code: 0357606 - 4 Domiciled in Helsinki This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-13 9:10 ` [Beignet] " David Weinehall @ 2015-03-13 9:18 ` Chris Wilson 2015-03-13 9:27 ` Daniel Vetter 1 sibling, 0 replies; 31+ messages in thread From: Chris Wilson @ 2015-03-13 9:18 UTC (permalink / raw) To: David Weinehall Cc: Daniel Vetter, Zou, Nanhai, beignet, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R On Fri, Mar 13, 2015 at 11:10:56AM +0200, David Weinehall wrote: > On 2015-03-09 14:02, Chris Wilson wrote: > >On Mon, Mar 09, 2015 at 02:34:46AM +0000, Zou, Nanhai wrote: > >>We don't need MAP_FIXED, we just want to avoid address 0 to be allocated. > >> > >>Though I think using MAP_FIXED is overkill, will bring much unnecessary complexity on both kernel and beignet side. > >>I don't mind if people can provide stable MAP_FIXED patches to resolve this problem a few months or years later. > >> > >>At that time, kernel driver can revert the reserve page 0 patch. > >>Before that reserve page 0 can benefit all the Beignet user without breaking anything. > > > >The point is that is becomes ABI. So no the kernel can't just revert it. > >There is nothing special about address 0 in ether GTT or virtual memory. > >If you require a special object allocated at address 0, allocate a > >special object at address 0. > > I've explained the ABI issue in a separate e-mail discussion, and I > believe that they now fully understand what you meant. > > That said, their main chain of reasoning makes some sense -- there is a > race condition if we rely on using MAP_FIXED, at least on systems that > do not support ppgtt. Ending up in a situation where opencl > applications work on other hw, but fails when run on an i915-system > would, at least in my opinion, not be ideal, no matter if it's due > to an unfortunate design. > > *If* a MAP_FIXED solution is decided upon, how can userland be sure > that the GTT page mapped to 0 is indeed usable as the NULL pointer? > ON a PPGTT system that would be easy enough -- it's per process, so > we'll be the only process allocating a page at 0, but if allocations > use a global address space that won't be possible to guarantee. It's simple, userspace has no control over allocation of the GGTT. Espcially something like address 0. They can make a request to have their object at a certain location, but the kernel is entirely likely to reject their request because it is pinned to hardware. > I realise that the first submitted patch didn't cover the GTT case, > since the first indication I got was that not only was there a > special case for GTT to need page 0 for other things, but also that > this was "good enough" for opencl, but it seems that a full solution > would be needed. Exactly. -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-13 9:10 ` [Beignet] " David Weinehall 2015-03-13 9:18 ` [Intel-gfx] " Chris Wilson @ 2015-03-13 9:27 ` Daniel Vetter 2015-03-13 16:58 ` [Beignet] " Chris Wilson 1 sibling, 1 reply; 31+ messages in thread From: Daniel Vetter @ 2015-03-13 9:27 UTC (permalink / raw) To: David Weinehall Cc: Daniel Vetter, Zou, Nanhai, beignet, Chris Wilson, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R On Fri, Mar 13, 2015 at 11:10:56AM +0200, David Weinehall wrote: > On 2015-03-09 14:02, Chris Wilson wrote: > >On Mon, Mar 09, 2015 at 02:34:46AM +0000, Zou, Nanhai wrote: > >>We don't need MAP_FIXED, we just want to avoid address 0 to be allocated. > >> > >>Though I think using MAP_FIXED is overkill, will bring much unnecessary complexity on both kernel and beignet side. > >>I don't mind if people can provide stable MAP_FIXED patches to resolve this problem a few months or years later. > >> > >>At that time, kernel driver can revert the reserve page 0 patch. > >>Before that reserve page 0 can benefit all the Beignet user without breaking anything. > > > >The point is that is becomes ABI. So no the kernel can't just revert it. > >There is nothing special about address 0 in ether GTT or virtual memory. > >If you require a special object allocated at address 0, allocate a > >special object at address 0. > > I've explained the ABI issue in a separate e-mail discussion, and I believe > that they now fully understand what you meant. > > That said, their main chain of reasoning makes some sense -- there is a > race condition if we rely on using MAP_FIXED, at least on systems that > do not support ppgtt. Ending up in a situation where opencl applications > work on other hw, but fails when run on an i915-system would, at least in my > opinion, not be ideal, no matter if it's due to an unfortunate design. > > *If* a MAP_FIXED solution is decided upon, how can userland be sure that the > GTT page mapped to 0 is indeed usable as the NULL pointer? > ON a PPGTT system that would be easy enough -- it's per process, so we'll be > the only process allocating a page at 0, but if allocations > use a global address space that won't be possible to guarantee. > > I realise that the first submitted patch didn't cover the GTT case, since > the first indication I got was that not only was there a special case for > GTT to need page 0 for other things, but also that this was "good enough" > for opencl, but it seems that a full solution would be needed. > > Since this is a memory area we're talking about it's not uncommon to have > the 0th page represent the NULL pointer and impossible for applications to > reserve, so it would hardly be an unusual and inexplicable solution. > > All this said, how do "the other two" (NVidia, ATI) deal with this? > Implicit NULL-page, explicit MAP_FIXED allocation, or something else? Afaik there's no opensource ocl implementation yet that needs this. So no idea what they do. If supporting systems without full ppgtt is a requirement for you (still wonky on gen8 a bit, so might be a good strategy) then imo it's the PIN_BIAS idea I've laid out earlier in this thread. That one will work everywhere. softpin can unexpectedly fail without full ppgtt if the kernel decides to put something at a given spot, which imo means we should only expose it on full ppgtt systems. And PIN_BIAS should be fairly easy to wire up since the internal logic is all there already. So "just" needs an execbuf flag, igt test and appropriate userspace to set that new bit. > Kind regards, David > --------------------------------------------------------------------- > Intel Finland Oy > Registered Address: PL 281, 00181 Helsinki Business Identity Code: 0357606 - > 4 Domiciled in Helsinki > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. You want to remove this when submitting to public mailing lists. Either apply for an exception or grab a 2nd mail address for mailing list communication. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-13 9:27 ` Daniel Vetter @ 2015-03-13 16:58 ` Chris Wilson 2015-03-13 17:13 ` [Intel-gfx] " Daniel Vetter 0 siblings, 1 reply; 31+ messages in thread From: Chris Wilson @ 2015-03-13 16:58 UTC (permalink / raw) To: Daniel Vetter Cc: Yang, Rong R, beignet, Song, Ruiling, David Weinehall, Vetter, Daniel, intel-gfx On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > If supporting systems without full ppgtt is a requirement for you (still > wonky on gen8 a bit, so might be a good strategy) then imo it's the > PIN_BIAS idea I've laid out earlier in this thread. That one will work > everywhere. softpin can unexpectedly fail without full ppgtt if the kernel > decides to put something at a given spot, which imo means we should only > expose it on full ppgtt systems. > > And PIN_BIAS should be fairly easy to wire up since the internal logic is > all there already. So "just" needs an execbuf flag, igt test and > appropriate userspace to set that new bit. It doesn't though. To provide the guarantee userspace is asking for (which is that address 0 goes to a special, preferrably inaccessible, page), you have to evict the first N pages in the GGTT. That is just as likely to fail with an execbuffer flag as it would with an execobject flag. -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-13 16:58 ` [Beignet] " Chris Wilson @ 2015-03-13 17:13 ` Daniel Vetter 2015-03-13 17:34 ` [Beignet] " Chris Wilson 2015-03-16 2:29 ` [Beignet] " Song, Ruiling 0 siblings, 2 replies; 31+ messages in thread From: Daniel Vetter @ 2015-03-13 17:13 UTC (permalink / raw) To: Chris Wilson, Daniel Vetter, David Weinehall, Zou, Nanhai, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > > If supporting systems without full ppgtt is a requirement for you (still > > wonky on gen8 a bit, so might be a good strategy) then imo it's the > > PIN_BIAS idea I've laid out earlier in this thread. That one will work > > everywhere. softpin can unexpectedly fail without full ppgtt if the kernel > > decides to put something at a given spot, which imo means we should only > > expose it on full ppgtt systems. > > > > And PIN_BIAS should be fairly easy to wire up since the internal logic is > > all there already. So "just" needs an execbuf flag, igt test and > > appropriate userspace to set that new bit. > > It doesn't though. To provide the guarantee userspace is asking for > (which is that address 0 goes to a special, preferrably inaccessible, > page), you have to evict the first N pages in the GGTT. That is just as > likely to fail with an execbuffer flag as it would with an execobject flag. Afaiui userspace only needs the guarantee that NULL is never a valid address. Which means it's never a valid address for its own buffer objects. I don't think it cares one bit what's actually there, it's not mandatory to fault apparently. And faulting is what's not possible. I guess the standard is like normal C: If you access a NULL pointer, anything can happen (including garbage on the frontbuffer), the only guarantee you need to make is that NULL is never a valid address. At least if no one plays tricks ;-) -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-13 17:13 ` [Intel-gfx] " Daniel Vetter @ 2015-03-13 17:34 ` Chris Wilson 2015-03-13 17:49 ` [Intel-gfx] " Daniel Vetter 2015-03-16 2:29 ` [Beignet] " Song, Ruiling 1 sibling, 1 reply; 31+ messages in thread From: Chris Wilson @ 2015-03-13 17:34 UTC (permalink / raw) To: Daniel Vetter Cc: Yang, Rong R, beignet, Song, Ruiling, David Weinehall, Vetter, Daniel, intel-gfx On Fri, Mar 13, 2015 at 06:13:39PM +0100, Daniel Vetter wrote: > On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > > On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > > > If supporting systems without full ppgtt is a requirement for you (still > > > wonky on gen8 a bit, so might be a good strategy) then imo it's the > > > PIN_BIAS idea I've laid out earlier in this thread. That one will work > > > everywhere. softpin can unexpectedly fail without full ppgtt if the kernel > > > decides to put something at a given spot, which imo means we should only > > > expose it on full ppgtt systems. > > > > > > And PIN_BIAS should be fairly easy to wire up since the internal logic is > > > all there already. So "just" needs an execbuf flag, igt test and > > > appropriate userspace to set that new bit. > > > > It doesn't though. To provide the guarantee userspace is asking for > > (which is that address 0 goes to a special, preferrably inaccessible, > > page), you have to evict the first N pages in the GGTT. That is just as > > likely to fail with an execbuffer flag as it would with an execobject flag. > > Afaiui userspace only needs the guarantee that NULL is never a valid > address. Which means it's never a valid address for its own buffer > objects. I don't think it cares one bit what's actually there, it's not > mandatory to fault apparently. And faulting is what's not possible. You are bending ABI to allow userspace to use absolute addressing (no relocations). The kernel has to make sure that nothing else is at that address. > I guess the standard is like normal C: If you access a NULL pointer, > anything can happen (including garbage on the frontbuffer), the only > guarantee you need to make is that NULL is never a valid address. At least > if no one plays tricks ;-) No. The kernel is quite strict about *NULL. It certainly doesn't allow trivial information leakage. -Chris -- Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-13 17:34 ` [Beignet] " Chris Wilson @ 2015-03-13 17:49 ` Daniel Vetter 0 siblings, 0 replies; 31+ messages in thread From: Daniel Vetter @ 2015-03-13 17:49 UTC (permalink / raw) To: Chris Wilson, Daniel Vetter, David Weinehall, Zou, Nanhai, Song, Ruiling, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet On Fri, Mar 13, 2015 at 05:34:22PM +0000, Chris Wilson wrote: > On Fri, Mar 13, 2015 at 06:13:39PM +0100, Daniel Vetter wrote: > > On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > > > On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > > > > If supporting systems without full ppgtt is a requirement for you (still > > > > wonky on gen8 a bit, so might be a good strategy) then imo it's the > > > > PIN_BIAS idea I've laid out earlier in this thread. That one will work > > > > everywhere. softpin can unexpectedly fail without full ppgtt if the kernel > > > > decides to put something at a given spot, which imo means we should only > > > > expose it on full ppgtt systems. > > > > > > > > And PIN_BIAS should be fairly easy to wire up since the internal logic is > > > > all there already. So "just" needs an execbuf flag, igt test and > > > > appropriate userspace to set that new bit. > > > > > > It doesn't though. To provide the guarantee userspace is asking for > > > (which is that address 0 goes to a special, preferrably inaccessible, > > > page), you have to evict the first N pages in the GGTT. That is just as > > > likely to fail with an execbuffer flag as it would with an execobject flag. > > > > Afaiui userspace only needs the guarantee that NULL is never a valid > > address. Which means it's never a valid address for its own buffer > > objects. I don't think it cares one bit what's actually there, it's not > > mandatory to fault apparently. And faulting is what's not possible. > > You are bending ABI to allow userspace to use absolute addressing (no > relocations). The kernel has to make sure that nothing else is at that > address. > > > I guess the standard is like normal C: If you access a NULL pointer, > > anything can happen (including garbage on the frontbuffer), the only > > guarantee you need to make is that NULL is never a valid address. At least > > if no one plays tricks ;-) > > No. The kernel is quite strict about *NULL. It certainly doesn't allow > trivial information leakage. We already have that trivial information leakage anyway. Just because we make it a notch more likely that a special address will leak information that doesn't make it different that without full ppgtt you can read any other gpu clients stuff easily. And also write to it if you feel like. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-13 17:13 ` [Intel-gfx] " Daniel Vetter 2015-03-13 17:34 ` [Beignet] " Chris Wilson @ 2015-03-16 2:29 ` Song, Ruiling 2015-03-16 8:52 ` Daniel Vetter 1 sibling, 1 reply; 31+ messages in thread From: Song, Ruiling @ 2015-03-16 2:29 UTC (permalink / raw) To: Daniel Vetter, Chris Wilson, Weinehall, David, Zou, Nanhai, Vetter, Daniel, intel-gfx, Yang, Rong R, beignet > -----Original Message----- > From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel > Vetter > Sent: Saturday, March 14, 2015 1:14 AM > To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling; > Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; > beignet@lists.freedesktop.org > Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address > allocation > > On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > > On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > > > If supporting systems without full ppgtt is a requirement for you > > > (still wonky on gen8 a bit, so might be a good strategy) then imo > > > it's the PIN_BIAS idea I've laid out earlier in this thread. That > > > one will work everywhere. softpin can unexpectedly fail without full > > > ppgtt if the kernel decides to put something at a given spot, which > > > imo means we should only expose it on full ppgtt systems. > > > > > > And PIN_BIAS should be fairly easy to wire up since the internal > > > logic is all there already. So "just" needs an execbuf flag, igt > > > test and appropriate userspace to set that new bit. > > > > It doesn't though. To provide the guarantee userspace is asking for > > (which is that address 0 goes to a special, preferrably inaccessible, > > page), you have to evict the first N pages in the GGTT. That is just > > as likely to fail with an execbuffer flag as it would with an execobject flag. > > Afaiui userspace only needs the guarantee that NULL is never a valid address. > Which means it's never a valid address for its own buffer objects. I don't > think it cares one bit what's actually there, it's not mandatory to fault > apparently. And faulting is what's not possible. Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language. But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension. Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation. Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address? So that PIN_BIAS is also effective on the shared buffer, right? Thanks! Ruiling > I guess the standard is like normal C: If you access a NULL pointer, anything > can happen (including garbage on the frontbuffer), the only guarantee you > need to make is that NULL is never a valid address. At least if no one plays > tricks ;-) -Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-16 2:29 ` [Beignet] " Song, Ruiling @ 2015-03-16 8:52 ` Daniel Vetter 2015-03-16 20:10 ` Jesse Barnes 2015-03-16 20:11 ` Jesse Barnes 0 siblings, 2 replies; 31+ messages in thread From: Daniel Vetter @ 2015-03-16 8:52 UTC (permalink / raw) To: Song, Ruiling Cc: Yang, Rong R, beignet, Weinehall, David, Vetter, Daniel, intel-gfx On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote: > > > > -----Original Message----- > > From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel > > Vetter > > Sent: Saturday, March 14, 2015 1:14 AM > > To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling; > > Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; > > beignet@lists.freedesktop.org > > Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address > > allocation > > > > On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > > > On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > > > > If supporting systems without full ppgtt is a requirement for you > > > > (still wonky on gen8 a bit, so might be a good strategy) then imo > > > > it's the PIN_BIAS idea I've laid out earlier in this thread. That > > > > one will work everywhere. softpin can unexpectedly fail without full > > > > ppgtt if the kernel decides to put something at a given spot, which > > > > imo means we should only expose it on full ppgtt systems. > > > > > > > > And PIN_BIAS should be fairly easy to wire up since the internal > > > > logic is all there already. So "just" needs an execbuf flag, igt > > > > test and appropriate userspace to set that new bit. > > > > > > It doesn't though. To provide the guarantee userspace is asking for > > > (which is that address 0 goes to a special, preferrably inaccessible, > > > page), you have to evict the first N pages in the GGTT. That is just > > > as likely to fail with an execbuffer flag as it would with an execobject flag. > > > > Afaiui userspace only needs the guarantee that NULL is never a valid address. > > Which means it's never a valid address for its own buffer objects. I don't > > think it cares one bit what's actually there, it's not mandatory to fault > > apparently. And faulting is what's not possible. > Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language. > But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension. > Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation. > Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address? > So that PIN_BIAS is also effective on the shared buffer, right? Yeah we'll rebind if needed. We can make this an execbuf or context flag, in either case anything that gets executed by ocl will be moved around if it accidentally ended up at the wrong place. The only exception is if a buffer is pinned already, i.e. if you're doing direct rendering to the frontbuffer. That will give you an EBUSY, but otoh that also shouldn't ever happen really. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-16 8:52 ` Daniel Vetter @ 2015-03-16 20:10 ` Jesse Barnes 2015-03-17 1:19 ` [Intel-gfx] " Zhigang Gong ` (2 more replies) 2015-03-16 20:11 ` Jesse Barnes 1 sibling, 3 replies; 31+ messages in thread From: Jesse Barnes @ 2015-03-16 20:10 UTC (permalink / raw) To: Daniel Vetter, Song, Ruiling Cc: Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David On 03/16/2015 01:52 AM, Daniel Vetter wrote: > On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote: >> >> >>> -----Original Message----- >>> From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel >>> Vetter >>> Sent: Saturday, March 14, 2015 1:14 AM >>> To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling; >>> Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; >>> beignet@lists.freedesktop.org >>> Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address >>> allocation >>> >>> On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: >>>> On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: >>>>> If supporting systems without full ppgtt is a requirement for you >>>>> (still wonky on gen8 a bit, so might be a good strategy) then imo >>>>> it's the PIN_BIAS idea I've laid out earlier in this thread. That >>>>> one will work everywhere. softpin can unexpectedly fail without full >>>>> ppgtt if the kernel decides to put something at a given spot, which >>>>> imo means we should only expose it on full ppgtt systems. >>>>> >>>>> And PIN_BIAS should be fairly easy to wire up since the internal >>>>> logic is all there already. So "just" needs an execbuf flag, igt >>>>> test and appropriate userspace to set that new bit. >>>> >>>> It doesn't though. To provide the guarantee userspace is asking for >>>> (which is that address 0 goes to a special, preferrably inaccessible, >>>> page), you have to evict the first N pages in the GGTT. That is just >>>> as likely to fail with an execbuffer flag as it would with an execobject flag. >>> >>> Afaiui userspace only needs the guarantee that NULL is never a valid address. >>> Which means it's never a valid address for its own buffer objects. I don't >>> think it cares one bit what's actually there, it's not mandatory to fault >>> apparently. And faulting is what's not possible. >> Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language. >> But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension. >> Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation. >> Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address? >> So that PIN_BIAS is also effective on the shared buffer, right? > > Yeah we'll rebind if needed. We can make this an execbuf or context flag, > in either case anything that gets executed by ocl will be moved around if > it accidentally ended up at the wrong place. The only exception is if a > buffer is pinned already, i.e. if you're doing direct rendering to the > frontbuffer. That will give you an EBUSY, but otoh that also shouldn't > ever happen really. Ruiling, are you working on this or someone from your team, presumably based on the patch Chris posted earlier? The zero page reservation certainly seems simpler to me, but the MAP_FIXED approach is a lot more flexible, and can be used for other types of debug and usages as well (we'll need something like it for OCL pointer sharing for example), so seems like a good thing to pursue regardless. Thanks, Jesse _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-16 20:10 ` Jesse Barnes @ 2015-03-17 1:19 ` Zhigang Gong 2015-03-17 2:29 ` Zou, Nanhai 2015-03-17 10:01 ` Daniel Vetter 2 siblings, 0 replies; 31+ messages in thread From: Zhigang Gong @ 2015-03-17 1:19 UTC (permalink / raw) To: Jesse Barnes Cc: Daniel Vetter, intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel, Yang, Rong R On Mon, Mar 16, 2015 at 01:10:28PM -0700, Jesse Barnes wrote: > On 03/16/2015 01:52 AM, Daniel Vetter wrote: > > On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote: > >> > >> > >>> -----Original Message----- > >>> From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel > >>> Vetter > >>> Sent: Saturday, March 14, 2015 1:14 AM > >>> To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling; > >>> Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; > >>> beignet@lists.freedesktop.org > >>> Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address > >>> allocation > >>> > >>> On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > >>>> On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > >>>>> If supporting systems without full ppgtt is a requirement for you > >>>>> (still wonky on gen8 a bit, so might be a good strategy) then imo > >>>>> it's the PIN_BIAS idea I've laid out earlier in this thread. That > >>>>> one will work everywhere. softpin can unexpectedly fail without full > >>>>> ppgtt if the kernel decides to put something at a given spot, which > >>>>> imo means we should only expose it on full ppgtt systems. > >>>>> > >>>>> And PIN_BIAS should be fairly easy to wire up since the internal > >>>>> logic is all there already. So "just" needs an execbuf flag, igt > >>>>> test and appropriate userspace to set that new bit. > >>>> > >>>> It doesn't though. To provide the guarantee userspace is asking for > >>>> (which is that address 0 goes to a special, preferrably inaccessible, > >>>> page), you have to evict the first N pages in the GGTT. That is just > >>>> as likely to fail with an execbuffer flag as it would with an execobject flag. > >>> > >>> Afaiui userspace only needs the guarantee that NULL is never a valid address. > >>> Which means it's never a valid address for its own buffer objects. I don't > >>> think it cares one bit what's actually there, it's not mandatory to fault > >>> apparently. And faulting is what's not possible. > >> Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language. > >> But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension. > >> Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation. > >> Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address? > >> So that PIN_BIAS is also effective on the shared buffer, right? > > > > Yeah we'll rebind if needed. We can make this an execbuf or context flag, > > in either case anything that gets executed by ocl will be moved around if > > it accidentally ended up at the wrong place. The only exception is if a > > buffer is pinned already, i.e. if you're doing direct rendering to the > > frontbuffer. That will give you an EBUSY, but otoh that also shouldn't > > ever happen really. > > Ruiling, are you working on this or someone from your team, presumably > based on the patch Chris posted earlier? The zero page reservation > certainly seems simpler to me, but the MAP_FIXED approach is a lot more > flexible, and can be used for other types of debug and usages as well > (we'll need something like it for OCL pointer sharing for example), so > seems like a good thing to pursue regardless. Jesse, here is my 2 cent comment: IMO, the start of this discussion is from a very simple issue: 1. Make sure any valid buffer object will not be mapped to zero page. OCL wants to avoid 0 virtual address for any valid buffer object. Thus the following type of code will work as expected. __kernel foo(__global int *src, __global int* dst) { if (dst == NULL) return; } Although the above type code may not make much sense, but it should be supported as well. This is a simple requirement for all the bo's binding operations from CL application. And we think that simply skip the zero-page in KMD is a good enough solution and don't see any side-effect till now. Then let's talk about the MAP_FIXED approach: 2. About the MAP_FIXED approach. If my understanding is correct, this is a intermediate solution towards FULL SVM function. From OCL's view, it is usefull for the 2.0's. There are three types of SVM options. Even for the most basic "Coarse-Grained buffer SVM", something like MAP_FIXED is useful. Becuase it needs to pass a linked list object to the OCL kernel directly. If we can use MAP_FIXED to map the bo to the VM address which is used by host, then we can access such a linked list easily on OCL kernel side, otherwise, we may need some tricks to do extra address adjustment. So this feature is useful for Beignet before we get full SVM support. My conclusion is: * The 1st is a totally *passive* requirement and need to be applied to each buffer binding. And it's better to be transparent to userspace. * The 2nd is an *active* requirement and need the kernel to provide SVM like interface to the user space. And only those SVM buffers (for the first 2 OCL SVM options) need to call the new interfaces explicitly. Thanks, Zhigang Gong. > > Thanks, > Jesse > > _______________________________________________ > Beignet mailing list > Beignet@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/beignet _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-16 20:10 ` Jesse Barnes 2015-03-17 1:19 ` [Intel-gfx] " Zhigang Gong @ 2015-03-17 2:29 ` Zou, Nanhai 2015-03-17 15:13 ` [Beignet] " Jesse Barnes 2015-03-17 10:01 ` Daniel Vetter 2 siblings, 1 reply; 31+ messages in thread From: Zou, Nanhai @ 2015-03-17 2:29 UTC (permalink / raw) To: Jesse Barnes, Daniel Vetter, Song, Ruiling Cc: Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David > -----Original Message----- > From: Intel-gfx [mailto:intel-gfx-bounces@lists.freedesktop.org] On Behalf Of > Jesse Barnes > Sent: Tuesday, March 17, 2015 4:10 AM > To: Daniel Vetter; Song, Ruiling > Cc: Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; > beignet@lists.freedesktop.org; Weinehall, David > Subject: Re: [Intel-gfx] [Beignet] Preventing zero GPU virtual address allocation > > On 03/16/2015 01:52 AM, Daniel Vetter wrote: > > On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote: > >> > >> > >>> -----Original Message----- > >>> From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of > >>> Daniel Vetter > >>> Sent: Saturday, March 14, 2015 1:14 AM > >>> To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; > >>> Song, Ruiling; Vetter, Daniel; intel-gfx@lists.freedesktop.org; > >>> Yang, Rong R; beignet@lists.freedesktop.org > >>> Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual > >>> address allocation > >>> > >>> On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > >>>> On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > >>>>> If supporting systems without full ppgtt is a requirement for you > >>>>> (still wonky on gen8 a bit, so might be a good strategy) then imo > >>>>> it's the PIN_BIAS idea I've laid out earlier in this thread. That > >>>>> one will work everywhere. softpin can unexpectedly fail without > >>>>> full ppgtt if the kernel decides to put something at a given spot, > >>>>> which imo means we should only expose it on full ppgtt systems. > >>>>> > >>>>> And PIN_BIAS should be fairly easy to wire up since the internal > >>>>> logic is all there already. So "just" needs an execbuf flag, igt > >>>>> test and appropriate userspace to set that new bit. > >>>> > >>>> It doesn't though. To provide the guarantee userspace is asking for > >>>> (which is that address 0 goes to a special, preferrably > >>>> inaccessible, page), you have to evict the first N pages in the > >>>> GGTT. That is just as likely to fail with an execbuffer flag as it would with > an execobject flag. > >>> > >>> Afaiui userspace only needs the guarantee that NULL is never a valid > address. > >>> Which means it's never a valid address for its own buffer objects. I > >>> don't think it cares one bit what's actually there, it's not > >>> mandatory to fault apparently. And faulting is what's not possible. > >> Yes, This is what exactly what we need, that is make NULL as an invalid > address. It's just like C language. > >> But I have some more comment. The buffer object used in opencl may be > allocated in libva/opengl and shared for opencl usage through some opencl > extension. > >> Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer > object allocation. > >> Will libdrm re-bind such kind of shared buffer object to a new graphics virtual > address? > >> So that PIN_BIAS is also effective on the shared buffer, right? > > > > Yeah we'll rebind if needed. We can make this an execbuf or context > > flag, in either case anything that gets executed by ocl will be moved > > around if it accidentally ended up at the wrong place. The only > > exception is if a buffer is pinned already, i.e. if you're doing > > direct rendering to the frontbuffer. That will give you an EBUSY, but > > otoh that also shouldn't ever happen really. > > Ruiling, are you working on this or someone from your team, presumably based > on the patch Chris posted earlier? The zero page reservation certainly seems > simpler to me, but the MAP_FIXED approach is a lot more flexible, and can be > used for other types of debug and usages as well (we'll need something like it > for OCL pointer sharing for example), so seems like a good thing to pursue > regardless. > Hi Jesse, MAP_FIXED cannot solve this issue. You may see my previous comments for this topic, There could be many components in on single process, Beignet cannot be guaranteed to be the first one who has allocated address 0. Thanks Zou Nanhai > Thanks, > Jesse > > _______________________________________________ > Intel-gfx mailing list > Intel-gfx@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/intel-gfx _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-17 2:29 ` Zou, Nanhai @ 2015-03-17 15:13 ` Jesse Barnes 2015-03-19 3:22 ` [Intel-gfx] " Song, Ruiling 0 siblings, 1 reply; 31+ messages in thread From: Jesse Barnes @ 2015-03-17 15:13 UTC (permalink / raw) To: Zou, Nanhai, Daniel Vetter, Song, Ruiling Cc: Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David >>> Yeah we'll rebind if needed. We can make this an execbuf or context >>> flag, in either case anything that gets executed by ocl will be moved >>> around if it accidentally ended up at the wrong place. The only >>> exception is if a buffer is pinned already, i.e. if you're doing >>> direct rendering to the frontbuffer. That will give you an EBUSY, but >>> otoh that also shouldn't ever happen really. >> >> Ruiling, are you working on this or someone from your team, presumably based >> on the patch Chris posted earlier? The zero page reservation certainly seems >> simpler to me, but the MAP_FIXED approach is a lot more flexible, and can be >> used for other types of debug and usages as well (we'll need something like it >> for OCL pointer sharing for example), so seems like a good thing to pursue >> regardless. >> > Hi Jesse, > MAP_FIXED cannot solve this issue. You may see my previous comments for this topic, > There could be many components in on single process, Beignet cannot be guaranteed to be the first one who has allocated address 0. Yeah, MAP_FIXED sounds a bit more ambitious and though I think it would work for OCL 2.0 pointer sharing, it's a little different than we were planning. To summarize, we have three possible approaches, each with its own problems: 1) simple patch to avoid binding at address 0 in PPGTT: does impact the ABI (though generally not in a harmful way), and may not be possible with aliasing PPGTT with e.g. framebuffers bound at offset 0 2) exposing PIN_BIAS to userspace Would allow userspace to avoid pinning any buffers at offset 0 at execbuf time, but still has the problem with previously bound buffers and aliasing PPGTT 3) MAP_FIXED interface Flexible approach allowing userspace to manage its own virtual memory, but still has the same issues with aliasing PPGTT, and with shared contexts, which would have to negotiate between libraries how to handle the zero page For (1) and (2) the kernel pieces are really already in place, the main thing we need is a new flag to userspace to indicate behavior. I'd prefer (1) with a context creation flag to indicate "don't bind at 0". Execbuf would try to honor this, and userspace could check if any buffers ended up at 0 in the aliasing PPGTT case by checking the resulting offsets following the call. I expect in most cases this would be fine. It should be pretty easy to extend Ruiling's patch to use a context flag to determine the behavior; is that something you can do? Any objections to this approach? It does mean that shared contexts need to be handled specially, or won't get the 0 page protection, but I think Mesa wants this behavior too, and libva probably wouldn't mind, so you could just require new versions of those that set this flag when telling people what's supported for proper NULL pointer handling. Any objections to that approach? Thanks, Jesse _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Intel-gfx] Preventing zero GPU virtual address allocation 2015-03-17 15:13 ` [Beignet] " Jesse Barnes @ 2015-03-19 3:22 ` Song, Ruiling 2015-03-19 10:09 ` [Beignet] " David Weinehall 0 siblings, 1 reply; 31+ messages in thread From: Song, Ruiling @ 2015-03-19 3:22 UTC (permalink / raw) To: Jesse Barnes, Weinehall, David Cc: Yang, Rong R, intel-gfx, beignet, Daniel Vetter, Vetter, Daniel, Zou, Nanhai > Yeah, MAP_FIXED sounds a bit more ambitious and though I think it would > work for OCL 2.0 pointer sharing, it's a little different than we were planning. > To summarize, we have three possible approaches, each with its own > problems: > 1) simple patch to avoid binding at address 0 in PPGTT: > does impact the ABI (though generally not in a harmful way), and > may not be possible with aliasing PPGTT with e.g. framebuffers > bound at offset 0 > 2) exposing PIN_BIAS to userspace > Would allow userspace to avoid pinning any buffers at offset 0 at > execbuf time, but still has the problem with previously bound buffers > and aliasing PPGTT > 3) MAP_FIXED interface > Flexible approach allowing userspace to manage its own virtual > memory, but still has the same issues with aliasing PPGTT, and with > shared contexts, which would have to negotiate between libraries > how to > handle the zero page > > For (1) and (2) the kernel pieces are really already in place, the main thing we > need is a new flag to userspace to indicate behavior. I'd prefer (1) with a > context creation flag to indicate "don't bind at 0". > Execbuf would try to honor this, and userspace could check if any buffers > ended up at 0 in the aliasing PPGTT case by checking the resulting offsets > following the call. I expect in most cases this would be fine. > > It should be pretty easy to extend Ruiling's patch to use a context flag to > determine the behavior; is that something you can do? Any objections to > this approach? I am ok with adding a context flag to indicate "don't bind at 0". Any objections from others? The patch is not from me, it is from David. I am not familiar with KMD. David, could you help on this patch? > It does mean that shared contexts need to be handled specially, or won't get > the 0 page protection, but I think Mesa wants this behavior too, and libva > probably wouldn't mind, so you could just require new versions of those that > set this flag when telling people what's supported for proper NULL pointer > handling. > > Any objections to that approach? > > Thanks, > Jesse _______________________________________________ Beignet mailing list Beignet@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/beignet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-19 3:22 ` [Intel-gfx] " Song, Ruiling @ 2015-03-19 10:09 ` David Weinehall 2015-03-19 14:58 ` Daniel Vetter 0 siblings, 1 reply; 31+ messages in thread From: David Weinehall @ 2015-03-19 10:09 UTC (permalink / raw) To: Song, Ruiling Cc: Yang, Rong R, intel-gfx, beignet, Weinehall, David, Vetter, Daniel On Thu, Mar 19, 2015 at 03:22:42AM +0000, Song, Ruiling wrote: > > > Yeah, MAP_FIXED sounds a bit more ambitious and though I think it would > > work for OCL 2.0 pointer sharing, it's a little different than we were planning. > > To summarize, we have three possible approaches, each with its own > > problems: > > 1) simple patch to avoid binding at address 0 in PPGTT: > > does impact the ABI (though generally not in a harmful way), and > > may not be possible with aliasing PPGTT with e.g. framebuffers > > bound at offset 0 > > 2) exposing PIN_BIAS to userspace > > Would allow userspace to avoid pinning any buffers at offset 0 at > > execbuf time, but still has the problem with previously bound buffers > > and aliasing PPGTT > > 3) MAP_FIXED interface > > Flexible approach allowing userspace to manage its own virtual > > memory, but still has the same issues with aliasing PPGTT, and with > > shared contexts, which would have to negotiate between libraries > > how to > > handle the zero page > > > > For (1) and (2) the kernel pieces are really already in place, the main thing we > > need is a new flag to userspace to indicate behavior. I'd prefer (1) with a > > context creation flag to indicate "don't bind at 0". > > Execbuf would try to honor this, and userspace could check if any buffers > > ended up at 0 in the aliasing PPGTT case by checking the resulting offsets > > following the call. I expect in most cases this would be fine. > > > > It should be pretty easy to extend Ruiling's patch to use a context flag to > > determine the behavior; is that something you can do? Any objections to > > this approach? > > I am ok with adding a context flag to indicate "don't bind at 0". Any objections from others? > The patch is not from me, it is from David. I am not familiar with KMD. David, could you help on this patch? Yup, assuming, of course, that such an approach is acceptable. Kind regards, David _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-19 10:09 ` [Beignet] " David Weinehall @ 2015-03-19 14:58 ` Daniel Vetter 2015-03-20 3:01 ` Song, Ruiling 0 siblings, 1 reply; 31+ messages in thread From: Daniel Vetter @ 2015-03-19 14:58 UTC (permalink / raw) To: Song, Ruiling, Jesse Barnes, Weinehall, David, Yang, Rong R, intel-gfx, beignet, Vetter, Daniel On Thu, Mar 19, 2015 at 12:09:53PM +0200, David Weinehall wrote: > On Thu, Mar 19, 2015 at 03:22:42AM +0000, Song, Ruiling wrote: > > > > > Yeah, MAP_FIXED sounds a bit more ambitious and though I think it would > > > work for OCL 2.0 pointer sharing, it's a little different than we were planning. > > > To summarize, we have three possible approaches, each with its own > > > problems: > > > 1) simple patch to avoid binding at address 0 in PPGTT: > > > does impact the ABI (though generally not in a harmful way), and > > > may not be possible with aliasing PPGTT with e.g. framebuffers > > > bound at offset 0 > > > 2) exposing PIN_BIAS to userspace > > > Would allow userspace to avoid pinning any buffers at offset 0 at > > > execbuf time, but still has the problem with previously bound buffers > > > and aliasing PPGTT > > > 3) MAP_FIXED interface > > > Flexible approach allowing userspace to manage its own virtual > > > memory, but still has the same issues with aliasing PPGTT, and with > > > shared contexts, which would have to negotiate between libraries > > > how to > > > handle the zero page > > > > > > For (1) and (2) the kernel pieces are really already in place, the main thing we > > > need is a new flag to userspace to indicate behavior. I'd prefer (1) with a > > > context creation flag to indicate "don't bind at 0". > > > Execbuf would try to honor this, and userspace could check if any buffers > > > ended up at 0 in the aliasing PPGTT case by checking the resulting offsets > > > following the call. I expect in most cases this would be fine. > > > > > > It should be pretty easy to extend Ruiling's patch to use a context flag to > > > determine the behavior; is that something you can do? Any objections to > > > this approach? > > > > I am ok with adding a context flag to indicate "don't bind at 0". Any objections from others? > > The patch is not from me, it is from David. I am not familiar with KMD. David, could you help on this patch? > > Yup, assuming, of course, that such an approach is acceptable. Yeah my big concern was with not making this opt-in like the old patch or adding an interface which does a lot more than what we need right now (Chris' patch). Just a bitflag to ask for this seems best and is fine with me. And for the implementation I think we should reuse the PIN_BIAS logic since that'll work in all places where it's possible. One open from my side is how we should handle failures to move buffers (in case they ended up at 0 somehow) - we can either silently fail or return an error to userspace. Note that this is only possible if you render to an elg image from ocl, and if that egl image is a pinned frontbuffer and if we don't have full ppgtt support. I don't know what the spec requires us to do here, or whether we should care at all. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-19 14:58 ` Daniel Vetter @ 2015-03-20 3:01 ` Song, Ruiling 0 siblings, 0 replies; 31+ messages in thread From: Song, Ruiling @ 2015-03-20 3:01 UTC (permalink / raw) To: Daniel Vetter, Jesse Barnes, Weinehall, David, Yang, Rong R, intel-gfx, beignet, Vetter, Daniel > Yeah my big concern was with not making this opt-in like the old patch or > adding an interface which does a lot more than what we need right now > (Chris' patch). Just a bitflag to ask for this seems best and is fine with me. > > And for the implementation I think we should reuse the PIN_BIAS logic since > that'll work in all places where it's possible. One open from my side is how > we should handle failures to move buffers (in case they ended up at 0 > somehow) - we can either silently fail or return an error to userspace. > > Note that this is only possible if you render to an elg image from ocl, and if > that egl image is a pinned frontbuffer and if we don't have full ppgtt support. > I don't know what the spec requires us to do here, or whether we should > care at all. So the situation you mentioned only comes up when a pinned buffer under global gtt? Under global gtt, a buffer would rarely binded at offset 0, in fact the most often cases are under ppgtt. So, I think silent ignore moving pinned buffer under gtt is acceptable. I will add the check for the binded at zero case in beignet. Ruiling > -Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch > _______________________________________________ > Beignet mailing list > Beignet@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/beignet _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-16 20:10 ` Jesse Barnes 2015-03-17 1:19 ` [Intel-gfx] " Zhigang Gong 2015-03-17 2:29 ` Zou, Nanhai @ 2015-03-17 10:01 ` Daniel Vetter 2 siblings, 0 replies; 31+ messages in thread From: Daniel Vetter @ 2015-03-17 10:01 UTC (permalink / raw) To: Jesse Barnes Cc: intel-gfx, beignet, Song, Ruiling, Weinehall, David, Vetter, Daniel, Yang, Rong R On Mon, Mar 16, 2015 at 01:10:28PM -0700, Jesse Barnes wrote: > On 03/16/2015 01:52 AM, Daniel Vetter wrote: > > On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote: > >> > >> > >>> -----Original Message----- > >>> From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel > >>> Vetter > >>> Sent: Saturday, March 14, 2015 1:14 AM > >>> To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling; > >>> Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; > >>> beignet@lists.freedesktop.org > >>> Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address > >>> allocation > >>> > >>> On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: > >>>> On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: > >>>>> If supporting systems without full ppgtt is a requirement for you > >>>>> (still wonky on gen8 a bit, so might be a good strategy) then imo > >>>>> it's the PIN_BIAS idea I've laid out earlier in this thread. That > >>>>> one will work everywhere. softpin can unexpectedly fail without full > >>>>> ppgtt if the kernel decides to put something at a given spot, which > >>>>> imo means we should only expose it on full ppgtt systems. > >>>>> > >>>>> And PIN_BIAS should be fairly easy to wire up since the internal > >>>>> logic is all there already. So "just" needs an execbuf flag, igt > >>>>> test and appropriate userspace to set that new bit. > >>>> > >>>> It doesn't though. To provide the guarantee userspace is asking for > >>>> (which is that address 0 goes to a special, preferrably inaccessible, > >>>> page), you have to evict the first N pages in the GGTT. That is just > >>>> as likely to fail with an execbuffer flag as it would with an execobject flag. > >>> > >>> Afaiui userspace only needs the guarantee that NULL is never a valid address. > >>> Which means it's never a valid address for its own buffer objects. I don't > >>> think it cares one bit what's actually there, it's not mandatory to fault > >>> apparently. And faulting is what's not possible. > >> Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language. > >> But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension. > >> Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation. > >> Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address? > >> So that PIN_BIAS is also effective on the shared buffer, right? > > > > Yeah we'll rebind if needed. We can make this an execbuf or context flag, > > in either case anything that gets executed by ocl will be moved around if > > it accidentally ended up at the wrong place. The only exception is if a > > buffer is pinned already, i.e. if you're doing direct rendering to the > > frontbuffer. That will give you an EBUSY, but otoh that also shouldn't > > ever happen really. > > Ruiling, are you working on this or someone from your team, presumably > based on the patch Chris posted earlier? The zero page reservation > certainly seems simpler to me, but the MAP_FIXED approach is a lot more > flexible, and can be used for other types of debug and usages as well > (we'll need something like it for OCL pointer sharing for example), so > seems like a good thing to pursue regardless. I prefer not to merge fix MAP_FIXED with the justification that ocl will need the full power of it for buffered svm, without the pieces really being ready. And I don't think we should block this little fix for current ocl on the svm work either. Hence why I suggested to just expose the already existing PIN_BIAS support somehow. That also has the upside of working without full ppgtt (i.e. on hsw and earlier). -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Beignet] Preventing zero GPU virtual address allocation 2015-03-16 8:52 ` Daniel Vetter 2015-03-16 20:10 ` Jesse Barnes @ 2015-03-16 20:11 ` Jesse Barnes 1 sibling, 0 replies; 31+ messages in thread From: Jesse Barnes @ 2015-03-16 20:11 UTC (permalink / raw) To: Daniel Vetter, Song, Ruiling Cc: Vetter, Daniel, intel-gfx, Yang, Rong R, beignet, Weinehall, David On 03/16/2015 01:52 AM, Daniel Vetter wrote: > On Mon, Mar 16, 2015 at 02:29:24AM +0000, Song, Ruiling wrote: >> >> >>> -----Original Message----- >>> From: Daniel Vetter [mailto:daniel.vetter@ffwll.ch] On Behalf Of Daniel >>> Vetter >>> Sent: Saturday, March 14, 2015 1:14 AM >>> To: Chris Wilson; Daniel Vetter; Weinehall, David; Zou, Nanhai; Song, Ruiling; >>> Vetter, Daniel; intel-gfx@lists.freedesktop.org; Yang, Rong R; >>> beignet@lists.freedesktop.org >>> Subject: Re: [Beignet] [Intel-gfx] Preventing zero GPU virtual address >>> allocation >>> >>> On Fri, Mar 13, 2015 at 04:58:47PM +0000, Chris Wilson wrote: >>>> On Fri, Mar 13, 2015 at 10:27:38AM +0100, Daniel Vetter wrote: >>>>> If supporting systems without full ppgtt is a requirement for you >>>>> (still wonky on gen8 a bit, so might be a good strategy) then imo >>>>> it's the PIN_BIAS idea I've laid out earlier in this thread. That >>>>> one will work everywhere. softpin can unexpectedly fail without full >>>>> ppgtt if the kernel decides to put something at a given spot, which >>>>> imo means we should only expose it on full ppgtt systems. >>>>> >>>>> And PIN_BIAS should be fairly easy to wire up since the internal >>>>> logic is all there already. So "just" needs an execbuf flag, igt >>>>> test and appropriate userspace to set that new bit. >>>> >>>> It doesn't though. To provide the guarantee userspace is asking for >>>> (which is that address 0 goes to a special, preferrably inaccessible, >>>> page), you have to evict the first N pages in the GGTT. That is just >>>> as likely to fail with an execbuffer flag as it would with an execobject flag. >>> >>> Afaiui userspace only needs the guarantee that NULL is never a valid address. >>> Which means it's never a valid address for its own buffer objects. I don't >>> think it cares one bit what's actually there, it's not mandatory to fault >>> apparently. And faulting is what's not possible. >> Yes, This is what exactly what we need, that is make NULL as an invalid address. It's just like C language. >> But I have some more comment. The buffer object used in opencl may be allocated in libva/opengl and shared for opencl usage through some opencl extension. >> Afaiui, this would implicitly require libva/mesa also avoid zero-address buffer object allocation. >> Will libdrm re-bind such kind of shared buffer object to a new graphics virtual address? >> So that PIN_BIAS is also effective on the shared buffer, right? > > Yeah we'll rebind if needed. We can make this an execbuf or context flag, > in either case anything that gets executed by ocl will be moved around if > it accidentally ended up at the wrong place. The only exception is if a > buffer is pinned already, i.e. if you're doing direct rendering to the > frontbuffer. That will give you an EBUSY, but otoh that also shouldn't > ever happen really. Oh and we may need to re-introduce the ppgtt zero page reservation anyway due to bugs, but that's another topic... _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx ^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2015-03-20 3:01 UTC | newest] Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-03-05 2:56 Preventing zero GPU virtual address allocation Song, Ruiling 2015-03-05 12:52 ` [Intel-gfx] " Daniel Vetter 2015-03-05 13:01 ` Chris Wilson 2015-03-05 15:27 ` [Intel-gfx] " Daniel Vetter 2015-03-05 21:07 ` Chris Wilson 2015-03-09 15:46 ` Jesse Barnes 2015-03-09 15:49 ` Jesse Barnes 2015-03-06 2:11 ` [Intel-gfx] " Zou, Nanhai 2015-03-06 8:39 ` [Beignet] " Chris Wilson 2015-03-09 2:34 ` Zou, Nanhai 2015-03-09 12:02 ` [Intel-gfx] " Chris Wilson 2015-03-10 1:57 ` Zou, Nanhai 2015-03-13 9:10 ` [Beignet] " David Weinehall 2015-03-13 9:18 ` [Intel-gfx] " Chris Wilson 2015-03-13 9:27 ` Daniel Vetter 2015-03-13 16:58 ` [Beignet] " Chris Wilson 2015-03-13 17:13 ` [Intel-gfx] " Daniel Vetter 2015-03-13 17:34 ` [Beignet] " Chris Wilson 2015-03-13 17:49 ` [Intel-gfx] " Daniel Vetter 2015-03-16 2:29 ` [Beignet] " Song, Ruiling 2015-03-16 8:52 ` Daniel Vetter 2015-03-16 20:10 ` Jesse Barnes 2015-03-17 1:19 ` [Intel-gfx] " Zhigang Gong 2015-03-17 2:29 ` Zou, Nanhai 2015-03-17 15:13 ` [Beignet] " Jesse Barnes 2015-03-19 3:22 ` [Intel-gfx] " Song, Ruiling 2015-03-19 10:09 ` [Beignet] " David Weinehall 2015-03-19 14:58 ` Daniel Vetter 2015-03-20 3:01 ` Song, Ruiling 2015-03-17 10:01 ` Daniel Vetter 2015-03-16 20:11 ` Jesse Barnes
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.