* [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails @ 2017-06-06 7:17 Lucas Stach 2017-06-06 7:17 ` [PATCH 2/3] drm/etnaviv: reduce allocation failure message severity Lucas Stach ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Lucas Stach @ 2017-06-06 7:17 UTC (permalink / raw) To: Lucas Stach; +Cc: etnaviv, dri-devel, Russell King GPU buffers can be quite large, so userspace is expected to deal with allocation failure. Don't trigger the OOM killer when page allocation for the GEM objects fails, as this opens an easy possiblity for unprivileged applications to DOS the system,a s the shmem pages are not fully accounted to the allocating process. Signed-off-by: Lucas Stach <dev@lynxeye.de> --- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c index f0efc5d..4c53508 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c @@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, * going to pin these pages. */ mapping = obj->filp->f_mapping; - mapping_set_gfp_mask(mapping, GFP_HIGHUSER); + mapping_set_gfp_mask(mapping, GFP_HIGHUSER | + __GFP_NORETRY | __GFP_NOWARN); } if (ret) -- 2.9.4 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/3] drm/etnaviv: reduce allocation failure message severity 2017-06-06 7:17 [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Lucas Stach @ 2017-06-06 7:17 ` Lucas Stach 2017-06-06 7:17 ` [PATCH 3/3] drm/etnaviv: populate GEM objects on cpu_prep Lucas Stach 2017-06-20 9:06 ` [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Daniel Vetter 2 siblings, 0 replies; 8+ messages in thread From: Lucas Stach @ 2017-06-06 7:17 UTC (permalink / raw) To: Lucas Stach; +Cc: etnaviv, dri-devel, Russell King The GPU userspace is expected to deal with failure to allocate memory for the GPU buffers, there is no need to spam the log on failure. Signed-off-by: Lucas Stach <dev@lynxeye.de> --- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c index 4c53508..98243ba 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c @@ -68,7 +68,7 @@ static int etnaviv_gem_shmem_get_pages(struct etnaviv_gem_object *etnaviv_obj) struct page **p = drm_gem_get_pages(&etnaviv_obj->base); if (IS_ERR(p)) { - dev_err(dev->dev, "could not get pages: %ld\n", PTR_ERR(p)); + dev_dbg(dev->dev, "could not get pages: %ld\n", PTR_ERR(p)); return PTR_ERR(p); } -- 2.9.4 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/3] drm/etnaviv: populate GEM objects on cpu_prep 2017-06-06 7:17 [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Lucas Stach 2017-06-06 7:17 ` [PATCH 2/3] drm/etnaviv: reduce allocation failure message severity Lucas Stach @ 2017-06-06 7:17 ` Lucas Stach 2017-06-20 9:06 ` [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Daniel Vetter 2 siblings, 0 replies; 8+ messages in thread From: Lucas Stach @ 2017-06-06 7:17 UTC (permalink / raw) To: Lucas Stach; +Cc: etnaviv, dri-devel, Russell King CPU prep is the point where we can reasonably return an error to userspace when something goes wrong while populating the object. If we leave the object unpopulated at this point, the allocation will happen in the fault handler when userspace accesses the object through the mmap space, where we don't have any other option than to OOM the system. Signed-off-by: Lucas Stach <dev@lynxeye.de> --- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c index 98243ba..f6ac3f7 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c @@ -413,6 +413,16 @@ int etnaviv_gem_cpu_prep(struct drm_gem_object *obj, u32 op, bool write = !!(op & ETNA_PREP_WRITE); int ret; + if (!etnaviv_obj->sgt) { + void *ret; + + mutex_lock(&etnaviv_obj->lock); + ret = etnaviv_gem_get_pages(etnaviv_obj); + mutex_unlock(&etnaviv_obj->lock); + if (IS_ERR(ret)) + return PTR_ERR(ret); + } + if (op & ETNA_PREP_NOSYNC) { if (!reservation_object_test_signaled_rcu(etnaviv_obj->resv, write)) @@ -427,16 +437,6 @@ int etnaviv_gem_cpu_prep(struct drm_gem_object *obj, u32 op, } if (etnaviv_obj->flags & ETNA_BO_CACHED) { - if (!etnaviv_obj->sgt) { - void *ret; - - mutex_lock(&etnaviv_obj->lock); - ret = etnaviv_gem_get_pages(etnaviv_obj); - mutex_unlock(&etnaviv_obj->lock); - if (IS_ERR(ret)) - return PTR_ERR(ret); - } - dma_sync_sg_for_cpu(dev->dev, etnaviv_obj->sgt->sgl, etnaviv_obj->sgt->nents, etnaviv_op_to_dma_dir(op)); -- 2.9.4 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails 2017-06-06 7:17 [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Lucas Stach 2017-06-06 7:17 ` [PATCH 2/3] drm/etnaviv: reduce allocation failure message severity Lucas Stach 2017-06-06 7:17 ` [PATCH 3/3] drm/etnaviv: populate GEM objects on cpu_prep Lucas Stach @ 2017-06-20 9:06 ` Daniel Vetter 2017-06-20 9:22 ` Lucas Stach 2 siblings, 1 reply; 8+ messages in thread From: Daniel Vetter @ 2017-06-20 9:06 UTC (permalink / raw) To: Lucas Stach; +Cc: etnaviv, dri-devel, mhocko, Russell King On Tue, Jun 06, 2017 at 09:17:06AM +0200, Lucas Stach wrote: > GPU buffers can be quite large, so userspace is expected to deal with > allocation failure. Don't trigger the OOM killer when page allocation for > the GEM objects fails, as this opens an easy possiblity for unprivileged > applications to DOS the system,a s the shmem pages are not fully accounted > to the allocating process. > > Signed-off-by: Lucas Stach <dev@lynxeye.de> > --- > drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > index f0efc5d..4c53508 100644 > --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c > +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > @@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, > * going to pin these pages. > */ > mapping = obj->filp->f_mapping; > - mapping_set_gfp_mask(mapping, GFP_HIGHUSER); > + mapping_set_gfp_mask(mapping, GFP_HIGHUSER | > + __GFP_NORETRY | __GFP_NOWARN); _NORETRY means the mm does try hard at all to free memory. We've just done this patch in 4.12 and totally regret it, because now gpu tasks run out of memory with plenty of (gpu) memory available that could be reaped. There's some discussions going on with Michal Hocko and Chris Wilson about possible solutions for this without so much hacks. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails 2017-06-20 9:06 ` [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Daniel Vetter @ 2017-06-20 9:22 ` Lucas Stach 2017-06-20 9:25 ` Daniel Vetter 0 siblings, 1 reply; 8+ messages in thread From: Lucas Stach @ 2017-06-20 9:22 UTC (permalink / raw) To: Daniel Vetter; +Cc: etnaviv, dri-devel, Russell King, mhocko Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter: > On Tue, Jun 06, 2017 at 09:17:06AM +0200, Lucas Stach wrote: > > GPU buffers can be quite large, so userspace is expected to deal with > > allocation failure. Don't trigger the OOM killer when page allocation for > > the GEM objects fails, as this opens an easy possiblity for unprivileged > > applications to DOS the system,a s the shmem pages are not fully accounted > > to the allocating process. > > > > Signed-off-by: Lucas Stach <dev@lynxeye.de> > > --- > > drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > index f0efc5d..4c53508 100644 > > --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > @@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, > > * going to pin these pages. > > */ > > mapping = obj->filp->f_mapping; > > - mapping_set_gfp_mask(mapping, GFP_HIGHUSER); > > + mapping_set_gfp_mask(mapping, GFP_HIGHUSER | > > + __GFP_NORETRY | __GFP_NOWARN); > > _NORETRY means the mm does try hard at all to free memory. We've just done > this patch in 4.12 and totally regret it, because now gpu tasks run out of > memory with plenty of (gpu) memory available that could be reaped. > > There's some discussions going on with Michal Hocko and Chris Wilson > about possible solutions for this without so much hacks. Thanks for the heads up. In contrast to some of the other drivers Etnaviv hasn't grown a proper shrinker yet, so I still think this patch is a step in the right direction for the time being. Currently with Etnaviv sitting on lots of GPU memory and the MM trying hard to free more memory for GPU usage a simple piglit run might pressure the OOM killer hard enough to bring the system down by killing PID 1. Regards, Lucas _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails 2017-06-20 9:22 ` Lucas Stach @ 2017-06-20 9:25 ` Daniel Vetter 2017-06-26 5:52 ` Michal Hocko 0 siblings, 1 reply; 8+ messages in thread From: Daniel Vetter @ 2017-06-20 9:25 UTC (permalink / raw) To: Lucas Stach; +Cc: etnaviv, dri-devel, mhocko, Russell King On Tue, Jun 20, 2017 at 11:22:06AM +0200, Lucas Stach wrote: > Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter: > > On Tue, Jun 06, 2017 at 09:17:06AM +0200, Lucas Stach wrote: > > > GPU buffers can be quite large, so userspace is expected to deal with > > > allocation failure. Don't trigger the OOM killer when page allocation for > > > the GEM objects fails, as this opens an easy possiblity for unprivileged > > > applications to DOS the system,a s the shmem pages are not fully accounted > > > to the allocating process. > > > > > > Signed-off-by: Lucas Stach <dev@lynxeye.de> > > > --- > > > drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > index f0efc5d..4c53508 100644 > > > --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > @@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, > > > * going to pin these pages. > > > */ > > > mapping = obj->filp->f_mapping; > > > - mapping_set_gfp_mask(mapping, GFP_HIGHUSER); > > > + mapping_set_gfp_mask(mapping, GFP_HIGHUSER | > > > + __GFP_NORETRY | __GFP_NOWARN); > > > > _NORETRY means the mm does try hard at all to free memory. We've just done > > this patch in 4.12 and totally regret it, because now gpu tasks run out of > > memory with plenty of (gpu) memory available that could be reaped. > > > > There's some discussions going on with Michal Hocko and Chris Wilson > > about possible solutions for this without so much hacks. > > Thanks for the heads up. In contrast to some of the other drivers > Etnaviv hasn't grown a proper shrinker yet, so I still think this patch > is a step in the right direction for the time being. > Currently with Etnaviv sitting on lots of GPU memory and the MM trying > hard to free more memory for GPU usage a simple piglit run might > pressure the OOM killer hard enough to bring the system down by killing > PID 1. Ah ok, that changes the balance of course. Time to type that shrinker I'd say :-) -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails 2017-06-20 9:25 ` Daniel Vetter @ 2017-06-26 5:52 ` Michal Hocko 2017-06-26 6:48 ` Lucas Stach 0 siblings, 1 reply; 8+ messages in thread From: Michal Hocko @ 2017-06-26 5:52 UTC (permalink / raw) To: Daniel Vetter; +Cc: etnaviv, dri-devel, Russell King On Tue 20-06-17 11:25:24, Daniel Vetter wrote: > On Tue, Jun 20, 2017 at 11:22:06AM +0200, Lucas Stach wrote: > > Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter: > > > On Tue, Jun 06, 2017 at 09:17:06AM +0200, Lucas Stach wrote: > > > > GPU buffers can be quite large, so userspace is expected to deal with > > > > allocation failure. Don't trigger the OOM killer when page allocation for > > > > the GEM objects fails, as this opens an easy possiblity for unprivileged > > > > applications to DOS the system,a s the shmem pages are not fully accounted > > > > to the allocating process. > > > > > > > > Signed-off-by: Lucas Stach <dev@lynxeye.de> > > > > --- > > > > drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > index f0efc5d..4c53508 100644 > > > > --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > @@ -662,7 +662,8 @@ static struct drm_gem_object *__etnaviv_gem_new(struct drm_device *dev, > > > > * going to pin these pages. > > > > */ > > > > mapping = obj->filp->f_mapping; > > > > - mapping_set_gfp_mask(mapping, GFP_HIGHUSER); > > > > + mapping_set_gfp_mask(mapping, GFP_HIGHUSER | > > > > + __GFP_NORETRY | __GFP_NOWARN); > > > > > > _NORETRY means the mm does try hard at all to free memory. We've just done > > > this patch in 4.12 and totally regret it, because now gpu tasks run out of > > > memory with plenty of (gpu) memory available that could be reaped. > > > > > > There's some discussions going on with Michal Hocko and Chris Wilson > > > about possible solutions for this without so much hacks. > > > > Thanks for the heads up. In contrast to some of the other drivers > > Etnaviv hasn't grown a proper shrinker yet, so I still think this patch > > is a step in the right direction for the time being. > > Currently with Etnaviv sitting on lots of GPU memory and the MM trying > > hard to free more memory for GPU usage a simple piglit run might > > pressure the OOM killer hard enough to bring the system down by killing > > PID 1. > > Ah ok, that changes the balance of course. Time to type that shrinker I'd > say :-) Yeah, shrinker sounds like an absolute must. Anyway, I've posted __GFP_RETRY_MAYFAIL patchset [1] so you can use that flag. [1] http://lkml.kernel.org/r/20170623085345.11304-1-mhocko@kernel.org -- Michal Hocko SUSE Labs _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails 2017-06-26 5:52 ` Michal Hocko @ 2017-06-26 6:48 ` Lucas Stach 0 siblings, 0 replies; 8+ messages in thread From: Lucas Stach @ 2017-06-26 6:48 UTC (permalink / raw) To: Michal Hocko, Daniel Vetter; +Cc: Russell King, etnaviv, dri-devel Am Montag, den 26.06.2017, 07:52 +0200 schrieb Michal Hocko: > On Tue 20-06-17 11:25:24, Daniel Vetter wrote: > > On Tue, Jun 20, 2017 at 11:22:06AM +0200, Lucas Stach wrote: > > > Am Dienstag, den 20.06.2017, 11:06 +0200 schrieb Daniel Vetter: > > > > On Tue, Jun 06, 2017 at 09:17:06AM +0200, Lucas Stach wrote: > > > > > GPU buffers can be quite large, so userspace is expected to > > > > > deal with > > > > > allocation failure. Don't trigger the OOM killer when page > > > > > allocation for > > > > > the GEM objects fails, as this opens an easy possiblity for > > > > > unprivileged > > > > > applications to DOS the system,a s the shmem pages are not > > > > > fully accounted > > > > > to the allocating process. > > > > > > > > > > Signed-off-by: Lucas Stach <dev@lynxeye.de> > > > > > --- > > > > > drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- > > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > > > diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > > b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > > index f0efc5d..4c53508 100644 > > > > > --- a/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > > +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem.c > > > > > @@ -662,7 +662,8 @@ static struct drm_gem_object > > > > > *__etnaviv_gem_new(struct drm_device *dev, > > > > > * going to pin these pages. > > > > > */ > > > > > mapping = obj->filp->f_mapping; > > > > > - mapping_set_gfp_mask(mapping, GFP_HIGHUSER); > > > > > + mapping_set_gfp_mask(mapping, GFP_HIGHUSER | > > > > > + __GFP_NORETRY | > > > > > __GFP_NOWARN); > > > > > > > > _NORETRY means the mm does try hard at all to free memory. > > > > We've just done > > > > this patch in 4.12 and totally regret it, because now gpu tasks > > > > run out of > > > > memory with plenty of (gpu) memory available that could be > > > > reaped. > > > > > > > > There's some discussions going on with Michal Hocko and Chris > > > > Wilson > > > > about possible solutions for this without so much hacks. > > > > > > Thanks for the heads up. In contrast to some of the other drivers > > > Etnaviv hasn't grown a proper shrinker yet, so I still think this > > > patch > > > is a step in the right direction for the time being. > > > Currently with Etnaviv sitting on lots of GPU memory and the MM > > > trying > > > hard to free more memory for GPU usage a simple piglit run might > > > pressure the OOM killer hard enough to bring the system down by > > > killing > > > PID 1. > > > > Ah ok, that changes the balance of course. Time to type that > > shrinker I'd > > say :-) > > Yeah, shrinker sounds like an absolute must. Anyway, I've posted > __GFP_RETRY_MAYFAIL patchset [1] so you can use that flag. > > [1] http://lkml.kernel.org/r/20170623085345.11304-1-mhocko@kernel.org Thanks, I'll switch etnaviv over to use this flag when the patchset gets applied. A proper shrinker is on the TODO list, but there are only so much hours in a day... Regards, Lucas _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-06-26 6:48 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-06-06 7:17 [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Lucas Stach 2017-06-06 7:17 ` [PATCH 2/3] drm/etnaviv: reduce allocation failure message severity Lucas Stach 2017-06-06 7:17 ` [PATCH 3/3] drm/etnaviv: populate GEM objects on cpu_prep Lucas Stach 2017-06-20 9:06 ` [PATCH 1/3] drm/etnaviv: don't trigger OOM killer when page allocation fails Daniel Vetter 2017-06-20 9:22 ` Lucas Stach 2017-06-20 9:25 ` Daniel Vetter 2017-06-26 5:52 ` Michal Hocko 2017-06-26 6:48 ` Lucas Stach
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.