[PATCH] virtio-gpu: fix possible memory allocation failure

[PATCH] virtio-gpu: fix possible memory allocation failure

[Liu Yuntao]

When kmem_cache_zalloc in virtio_gpu_get_vbuf fails, it will return
an error code. But none of its callers checks this error code, and
a core dump will take place.

Considering many of its callers can't handle such error, I add
a __GFP_NOFAIL flag when calling kmem_cache_zalloc to make sure
it won't fail, and delete those unused error handlings.

Fixes: dc5698e80cf724 ("Add virtio gpu driver.")
Signed-off-by: Yuntao Liu <liuyuntao10@huawei.com>
---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index cf84d382dd41..5286cf110208 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -91,9 +91,7 @@ virtio_gpu_get_vbuf(struct virtio_gpu_device *vgdev,
 {
 	struct virtio_gpu_vbuffer *vbuf;
 
-	vbuf = kmem_cache_zalloc(vgdev->vbufs, GFP_KERNEL);
-	if (!vbuf)
-		return ERR_PTR(-ENOMEM);
+	vbuf = kmem_cache_zalloc(vgdev->vbufs, GFP_KERNEL | __GFP_NOFAIL);
 
 	BUG_ON(size > MAX_INLINE_CMD_SIZE ||
 	       size < sizeof(struct virtio_gpu_ctrl_hdr));
@@ -147,10 +145,6 @@ static void *virtio_gpu_alloc_cmd_resp(struct virtio_gpu_device *vgdev,
 
 	vbuf = virtio_gpu_get_vbuf(vgdev, cmd_size,
 				   resp_size, resp_buf, cb);
-	if (IS_ERR(vbuf)) {
-		*vbuffer_p = NULL;
-		return ERR_CAST(vbuf);
-	}
 	*vbuffer_p = vbuf;
 	return (struct virtio_gpu_command *)vbuf->buf;
 }
-- 
2.23.0

[PATCH] fix judgment error in shmem_is_huge()

[Liu Yuntao]

In the case of SHMEM_HUGE_WITHIN_SIZE, the page index is not rounded
up correctly. When the page index points to the first page in a huge
page, round_up() cannot bring it to the end of the huge page, but
to the end of the previous one.

an example:
HPAGE_PMD_NR on my machine is 512(2 MB huge page size).
After allcoating a 3000 KB buffer, I access it at location 2050 KB.
In shmem_is_huge(), the corresponding index happens to be 512.
After rounded up by HPAGE_PMD_NR, it will still be 512 which is
smaller than i_size, and shmem_is_huge() will return true.
As a result, my buffer takes an additional huge page, and that
shouldn't happen when shmem_enabled is set to within_size.

Fixes: f3f0e1d2150b2b ("khugepaged: add support of collapse for tmpfs/shmem pages")
Signed-off-by: Liu Yuntao <liuyuntao10@huawei.com>
---
 mm/shmem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index 88742953532c..5747572859d1 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -490,7 +490,7 @@ bool shmem_is_huge(struct vm_area_struct *vma,
 	case SHMEM_HUGE_ALWAYS:
 		return true;
 	case SHMEM_HUGE_WITHIN_SIZE:
-		index = round_up(index, HPAGE_PMD_NR);
+		index = round_up(index + 1, HPAGE_PMD_NR);
 		i_size = round_up(i_size_read(inode), PAGE_SIZE);
 		if (i_size >= HPAGE_PMD_SIZE && (i_size >> PAGE_SHIFT) >= index)
 			return true;
-- 
2.23.0

Re: [PATCH] fix judgment error in shmem_is_huge()

[Kirill A. Shutemov]

On Wed, Sep 08, 2021 at 06:26:48PM +0800, Liu Yuntao wrote:
> In the case of SHMEM_HUGE_WITHIN_SIZE, the page index is not rounded
> up correctly. When the page index points to the first page in a huge
> page, round_up() cannot bring it to the end of the huge page, but
> to the end of the previous one.
> 
> an example:
> HPAGE_PMD_NR on my machine is 512(2 MB huge page size).
> After allcoating a 3000 KB buffer, I access it at location 2050 KB.
> In shmem_is_huge(), the corresponding index happens to be 512.
> After rounded up by HPAGE_PMD_NR, it will still be 512 which is
> smaller than i_size, and shmem_is_huge() will return true.
> As a result, my buffer takes an additional huge page, and that
> shouldn't happen when shmem_enabled is set to within_size.
> 
> Fixes: f3f0e1d2150b2b ("khugepaged: add support of collapse for tmpfs/shmem pages")
> Signed-off-by: Liu Yuntao <liuyuntao10@huawei.com>
> ---
>  mm/shmem.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 88742953532c..5747572859d1 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -490,7 +490,7 @@ bool shmem_is_huge(struct vm_area_struct *vma,
>  	case SHMEM_HUGE_ALWAYS:
>  		return true;
>  	case SHMEM_HUGE_WITHIN_SIZE:
> -		index = round_up(index, HPAGE_PMD_NR);
> +		index = round_up(index + 1, HPAGE_PMD_NR);
>  		i_size = round_up(i_size_read(inode), PAGE_SIZE);
>  		if (i_size >= HPAGE_PMD_SIZE && (i_size >> PAGE_SHIFT) >= index)

With the change, the condition can be simplified to

		if (i_size >> PAGE_SHIFT >= index)

right?

>  			return true;
> -- 
> 2.23.0
> 

-- 
 Kirill A. Shutemov

Re: [PATCH] fix judgment error in shmem_is_huge()

[Liu Yuntao]

On Wed, 8 Sep 2021 17:58:44 +0300, Kirill A. Shutemov wrote:
> On Wed, Sep 08, 2021 at 06:26:48PM +0800, Liu Yuntao wrote:
> > In the case of SHMEM_HUGE_WITHIN_SIZE, the page index is not rounded
> > up correctly. When the page index points to the first page in a huge
> > page, round_up() cannot bring it to the end of the huge page, but
> > to the end of the previous one.
> > 
> > an example:
> > HPAGE_PMD_NR on my machine is 512(2 MB huge page size).
> > After allcoating a 3000 KB buffer, I access it at location 2050 KB.
> > In shmem_is_huge(), the corresponding index happens to be 512.
> > After rounded up by HPAGE_PMD_NR, it will still be 512 which is
> > smaller than i_size, and shmem_is_huge() will return true.
> > As a result, my buffer takes an additional huge page, and that
> > shouldn't happen when shmem_enabled is set to within_size.
> > 
> > Fixes: f3f0e1d2150b2b ("khugepaged: add support of collapse for tmpfs/shmem pages")
> > Signed-off-by: Liu Yuntao <liuyuntao10@huawei.com>
> > ---
> >  mm/shmem.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/mm/shmem.c b/mm/shmem.c
> > index 88742953532c..5747572859d1 100644
> > --- a/mm/shmem.c
> > +++ b/mm/shmem.c
> > @@ -490,7 +490,7 @@ bool shmem_is_huge(struct vm_area_struct *vma,
> >  	case SHMEM_HUGE_ALWAYS:
> >  		return true;
> >  	case SHMEM_HUGE_WITHIN_SIZE:
> > -		index = round_up(index, HPAGE_PMD_NR);
> > +		index = round_up(index + 1, HPAGE_PMD_NR);
> >  		i_size = round_up(i_size_read(inode), PAGE_SIZE);
> >  		if (i_size >= HPAGE_PMD_SIZE && (i_size >> PAGE_SHIFT) >= index)
> 
> With the change, the condition can be simplified to
> 
> 		if (i_size >> PAGE_SHIFT >= index)
> 
> right?

Yes, will add it.

> 
> >  			return true;
> > -- 
> > 2.23.0
> > 
> 
> -- 
>  Kirill A. Shutemov

[PATCH] virtio-gpu: fix possible memory allocation failure

[liuyuntao]

When kmem_cache_zalloc in virtio_gpu_get_vbuf fails, it will return
an error code. But none of its callers checks this error code, and
a core dump will take place.

Considering many of its callers can't handle such error, I add
a __GFP_NOFAIL flag when calling kmem_cache_zalloc to make sure
it won't fail, and delete those unused error handlings.

Fixes: dc5698e80cf724 ("Add virtio gpu driver.")
Signed-off-by: Yuntao Liu <liuyuntao10@huawei.com>
---
 drivers/gpu/drm/virtio/virtgpu_vq.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
index cf84d382dd41..5286cf110208 100644
--- a/drivers/gpu/drm/virtio/virtgpu_vq.c
+++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
@@ -91,9 +91,7 @@ virtio_gpu_get_vbuf(struct virtio_gpu_device *vgdev,
 {
 	struct virtio_gpu_vbuffer *vbuf;
 
-	vbuf = kmem_cache_zalloc(vgdev->vbufs, GFP_KERNEL);
-	if (!vbuf)
-		return ERR_PTR(-ENOMEM);
+	vbuf = kmem_cache_zalloc(vgdev->vbufs, GFP_KERNEL | __GFP_NOFAIL);
 
 	BUG_ON(size > MAX_INLINE_CMD_SIZE ||
 	       size < sizeof(struct virtio_gpu_ctrl_hdr));
@@ -147,10 +145,6 @@ static void *virtio_gpu_alloc_cmd_resp(struct virtio_gpu_device *vgdev,
 
 	vbuf = virtio_gpu_get_vbuf(vgdev, cmd_size,
 				   resp_size, resp_buf, cb);
-	if (IS_ERR(vbuf)) {
-		*vbuffer_p = NULL;
-		return ERR_CAST(vbuf);
-	}
 	*vbuffer_p = vbuf;
 	return (struct virtio_gpu_command *)vbuf->buf;
 }
-- 
2.23.0