* [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node
@ 2022-03-16 1:27 Ming Lei
2022-03-16 2:54 ` Chaitanya Kulkarni
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Ming Lei @ 2022-03-16 1:27 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block, Ming Lei, Ewan D . Milne
sbitmap has been used in scsi for replacing atomic operations on
sdev->device_busy, so IOPS on some fast scsi storage can be improved.
However, sdev->device_busy can be changed in fast path, so we have to
allocate the sb->map statically. sdev->device_busy has been capped to 1024,
but some drivers may configure the default depth as < 8, then
cause each sbitmap word to hold only one bit. Finally 1024 * 128(
sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5
allocation, sometimes it may fail.
Avoid the issue by using kvzalloc_node() for allocating sb->map.
Cc: Ewan D. Milne <emilne@redhat.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
include/linux/sbitmap.h | 2 +-
lib/sbitmap.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/sbitmap.h b/include/linux/sbitmap.h
index dffeb8281c2d..8f5a86e210b9 100644
--- a/include/linux/sbitmap.h
+++ b/include/linux/sbitmap.h
@@ -174,7 +174,7 @@ static inline unsigned int __map_depth(const struct sbitmap *sb, int index)
static inline void sbitmap_free(struct sbitmap *sb)
{
free_percpu(sb->alloc_hint);
- kfree(sb->map);
+ kvfree(sb->map);
sb->map = NULL;
}
diff --git a/lib/sbitmap.c b/lib/sbitmap.c
index 2eb3de18ded3..ae4fd4de9ebe 100644
--- a/lib/sbitmap.c
+++ b/lib/sbitmap.c
@@ -110,7 +110,7 @@ int sbitmap_init_node(struct sbitmap *sb, unsigned int depth, int shift,
sb->alloc_hint = NULL;
}
- sb->map = kcalloc_node(sb->map_nr, sizeof(*sb->map), flags, node);
+ sb->map = kvzalloc_node(sb->map_nr * sizeof(*sb->map), flags, node);
if (!sb->map) {
free_percpu(sb->alloc_hint);
return -ENOMEM;
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node
2022-03-16 1:27 [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node Ming Lei
@ 2022-03-16 2:54 ` Chaitanya Kulkarni
2022-03-22 0:25 ` Ming Lei
2022-03-22 2:52 ` Jens Axboe
2 siblings, 0 replies; 5+ messages in thread
From: Chaitanya Kulkarni @ 2022-03-16 2:54 UTC (permalink / raw)
To: Ming Lei, Jens Axboe; +Cc: linux-block, Ewan D . Milne
On 3/15/22 18:27, Ming Lei wrote:
> sbitmap has been used in scsi for replacing atomic operations on
> sdev->device_busy, so IOPS on some fast scsi storage can be improved.
>
> However, sdev->device_busy can be changed in fast path, so we have to
> allocate the sb->map statically. sdev->device_busy has been capped to 1024,
> but some drivers may configure the default depth as < 8, then
> cause each sbitmap word to hold only one bit. Finally 1024 * 128(
> sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5
> allocation, sometimes it may fail.
>
> Avoid the issue by using kvzalloc_node() for allocating sb->map.
>
> Cc: Ewan D. Milne <emilne@redhat.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
>
maybe considering the total size before actual allocation
that will not buy us anything ?
Looks good.
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
-ck
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node
2022-03-16 1:27 [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node Ming Lei
2022-03-16 2:54 ` Chaitanya Kulkarni
@ 2022-03-22 0:25 ` Ming Lei
2022-03-22 2:01 ` Jens Axboe
2022-03-22 2:52 ` Jens Axboe
2 siblings, 1 reply; 5+ messages in thread
From: Ming Lei @ 2022-03-22 0:25 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block, Ewan D . Milne
Hello,
On Wed, Mar 16, 2022 at 09:27:08AM +0800, Ming Lei wrote:
> sbitmap has been used in scsi for replacing atomic operations on
> sdev->device_busy, so IOPS on some fast scsi storage can be improved.
>
> However, sdev->device_busy can be changed in fast path, so we have to
> allocate the sb->map statically. sdev->device_busy has been capped to 1024,
> but some drivers may configure the default depth as < 8, then
> cause each sbitmap word to hold only one bit. Finally 1024 * 128(
> sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5
> allocation, sometimes it may fail.
>
> Avoid the issue by using kvzalloc_node() for allocating sb->map.
>
> Cc: Ewan D. Milne <emilne@redhat.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
Ping...
thanks,
Ming
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node
2022-03-22 0:25 ` Ming Lei
@ 2022-03-22 2:01 ` Jens Axboe
0 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2022-03-22 2:01 UTC (permalink / raw)
To: Ming Lei; +Cc: linux-block, Ewan D . Milne
On 3/21/22 6:25 PM, Ming Lei wrote:
> Hello,
>
> On Wed, Mar 16, 2022 at 09:27:08AM +0800, Ming Lei wrote:
>> sbitmap has been used in scsi for replacing atomic operations on
>> sdev->device_busy, so IOPS on some fast scsi storage can be improved.
>>
>> However, sdev->device_busy can be changed in fast path, so we have to
>> allocate the sb->map statically. sdev->device_busy has been capped to 1024,
>> but some drivers may configure the default depth as < 8, then
>> cause each sbitmap word to hold only one bit. Finally 1024 * 128(
>> sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5
>> allocation, sometimes it may fail.
>>
>> Avoid the issue by using kvzalloc_node() for allocating sb->map.
>>
>> Cc: Ewan D. Milne <emilne@redhat.com>
>> Signed-off-by: Ming Lei <ming.lei@redhat.com>
>
> Ping...
Was waiting on a response to the question on why we don't make this
dependent on size, but upon look at kvzalloc_node(), this is already
what it does. So I guess this looks fine to me too.
--
Jens Axboe
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node
2022-03-16 1:27 [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node Ming Lei
2022-03-16 2:54 ` Chaitanya Kulkarni
2022-03-22 0:25 ` Ming Lei
@ 2022-03-22 2:52 ` Jens Axboe
2 siblings, 0 replies; 5+ messages in thread
From: Jens Axboe @ 2022-03-22 2:52 UTC (permalink / raw)
To: Ming Lei; +Cc: linux-block, Ewan D . Milne
On Wed, 16 Mar 2022 09:27:08 +0800, Ming Lei wrote:
> sbitmap has been used in scsi for replacing atomic operations on
> sdev->device_busy, so IOPS on some fast scsi storage can be improved.
>
> However, sdev->device_busy can be changed in fast path, so we have to
> allocate the sb->map statically. sdev->device_busy has been capped to 1024,
> but some drivers may configure the default depth as < 8, then
> cause each sbitmap word to hold only one bit. Finally 1024 * 128(
> sizeof(sbitmap_word)) bytes is needed for sb->map, given it is order 5
> allocation, sometimes it may fail.
>
> [...]
Applied, thanks!
[1/1] lib/sbitmap: allocate sb->map via kvzalloc_node
commit: 863a66cdb4df25fd146d9851c3289072298566d5
Best regards,
--
Jens Axboe
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-03-22 2:52 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-16 1:27 [PATCH] lib/sbitmap: allocate sb->map via kvzalloc_node Ming Lei
2022-03-16 2:54 ` Chaitanya Kulkarni
2022-03-22 0:25 ` Ming Lei
2022-03-22 2:01 ` Jens Axboe
2022-03-22 2:52 ` Jens Axboe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.