* [PATCH] bcache: limit multiple flash devices size
@ 2022-09-14 6:06 mingzhe.zou
2022-09-15 15:23 ` Coly Li
0 siblings, 1 reply; 2+ messages in thread
From: mingzhe.zou @ 2022-09-14 6:06 UTC (permalink / raw)
To: colyli, linux-bcache; +Cc: zoumingzhe
From: mingzhe <mingzhe.zou@easystack.cn>
Bcache allows multiple flash devices to be created on the same cache.
We can create multiple flash devices, and the total size larger than
cache device's actual size.
```
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
[root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
└─bcache1 251:128 0 50G 0 disk
[root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
├─bcache2 251:256 0 50G 0 disk
└─bcache1 251:128 0 50G 0 disk
[root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
├─bcache3 251:256 0 50G 0 disk
├─bcache2 251:256 0 50G 0 disk
└─bcache1 251:128 0 50G 0 disk
```
This patch will limit the total size of multi-flash device, until no
free space to create a new flash device with an error.
```
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
[root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
└─bcache1 251:128 0 50G 0 disk
[root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
├─bcache2 251:256 0 39.9G 0 disk
└─bcache1 251:128 0 50G 0 disk
[root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
-bash: echo: write error: Invalid argument
[root@zou ~]# lsblk /dev/vdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vdd 252:48 0 100G 0 disk
├─bcache2 251:256 0 39.9G 0 disk
└─bcache1 251:128 0 50G 0 disk
```
Signed-off-by: mingzhe <mingzhe.zou@easystack.cn>
---
drivers/md/bcache/super.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index 214a384dc1d7..e019cfd793eb 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
@@ -1581,13 +1581,20 @@ static int flash_devs_run(struct cache_set *c)
static inline sector_t flash_dev_max_sectors(struct cache_set *c)
{
+ sector_t sectors;
+ struct uuid_entry *u;
size_t avail_nbuckets;
struct cache *ca = c->cache;
size_t first_bucket = ca->sb.first_bucket;
size_t njournal_buckets = ca->sb.njournal_buckets;
avail_nbuckets = c->nbuckets - first_bucket - njournal_buckets;
- return bucket_to_sector(c, avail_nbuckets / 100 * FLASH_DEV_AVAILABLE_RATIO);
+ sectors = bucket_to_sector(c, avail_nbuckets / 100 * FLASH_DEV_AVAILABLE_RATIO);
+
+ for (u = c->uuids; u < c->uuids + c->nr_uuids && sectors > 0; u++)
+ if (UUID_FLASH_ONLY(u))
+ sectors -= min(u->sectors, sectors);
+ return sectors;
}
int bch_flash_dev_create(struct cache_set *c, uint64_t size)
@@ -1612,6 +1619,10 @@ int bch_flash_dev_create(struct cache_set *c, uint64_t size)
SET_UUID_FLASH_ONLY(u, 1);
u->sectors = min(flash_dev_max_sectors(c), size >> 9);
+ if (!u->sectors) {
+ pr_err("Can't create volume, no free space");
+ return -EINVAL;
+ }
bch_uuid_write(c);
--
2.17.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH] bcache: limit multiple flash devices size
2022-09-14 6:06 [PATCH] bcache: limit multiple flash devices size mingzhe.zou
@ 2022-09-15 15:23 ` Coly Li
0 siblings, 0 replies; 2+ messages in thread
From: Coly Li @ 2022-09-15 15:23 UTC (permalink / raw)
To: mingzhe.zou; +Cc: linux-bcache, zoumingzhe
> 2022年9月14日 14:06，mingzhe.zou@easystack.cn 写道：
>
> From: mingzhe <mingzhe.zou@easystack.cn>
>
> Bcache allows multiple flash devices to be created on the same cache.
> We can create multiple flash devices, and the total size larger than
> cache device's actual size.
> ```
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> [root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> └─bcache1 251:128 0 50G 0 disk
> [root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> ├─bcache2 251:256 0 50G 0 disk
> └─bcache1 251:128 0 50G 0 disk
> [root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> ├─bcache3 251:256 0 50G 0 disk
> ├─bcache2 251:256 0 50G 0 disk
> └─bcache1 251:128 0 50G 0 disk
> ```
>
> This patch will limit the total size of multi-flash device, until no
> free space to create a new flash device with an error.
> ```
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> [root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> └─bcache1 251:128 0 50G 0 disk
> [root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> ├─bcache2 251:256 0 39.9G 0 disk
> └─bcache1 251:128 0 50G 0 disk
> [root@zou ~]# echo 50G > /sys/block/vdd/bcache/set/flash_vol_create
> -bash: echo: write error: Invalid argument
> [root@zou ~]# lsblk /dev/vdd
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> vdd 252:48 0 100G 0 disk
> ├─bcache2 251:256 0 39.9G 0 disk
> └─bcache1 251:128 0 50G 0 disk
> ```
>
> Signed-off-by: mingzhe <mingzhe.zou@easystack.cn>
> ---
> drivers/md/bcache/super.c | 13 ++++++++++++-
> 1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
> index 214a384dc1d7..e019cfd793eb 100644
> --- a/drivers/md/bcache/super.c
> +++ b/drivers/md/bcache/super.c
> @@ -1581,13 +1581,20 @@ static int flash_devs_run(struct cache_set *c)
>
> static inline sector_t flash_dev_max_sectors(struct cache_set *c)
> {
> + sector_t sectors;
> + struct uuid_entry *u;
> size_t avail_nbuckets;
> struct cache *ca = c->cache;
> size_t first_bucket = ca->sb.first_bucket;
> size_t njournal_buckets = ca->sb.njournal_buckets;
>
> avail_nbuckets = c->nbuckets - first_bucket - njournal_buckets;
> - return bucket_to_sector(c, avail_nbuckets / 100 * FLASH_DEV_AVAILABLE_RATIO);
> + sectors = bucket_to_sector(c, avail_nbuckets / 100 * FLASH_DEV_AVAILABLE_RATIO);
> +
> + for (u = c->uuids; u < c->uuids + c->nr_uuids && sectors > 0; u++)
> + if (UUID_FLASH_ONLY(u))
> + sectors -= min(u->sectors, sectors);
> + return sectors;
The value returned from flash_dev_max_sectors() is the buckets number which not allocated to flash devices. But it might not always be the allocable free buckets for new flash device. Because some of the buckets might be allocated to btree nodes, or cached dirty data. Although these space might be shrunk eventually, we should always avoid to use up all the free buckets.
Therefore, the exact free bucket amount should be calculated —-- no cheap method to do it.
There is a variable cache_set->avail_nbuckets for current available buckets, but it is updated after gc accomplished and not a updated-in-time value. So this value is always <= real available buckets. That is to say, if the creating flash device size < (cache_set->avail_nbuckets - reserved_buckets), the creation failed but there might be enough free buckets for the creating flash device. This is very probably to happen because cache_set->avail_nbuckets is not refreshed frequently.
> }
>
> int bch_flash_dev_create(struct cache_set *c, uint64_t size)
> @@ -1612,6 +1619,10 @@ int bch_flash_dev_create(struct cache_set *c, uint64_t size)
>
> SET_UUID_FLASH_ONLY(u, 1);
> u->sectors = min(flash_dev_max_sectors(c), size >> 9);
> + if (!u->sectors) {
> + pr_err("Can't create volume, no free space");
> + return -EINVAL;
> + }
The idea is cool. But current code doesn’t solve the target problem, and I don’t have better solution in my brain yet...
Thanks.
Coly Li
>
> bch_uuid_write(c);
>
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-09-15 15:24 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-14 6:06 [PATCH] bcache: limit multiple flash devices size mingzhe.zou
2022-09-15 15:23 ` Coly Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).