* make-bcache -B /dev/bcache0 fails, Can't stack bcache on top of bcache
@ 2021-01-02 19:18
0 siblings, 0 replies; 5+ messages in thread
From: @ 2021-01-02 19:18 UTC (permalink / raw)
To: linux-bcache
TLDR: I can't stack bcache on top of bcache, This command fails:
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
[ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 5187.054670] Call Trace:
[ 5187.054681] dump_stack+0x6b/0x83
[ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
[ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
[ 5187.054699] kobject_add_internal+0xab/0x2f0
[ 5187.054703] kobject_add+0x98/0xd0
[ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
[ 5187.054723] register_bdev+0x337/0x360 [bcache]
[ 5187.054735] register_bcache+0x43c/0x910 [bcache]
[ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
[ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
[ 5187.054753] kernfs_fop_write+0xce/0x1b0
[ 5187.054756] vfs_write+0xc7/0x210
[ 5187.054759] ksys_write+0x67/0xe0
[ 5187.054763] do_syscall_64+0x33/0x40
[ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 5187.054769] RIP: 0033:0x7fbd1edb3f67
[ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
[ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
[ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
[ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
[ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
[ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
[ 5187.054794] bcache: register_bcache() error : failed to register device
[ 5187.054963] bcache: bcache_device_free() bcache3 stopped
(very) long version:
I want to create a silent and fast NAS. I want to keep the hard drives idle for as long as possible. Therefore I need flash memory as write cache, so the hard drives stay idle when the user writes a file. The write caches rarely get flushed into the drives, so they are dirty for a long time. Therefore each drive needs it's own write cache device, so btrfs can recreate the lost data when a device fails. The read cache can be shared across all drives, as all the data in it also exists in the devices under it, so failure of the read cache does not result in data loss.
I'm testing this on a single drive with a lot of partitions. This kills performance (and fault tolerance). In production, each partition will be on a different drive.
+--------------------------------------------+
| btrfs raid 1 (2 copies) /mnt |
+--------------+--------------+--------------+
| /dev/bcache3 | /dev/bcache4 | /dev/bcache5 |
+--------------+--------------+--------------+
| Read Cache (SSD) |
| /dev/sda4 |
+--------------+--------------+--------------+
| /dev/bcache0 | /dev/bcache1 | /dev/bcache2 |
+--------------+--------------+--------------+
| Write Cache | Write Cache | Write Cache |
|(Flash Drive) |(Flash Drive) |(Flash Drive) |
| /dev/sda5 | /dev/sda6 | /dev/sda7 |
+--------------+--------------+--------------+
| Data | Data | Data |
| /dev/sda8 | /dev/sda9 | /dev/sda10 |
+--------------+--------------+--------------+
0. make the partitions on the drive:
# fdisk -l
/dev/sda1 2048 6293503 6291456 3G 83 Linux
/dev/sda2 6293504 14682111 8388608 4G 83 Linux
/dev/sda3 14682112 25167871 10485760 5G 83 Linux
/dev/sda4 25167872 976773167 951605296 453.8G 5 Extended
/dev/sda5 25169920 37752831 12582912 6G 83 Linux
/dev/sda6 37754880 52434943 14680064 7G 83 Linux
/dev/sda7 52436992 69214207 16777216 8G 83 Linux
/dev/sda8 69216256 88090623 18874368 9G 83 Linux
/dev/sda9 88092672 109064191 20971520 10G 83 Linux
/dev/sda10 109066240 132134911 23068672 11G 83 Linux
Wipe the partitions:
[root@bcache-test cedric]# wipefs -a /dev/sda5
[root@bcache-test cedric]# wipefs -a /dev/sda6
[root@bcache-test cedric]# wipefs -a /dev/sda7
[root@bcache-test cedric]# wipefs -a /dev/sda8
[root@bcache-test cedric]# wipefs -a /dev/sda9
[root@bcache-test cedric]# wipefs -a /dev/sda10
Check for old bcache devices:
# ls /dev/bca*
ls: cannot access '/dev/bca*': No such file or directory
1. Format the backing devices:
root@bcache-test cedric]# make-bcache -B /dev/sda8
Name /dev/sda8
Label
Type data
UUID: 3bae90e9-d2c5-4d62-96a8-07dc1cb1440d
Set UUID: 271e657c-2a3a-417e-9a8d-430129095a47
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 4196.603418] bcache: register_bdev() registered backing device sda8
# make-bcache -B /dev/sda9
Name /dev/sda9
Label
Type data
UUID: 28ce8fa5-6918-4637-b612-374cf75635c2
Set UUID: c12faa51-6042-4b46-8448-ee4a5acd0e8e
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 4269.009004] bcache: register_bdev() registered backing device sda9
[root@bcache-test cedric]# make-bcache -B /dev/sda10
Name /dev/sda10
Label
Type data
UUID: 4bca4225-4143-41af-b0e2-2ca2700e4931
Set UUID: 5442ccbe-6f12-40df-a309-30f66191773f
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 4285.151623] bcache: register_bdev() registered backing device sda10
Now 3 bcache devices are created:
[root@bcache-test cedric]# ls -l /dev/bca*
brw-rw---- 1 root disk 254, 0 Jan 2 20:46 /dev/bcache0
brw-rw---- 1 root disk 254, 128 Jan 2 20:47 /dev/bcache1
brw-rw---- 1 root disk 254, 256 Jan 2 20:47 /dev/bcache2
/dev/bcache:
total 0
drwxr-xr-x 2 root root 100 Jan 2 20:47 by-uuid
2. Format the cache devices:
[root@bcache-test cedric]# make-bcache -C /dev/sda5
Name /dev/sda5
Label
Type cache
UUID: 6e7610b2-5f28-4699-aed0-ffd78c1db77d
Set UUID: 924e51c2-9b49-4802-b87f-b379cd9a06a6
version: 0
nbuckets: 12288
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
[ 4505.224494] bcache: run_cache_set() invalidating existing data
[ 4505.265255] bcache: register_cache() registered cache device sda5
[root@bcache-test cedric]# make-bcache -C /dev/sda6
Name /dev/sda6
Label
Type cache
UUID: 4ff65c49-e225-4975-aac1-1be7da22d434
Set UUID: ab10dafe-e8a4-4aa9-89f7-1a0575659f06
version: 0
nbuckets: 14336
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
[ 4558.676081] bcache: run_cache_set() invalidating existing data
[ 4558.724051] bcache: register_cache() registered cache device sda6
[root@bcache-test cedric]# make-bcache -C /dev/sda7
Name /dev/sda7
Label
Type cache
UUID: 02d4fe5e-5c01-4c94-9e89-3a97b01118b2
Set UUID: 31350209-3c13-4e78-b782-2c16e9d52e6a
version: 0
nbuckets: 16384
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
[ 4583.036101] bcache: run_cache_set() invalidating existing data
[ 4583.077035] bcache: register_cache() registered cache device sda7
Register the cache devices against the backing devices.
# echo 924e51c2-9b49-4802-b87f-b379cd9a06a6 > /sys/block/bcache0/bcache/attach
[ 5008.251480] bcache: bch_cached_dev_run() cached dev sda8 is running already
[ 5008.251500] bcache: bch_cached_dev_attach() Caching sda8 as bcache0 on set 924e51c2-9b49-4802-b87f-b379cd9a06a6
# echo ab10dafe-e8a4-4aa9-89f7-1a0575659f06 > /sys/block/bcache1/bcache/attach
[ 5037.562635] bcache: bch_cached_dev_run() cached dev sda9 is running already
[ 5037.562648] bcache: bch_cached_dev_attach() Caching sda9 as bcache1 on set ab10dafe-e8a4-4aa9-89f7-1a0575659f06
# echo 31350209-3c13-4e78-b782-2c16e9d52e6a > /sys/block/bcache2/bcache/attach
[ 5061.280096] bcache: bch_cached_dev_run() cached dev sda10 is running already
[ 5061.280110] bcache: bch_cached_dev_attach() Caching sda10 as bcache2 on set 31350209-3c13-4e78-b782-2c16e9d52e6a
Format /dev/bcache0 as backing devices for the read cache fails:
make-bcache -B /dev/bcache0
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
[ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 5187.054670] Call Trace:
[ 5187.054681] dump_stack+0x6b/0x83
[ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
[ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
[ 5187.054699] kobject_add_internal+0xab/0x2f0
[ 5187.054703] kobject_add+0x98/0xd0
[ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
[ 5187.054723] register_bdev+0x337/0x360 [bcache]
[ 5187.054735] register_bcache+0x43c/0x910 [bcache]
[ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
[ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
[ 5187.054753] kernfs_fop_write+0xce/0x1b0
[ 5187.054756] vfs_write+0xc7/0x210
[ 5187.054759] ksys_write+0x67/0xe0
[ 5187.054763] do_syscall_64+0x33/0x40
[ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 5187.054769] RIP: 0033:0x7fbd1edb3f67
[ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
[ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
[ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
[ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
[ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
[ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
[ 5187.054794] bcache: register_bcache() error : failed to register device
[ 5187.054963] bcache: bcache_device_free() bcache3 stopped
I expected this command to create /dev/bcache3
My versions:
# uname -a
Linux bcache-test 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
# pacman -Q bcache-tools
bcache-tools 1.1-1
I've used this guide for inspiration:
https://wiki.archlinux.org/index.php/Bcache
---
Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: make-bcache -B /dev/bcache0 fails, Can't stack bcache on top of bcache
2021-01-02 19:38
@ 2021-01-04 2:57 ` Dongsheng Yang
0 siblings, 0 replies; 5+ messages in thread
From: Dongsheng Yang @ 2021-01-04 2:57 UTC (permalink / raw)
To: Cedric.dewijs, linux-bcache
Hi Cedric,
I have a patch to support overlay bcache, you can find it in branch:
https://github.com/yangdongsheng/linux/commits/bcache_writeback
"bcache: support overlay bcache "
https://github.com/yangdongsheng/linux/commit/c72756df2d08acbb58565135bbde61134fb3208a
You can give it a try.
Thanx
在 2021/1/3 星期日 上午 3:38, Cedric.dewijs@eclipso.eu 写道:
> I can't stack bcache on top of bcache, This command fails:
> # make-bcache -B /dev/bcache0
> Name /dev/bcache0
> Label
> Type data
> UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
> Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
> version: 1
> block_size_in_sectors: 1
> data_offset_in_sectors: 16
> [ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
> [ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
> [ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
> [ 5187.054670] Call Trace:
> [ 5187.054681] dump_stack+0x6b/0x83
> [ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
> [ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
> [ 5187.054699] kobject_add_internal+0xab/0x2f0
> [ 5187.054703] kobject_add+0x98/0xd0
> [ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
> [ 5187.054723] register_bdev+0x337/0x360 [bcache]
> [ 5187.054735] register_bcache+0x43c/0x910 [bcache]
> [ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
> [ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
> [ 5187.054753] kernfs_fop_write+0xce/0x1b0
> [ 5187.054756] vfs_write+0xc7/0x210
> [ 5187.054759] ksys_write+0x67/0xe0
> [ 5187.054763] do_syscall_64+0x33/0x40
> [ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 5187.054769] RIP: 0033:0x7fbd1edb3f67
> [ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
> [ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
> [ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
> [ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
> [ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
> [ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
> [ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
> [ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
> [ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
> [ 5187.054794] bcache: register_bcache() error : failed to register device
> [ 5187.054963] bcache: bcache_device_free() bcache3 stopped
>
> I expected this command to create a new /dev/bcacheX node. Instead it tries to overwrite /devices/virtual/block/bcache0/bcache
>
> My versions:
> # uname -a
> Linux bcache-test 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
> # pacman -Q bcache-tools
> bcache-tools 1.1-1
>
> I've used this guide for inspiration:
> https://wiki.archlinux.org/index.php/Bcache
>
>
> ---
>
> Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* make-bcache -B /dev/bcache0 fails, Can't stack bcache on top of bcache
@ 2021-01-02 21:36
0 siblings, 0 replies; 5+ messages in thread
From: @ 2021-01-02 21:36 UTC (permalink / raw)
To: , linux-bcache@vger.kernel.org
I can't stack bcache on top of bcache, This command fails:
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
[ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 5187.054670] Call Trace:
[ 5187.054681] dump_stack+0x6b/0x83
[ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
[ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
[ 5187.054699] kobject_add_internal+0xab/0x2f0
[ 5187.054703] kobject_add+0x98/0xd0
[ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
[ 5187.054723] register_bdev+0x337/0x360 [bcache]
[ 5187.054735] register_bcache+0x43c/0x910 [bcache]
[ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
[ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
[ 5187.054753] kernfs_fop_write+0xce/0x1b0
[ 5187.054756] vfs_write+0xc7/0x210
[ 5187.054759] ksys_write+0x67/0xe0
[ 5187.054763] do_syscall_64+0x33/0x40
[ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 5187.054769] RIP: 0033:0x7fbd1edb3f67
[ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
[ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
[ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
[ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
[ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
[ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
[ 5187.054794] bcache: register_bcache() error : failed to register device
[ 5187.054963] bcache: bcache_device_free() bcache3 stopped
I expected this command to create a new /dev/bcacheX node. Instead it tries to overwrite /devices/virtual/block/bcache0/bcache
My versions:
# uname -a
Linux bcache-test 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
# pacman -Q bcache-tools
bcache-tools 1.1-1
I've used this guide for inspiration:
https://wiki.archlinux.org/index.php/Bcache
---
Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
^ permalink raw reply [flat|nested] 5+ messages in thread
* make-bcache -B /dev/bcache0 fails, Can't stack bcache on top of bcache
@ 2021-01-02 21:34
0 siblings, 0 replies; 5+ messages in thread
From: @ 2021-01-02 21:34 UTC (permalink / raw)
To: linux-bcache
TLDR: I can't stack bcache on top of bcache, This command fails:
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
[ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 5187.054670] Call Trace:
[ 5187.054681] dump_stack+0x6b/0x83
[ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
[ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
[ 5187.054699] kobject_add_internal+0xab/0x2f0
[ 5187.054703] kobject_add+0x98/0xd0
[ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
[ 5187.054723] register_bdev+0x337/0x360 [bcache]
[ 5187.054735] register_bcache+0x43c/0x910 [bcache]
[ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
[ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
[ 5187.054753] kernfs_fop_write+0xce/0x1b0
[ 5187.054756] vfs_write+0xc7/0x210
[ 5187.054759] ksys_write+0x67/0xe0
[ 5187.054763] do_syscall_64+0x33/0x40
[ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 5187.054769] RIP: 0033:0x7fbd1edb3f67
[ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
[ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
[ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
[ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
[ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
[ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
[ 5187.054794] bcache: register_bcache() error : failed to register device
[ 5187.054963] bcache: bcache_device_free() bcache3 stopped
(very) long version:
I want to create a silent and fast NAS. I want to keep the hard drives idle for as long as possible. Therefore I need flash memory as write cache, so the hard drives stay idle when the user writes a file. The write caches rarely get flushed into the drives, so they are dirty for a long time. Therefore each drive needs it's own write cache device, so btrfs can recreate the lost data when a device fails. The read cache can be shared across all drives, as all the data in it also exists in the devices under it, so failure of the read cache does not result in data loss.
I'm testing this on a single drive with a lot of partitions. This kills performance (and fault tolerance). In production, each partition will be on a different drive.
+--------------------------------------------+
| btrfs raid 1 (2 copies) /mnt |
+--------------+--------------+--------------+
| /dev/bcache3 | /dev/bcache4 | /dev/bcache5 |
+--------------+--------------+--------------+
| Read Cache (SSD) |
| /dev/sda4 |
+--------------+--------------+--------------+
| /dev/bcache0 | /dev/bcache1 | /dev/bcache2 |
+--------------+--------------+--------------+
| Write Cache | Write Cache | Write Cache |
|(Flash Drive) |(Flash Drive) |(Flash Drive) |
| /dev/sda5 | /dev/sda6 | /dev/sda7 |
+--------------+--------------+--------------+
| Data | Data | Data |
| /dev/sda8 | /dev/sda9 | /dev/sda10 |
+--------------+--------------+--------------+
0. make the partitions on the drive:
# fdisk -l
/dev/sda1 2048 6293503 6291456 3G 83 Linux
/dev/sda2 6293504 14682111 8388608 4G 83 Linux
/dev/sda3 14682112 25167871 10485760 5G 83 Linux
/dev/sda4 25167872 976773167 951605296 453.8G 5 Extended
/dev/sda5 25169920 37752831 12582912 6G 83 Linux
/dev/sda6 37754880 52434943 14680064 7G 83 Linux
/dev/sda7 52436992 69214207 16777216 8G 83 Linux
/dev/sda8 69216256 88090623 18874368 9G 83 Linux
/dev/sda9 88092672 109064191 20971520 10G 83 Linux
/dev/sda10 109066240 132134911 23068672 11G 83 Linux
Wipe the partitions:
[root@bcache-test cedric]# wipefs -a /dev/sda5
[root@bcache-test cedric]# wipefs -a /dev/sda6
[root@bcache-test cedric]# wipefs -a /dev/sda7
[root@bcache-test cedric]# wipefs -a /dev/sda8
[root@bcache-test cedric]# wipefs -a /dev/sda9
[root@bcache-test cedric]# wipefs -a /dev/sda10
Check for old bcache devices:
# ls /dev/bca*
ls: cannot access '/dev/bca*': No such file or directory
1. Format the backing devices:
root@bcache-test cedric]# make-bcache -B /dev/sda8
Name /dev/sda8
Label
Type data
UUID: 3bae90e9-d2c5-4d62-96a8-07dc1cb1440d
Set UUID: 271e657c-2a3a-417e-9a8d-430129095a47
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 4196.603418] bcache: register_bdev() registered backing device sda8
# make-bcache -B /dev/sda9
Name /dev/sda9
Label
Type data
UUID: 28ce8fa5-6918-4637-b612-374cf75635c2
Set UUID: c12faa51-6042-4b46-8448-ee4a5acd0e8e
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 4269.009004] bcache: register_bdev() registered backing device sda9
[root@bcache-test cedric]# make-bcache -B /dev/sda10
Name /dev/sda10
Label
Type data
UUID: 4bca4225-4143-41af-b0e2-2ca2700e4931
Set UUID: 5442ccbe-6f12-40df-a309-30f66191773f
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 4285.151623] bcache: register_bdev() registered backing device sda10
Now 3 bcache devices are created:
[root@bcache-test cedric]# ls -l /dev/bca*
brw-rw---- 1 root disk 254, 0 Jan 2 20:46 /dev/bcache0
brw-rw---- 1 root disk 254, 128 Jan 2 20:47 /dev/bcache1
brw-rw---- 1 root disk 254, 256 Jan 2 20:47 /dev/bcache2
/dev/bcache:
total 0
drwxr-xr-x 2 root root 100 Jan 2 20:47 by-uuid
2. Format the cache devices:
[root@bcache-test cedric]# make-bcache -C /dev/sda5
Name /dev/sda5
Label
Type cache
UUID: 6e7610b2-5f28-4699-aed0-ffd78c1db77d
Set UUID: 924e51c2-9b49-4802-b87f-b379cd9a06a6
version: 0
nbuckets: 12288
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
[ 4505.224494] bcache: run_cache_set() invalidating existing data
[ 4505.265255] bcache: register_cache() registered cache device sda5
[root@bcache-test cedric]# make-bcache -C /dev/sda6
Name /dev/sda6
Label
Type cache
UUID: 4ff65c49-e225-4975-aac1-1be7da22d434
Set UUID: ab10dafe-e8a4-4aa9-89f7-1a0575659f06
version: 0
nbuckets: 14336
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
[ 4558.676081] bcache: run_cache_set() invalidating existing data
[ 4558.724051] bcache: register_cache() registered cache device sda6
[root@bcache-test cedric]# make-bcache -C /dev/sda7
Name /dev/sda7
Label
Type cache
UUID: 02d4fe5e-5c01-4c94-9e89-3a97b01118b2
Set UUID: 31350209-3c13-4e78-b782-2c16e9d52e6a
version: 0
nbuckets: 16384
block_size_in_sectors: 1
bucket_size_in_sectors: 1024
nr_in_set: 1
nr_this_dev: 0
first_bucket: 1
[ 4583.036101] bcache: run_cache_set() invalidating existing data
[ 4583.077035] bcache: register_cache() registered cache device sda7
Register the cache devices against the backing devices.
# echo 924e51c2-9b49-4802-b87f-b379cd9a06a6 > /sys/block/bcache0/bcache/attach
[ 5008.251480] bcache: bch_cached_dev_run() cached dev sda8 is running already
[ 5008.251500] bcache: bch_cached_dev_attach() Caching sda8 as bcache0 on set 924e51c2-9b49-4802-b87f-b379cd9a06a6
# echo ab10dafe-e8a4-4aa9-89f7-1a0575659f06 > /sys/block/bcache1/bcache/attach
[ 5037.562635] bcache: bch_cached_dev_run() cached dev sda9 is running already
[ 5037.562648] bcache: bch_cached_dev_attach() Caching sda9 as bcache1 on set ab10dafe-e8a4-4aa9-89f7-1a0575659f06
# echo 31350209-3c13-4e78-b782-2c16e9d52e6a > /sys/block/bcache2/bcache/attach
[ 5061.280096] bcache: bch_cached_dev_run() cached dev sda10 is running already
[ 5061.280110] bcache: bch_cached_dev_attach() Caching sda10 as bcache2 on set 31350209-3c13-4e78-b782-2c16e9d52e6a
Format /dev/bcache0 as backing devices for the read cache fails:
make-bcache -B /dev/bcache0
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
[ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 5187.054670] Call Trace:
[ 5187.054681] dump_stack+0x6b/0x83
[ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
[ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
[ 5187.054699] kobject_add_internal+0xab/0x2f0
[ 5187.054703] kobject_add+0x98/0xd0
[ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
[ 5187.054723] register_bdev+0x337/0x360 [bcache]
[ 5187.054735] register_bcache+0x43c/0x910 [bcache]
[ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
[ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
[ 5187.054753] kernfs_fop_write+0xce/0x1b0
[ 5187.054756] vfs_write+0xc7/0x210
[ 5187.054759] ksys_write+0x67/0xe0
[ 5187.054763] do_syscall_64+0x33/0x40
[ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 5187.054769] RIP: 0033:0x7fbd1edb3f67
[ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
[ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
[ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
[ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
[ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
[ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
[ 5187.054794] bcache: register_bcache() error : failed to register device
[ 5187.054963] bcache: bcache_device_free() bcache3 stopped
I expected this command to create /dev/bcache3
My versions:
# uname -a
Linux bcache-test 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
# pacman -Q bcache-tools
bcache-tools 1.1-1
I've used this guide for inspiration:
https://wiki.archlinux.org/index.php/Bcache
---
Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
^ permalink raw reply [flat|nested] 5+ messages in thread
* make-bcache -B /dev/bcache0 fails, Can't stack bcache on top of bcache
@ 2021-01-02 19:38
2021-01-04 2:57 ` Dongsheng Yang
0 siblings, 1 reply; 5+ messages in thread
From: @ 2021-01-02 19:38 UTC (permalink / raw)
To: , linux-bcache@vger.kernel.org
I can't stack bcache on top of bcache, This command fails:
# make-bcache -B /dev/bcache0
Name /dev/bcache0
Label
Type data
UUID: 758be840-f232-44dd-8e56-83fb55ba20ea
Set UUID: ddbcc87b-d25d-483f-8874-49b80ba26472
version: 1
block_size_in_sectors: 1
data_offset_in_sectors: 16
[ 5187.054662] sysfs: cannot create duplicate filename '/devices/virtual/block/bcache0/bcache'
[ 5187.054667] CPU: 2 PID: 567 Comm: bcache-register Not tainted 5.9.14-arch1-1 #1
[ 5187.054669] Hardware name: Hewlett-Packard HP Compaq 8200 Elite CMT PC/1494, BIOS J01 v02.28 03/24/2015
[ 5187.054670] Call Trace:
[ 5187.054681] dump_stack+0x6b/0x83
[ 5187.054686] sysfs_warn_dup.cold+0x17/0x24
[ 5187.054694] sysfs_create_dir_ns+0xc6/0xe0
[ 5187.054699] kobject_add_internal+0xab/0x2f0
[ 5187.054703] kobject_add+0x98/0xd0
[ 5187.054707] ? blk_queue_write_cache+0x2f/0x60
[ 5187.054723] register_bdev+0x337/0x360 [bcache]
[ 5187.054735] register_bcache+0x43c/0x910 [bcache]
[ 5187.054740] ? kernfs_fop_write+0xce/0x1b0
[ 5187.054750] ? register_cache+0x1290/0x1290 [bcache]
[ 5187.054753] kernfs_fop_write+0xce/0x1b0
[ 5187.054756] vfs_write+0xc7/0x210
[ 5187.054759] ksys_write+0x67/0xe0
[ 5187.054763] do_syscall_64+0x33/0x40
[ 5187.054766] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 5187.054769] RIP: 0033:0x7fbd1edb3f67
[ 5187.054772] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 5187.054774] RSP: 002b:00007fff335fa418 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 5187.054777] RAX: ffffffffffffffda RBX: 000000000000000d RCX: 00007fbd1edb3f67
[ 5187.054778] RDX: 000000000000000d RSI: 0000562bf3f902a0 RDI: 0000000000000003
[ 5187.054780] RBP: 0000562bf3f902a0 R08: 00007fbd1ee4a040 R09: 00007fbd1ee4a0c0
[ 5187.054781] R10: 00007fbd1ee49fc0 R11: 0000000000000246 R12: 000000000000000d
[ 5187.054782] R13: 00007fff335fa4a0 R14: 000000000000000d R15: 00007fbd1ee86720
[ 5187.054787] kobject_add_internal failed for bcache with -EEXIST, don't try to register things with the same name in the same directory.
[ 5187.054789] bcache: register_bdev() error bcache0: error creating kobject
[ 5187.054794] bcache: register_bcache() error : failed to register device
[ 5187.054963] bcache: bcache_device_free() bcache3 stopped
I expected this command to create a new /dev/bcacheX node. Instead it tries to overwrite /devices/virtual/block/bcache0/bcache
My versions:
# uname -a
Linux bcache-test 5.9.14-arch1-1 #1 SMP PREEMPT Sat, 12 Dec 2020 14:37:12 +0000 x86_64 GNU/Linux
# pacman -Q bcache-tools
bcache-tools 1.1-1
I've used this guide for inspiration:
https://wiki.archlinux.org/index.php/Bcache
---
Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-01-04 2:58 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-02 19:18 make-bcache -B /dev/bcache0 fails, Can't stack bcache on top of bcache
2021-01-02 19:38
2021-01-04 2:57 ` Dongsheng Yang
2021-01-02 21:34
2021-01-02 21:36
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.