* How to create bcachefs? @ 2016-08-19 18:07 marcin 2016-08-24 6:52 ` Kent Overstreet 0 siblings, 1 reply; 13+ messages in thread From: marcin @ 2016-08-19 18:07 UTC (permalink / raw) To: linux-bcache Hi! I'm spamming here with simple questions because it looks that documentation doesn't follow current state of bcachefs. I've upgraded bcache-tools to b0f08fc1e30daafb274181234dae3408fb2eb69b and linux-bcache to 9c79c00b0d02f0da5e01ad5cd1bf4ae2ce421d5d. And now... 1. How to create simple bcachefs on one disk, without any cache? # ./bcache format --meta_replicas=2 /dev/sdd1 Please supply a device Help displays: # ./bcache format --help Usage: bcache format [OPTION]... <list of devices> Create a new bcache volume from one or more devices Options: -C, --cache=dev Format a cache device -w, --block=size block size -n, --btree_node=size Btree node size, default 256k --metadata_csum_type=(none|crc32c|crc64) Checksum type --data_csum_type=(none|crc32c|crc64) Checksum type --compression_type=(none|gzip) Compression type --encrypted enable encryption --meta_replicas=# number of metadata replicas --data_replicas=# number of data replicas --error_action=(continue|readonly|panic) Action to take on filesystem error -l, --label=label label --uuid=uuid filesystem UUID --fs_size=size Size of filesystem on device -b, --bucket=size bucket size -t, --tier=# tier of subsequent devices --discard Enable discards -q, --quiet reduce output to errors only -v, --verbose increase output to include informational messages --help display this help and exit --version output version information and exit Report bugs to <linux-bcache@vger.kernel.org> I don't know what is correct syntax to format block device. 2. How to create and mount bcachefs witch cache device? # ./bcache format --compression_type=lz4 --meta_replicas=2 /dev/sdd1 -C /dev/sde1 UUID: 2cd51dcb-6af3-4e92-8b01-64bac27300b2 Set UUID: fd005c93-26f4-4da3-a991-18630ea37997 version: 6 nbuckets: 20480 block_size: 1 bucket_size: 1024 nr_in_set: 1 nr_this_dev: 0 first_bucket: 3 and what next? # mount /dev/sdd1 /mnt/test/ # # ls -l /sys/fs/bcache/33802bf3-abe4-4d85-bd44-380c1fa9c3d0/|grep cache -r--r--r-- 1 root root 4096 sie 19 20:02 btree_cache_size lrwxrwxrwx 1 root root 0 sie 19 20:02 cache0 -> ../../../devices/pci0000:00/0000:00:1f.2/ata6/host5/target5:0:0/5:0:0:0/block/sdd/sdd1/bcache -r--r--r-- 1 root root 4096 sie 19 20:02 cache_available_percent It looks for me that cache device /dev/sde1 isn't used. How to use /dev/sde1 as a cache device? 3. Tiering. Just simple: how to?:) Thanks in advance for tips. Marcin ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-19 18:07 How to create bcachefs? marcin @ 2016-08-24 6:52 ` Kent Overstreet 2016-08-24 21:21 ` marcin 0 siblings, 1 reply; 13+ messages in thread From: Kent Overstreet @ 2016-08-24 6:52 UTC (permalink / raw) To: marcin; +Cc: linux-bcache On Fri, Aug 19, 2016 at 08:07:31PM +0200, marcin@mejor.pl wrote: > Hi! > I'm spamming here with simple questions because it looks that documentation > doesn't follow current state of bcachefs. > > I've upgraded bcache-tools to b0f08fc1e30daafb274181234dae3408fb2eb69b and > linux-bcache to 9c79c00b0d02f0da5e01ad5cd1bf4ae2ce421d5d. > And now... > > 1. How to create simple bcachefs on one disk, without any cache? > # ./bcache format --meta_replicas=2 /dev/sdd1 > Please supply a device > > Help displays: > # ./bcache format --help > Usage: bcache format [OPTION]... <list of devices> > Create a new bcache volume from one or more devices > > Options: > -C, --cache=dev Format a cache device > -w, --block=size block size > -n, --btree_node=size Btree node size, default 256k > --metadata_csum_type=(none|crc32c|crc64) > Checksum type > --data_csum_type=(none|crc32c|crc64) > Checksum type > --compression_type=(none|gzip) > Compression type > --encrypted enable encryption > --meta_replicas=# number of metadata replicas > --data_replicas=# number of data replicas > --error_action=(continue|readonly|panic) > Action to take on filesystem error > -l, --label=label label > --uuid=uuid filesystem UUID > --fs_size=size Size of filesystem on device > -b, --bucket=size bucket size > -t, --tier=# tier of subsequent devices > --discard Enable discards > -q, --quiet reduce output to errors only > -v, --verbose increase output to include informational > messages > --help display this help and exit > --version output version information and exit > > Report bugs to <linux-bcache@vger.kernel.org> > > I don't know what is correct syntax to format block device. > > 2. How to create and mount bcachefs witch cache device? > # ./bcache format --compression_type=lz4 --meta_replicas=2 /dev/sdd1 -C > /dev/sde1 Hey, sorry for the long delay, been sick past several days. The way arguments were passed to bcache format was a holdover from old style make-bcache, and didn't make much sense for bcachefs - -C was used for all devices, and --tier specifies fast devices and slow devices. I finally got around to redoing the option parsing so we don't need the -C argument today - update your bcache tools, and the command you want is now: bcache format --tier 0 /dev/sde1 --tier 1 /dev/sdd1 (assuming sde1 is your fast device and sdd1 is your slow device). Replication is not yet supported (it's around 90% implemented, but I haven't worked on it in ages and the tests haven't been covering it so you'll likely hit bugs if you try it). Compression still isn't _quite_ done, I reverted the accounting until I fix an issue with copygc. You can enable it and nothing will break, but it won't actually let you store more data in your fs than if it was uncompressed. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-24 6:52 ` Kent Overstreet @ 2016-08-24 21:21 ` marcin 2016-08-24 23:12 ` Eric Wheeler ` (2 more replies) 0 siblings, 3 replies; 13+ messages in thread From: marcin @ 2016-08-24 21:21 UTC (permalink / raw) To: linux-bcache W dniu 24.08.2016 o 08:52, Kent Overstreet pisze: [...] Hi! > Hey, sorry for the long delay, been sick past several days. I was afraid that I'm asking to easy question:) And I hope everything is fine with you now! > The way arguments > were passed to bcache format was a holdover from old style make-bcache, > and > didn't make much sense for bcachefs - -C was used for all devices, and > --tier > specifies fast devices and slow devices. > > I finally got around to redoing the option parsing so we don't need the > -C > argument today - update your bcache tools, and the command you want is > now: > > bcache format --tier 0 /dev/sde1 --tier 1 /dev/sdd1 > > (assuming sde1 is your fast device and sdd1 is your slow device). Does it means that cache is unavailable and only tiering will be in bcachefs? And... How to mount tiered FS? When I pass one device in mount I'm getting: bcache: bch_open_as_blockdevs() register_cache_set err insufficient devices > Replication is not yet supported (it's around 90% implemented, but I > haven't > worked on it in ages and the tests haven't been covering it so you'll > likely hit > bugs if you try it). If you tell me how to turn it on I'll try it. In latest bcache-tools this option disappeared. Turning on using `echo 2 > /sys/fs/bcache/<uid>/options/{data_replicas,metadata_replicas}` doesn't work. > Compression still isn't _quite_ done, I reverted the accounting until I > fix an > issue with copygc. You can enable it and nothing will break, but it > won't > actually let you store more data in your fs than if it was > uncompressed. What means option in /sys/fs/..../options/posix_acl? I've got zero value but setfacl and getfacl shows that acl works (in theory, I didn't check if ACL really works). I also have feature request: it would be nice to see which file (with path) has checksum error, now it looks like this: Aug 24 17:15:23 localhost kernel: [ 4692.929060] bcache (dm-10): IO error on dm-10 for checksum error I can't help with writing code but I can try to help testing bcachefs. I'm especially interested in using: - cache - tiering - compression - erasure encoding (I know that it isn't started) Thank you, Marcin ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-24 21:21 ` marcin @ 2016-08-24 23:12 ` Eric Wheeler 2016-08-24 23:52 ` Kent Overstreet 2016-08-24 23:50 ` Kent Overstreet 2016-08-25 0:03 ` Christopher James Halse Rogers 2 siblings, 1 reply; 13+ messages in thread From: Eric Wheeler @ 2016-08-24 23:12 UTC (permalink / raw) To: marcin; +Cc: linux-bcache On Wed, 24 Aug 2016, marcin@mejor.pl wrote: > W dniu 24.08.2016 o 08:52, Kent Overstreet pisze: > [...] > > Hi! > > Hey, sorry for the long delay, been sick past several days. > > I was afraid that I'm asking to easy question:) And I hope everything is > fine with you now! > > > > The way arguments > > were passed to bcache format was a holdover from old style make-bcache, and > > didn't make much sense for bcachefs - -C was used for all devices, and > > --tier > > specifies fast devices and slow devices. > > > > I finally got around to redoing the option parsing so we don't need the -C > > argument today - update your bcache tools, and the command you want is now: > > > > bcache format --tier 0 /dev/sde1 --tier 1 /dev/sdd1 Neat! How many performance tiers are (will be) supported? Arbitrary? -- Eric Wheeler > > > > (assuming sde1 is your fast device and sdd1 is your slow device). > > Does it means that cache is unavailable and only tiering will be in bcachefs? > And... How to mount tiered FS? When I pass one device in mount I'm getting: > bcache: bch_open_as_blockdevs() register_cache_set err insufficient devices > > > > Replication is not yet supported (it's around 90% implemented, but I haven't > > worked on it in ages and the tests haven't been covering it so you'll likely > > hit > > bugs if you try it). > > If you tell me how to turn it on I'll try it. In latest bcache-tools > this option disappeared. Turning on using `echo 2 > > /sys/fs/bcache/<uid>/options/{data_replicas,metadata_replicas}` doesn't work. > > > Compression still isn't _quite_ done, I reverted the accounting until I fix > > an > > issue with copygc. You can enable it and nothing will break, but it won't > > actually let you store more data in your fs than if it was uncompressed. > > > What means option in /sys/fs/..../options/posix_acl? I've got zero value but > setfacl and getfacl shows that acl works (in theory, I didn't check if ACL > really works). > > I also have feature request: it would be nice to see which file (with path) > has > checksum error, now it looks like this: > Aug 24 17:15:23 localhost kernel: [ 4692.929060] bcache (dm-10): IO > error on dm-10 for checksum error > > I can't help with writing code but I can try to help testing bcachefs. I'm > especially interested in using: > - cache > - tiering > - compression > - erasure encoding (I know that it isn't started) > > > > Thank you, > Marcin > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-24 23:12 ` Eric Wheeler @ 2016-08-24 23:52 ` Kent Overstreet 0 siblings, 0 replies; 13+ messages in thread From: Kent Overstreet @ 2016-08-24 23:52 UTC (permalink / raw) To: Eric Wheeler; +Cc: marcin, linux-bcache On Wed, Aug 24, 2016 at 04:12:22PM -0700, Eric Wheeler wrote: > On Wed, 24 Aug 2016, marcin@mejor.pl wrote: > > > W dniu 24.08.2016 o 08:52, Kent Overstreet pisze: > > [...] > > > > Hi! > > > Hey, sorry for the long delay, been sick past several days. > > > > I was afraid that I'm asking to easy question:) And I hope everything is > > fine with you now! > > > > > > > The way arguments > > > were passed to bcache format was a holdover from old style make-bcache, and > > > didn't make much sense for bcachefs - -C was used for all devices, and > > > --tier > > > specifies fast devices and slow devices. > > > > > > I finally got around to redoing the option parsing so we don't need the -C > > > argument today - update your bcache tools, and the command you want is now: > > > > > > bcache format --tier 0 /dev/sde1 --tier 1 /dev/sdd1 > > Neat! How many performance tiers are (will be) supported? Arbitrary? Currently the code just handles two, but the on disk format allows for up to 16. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-24 21:21 ` marcin 2016-08-24 23:12 ` Eric Wheeler @ 2016-08-24 23:50 ` Kent Overstreet 2016-08-25 0:03 ` Christopher James Halse Rogers 2 siblings, 0 replies; 13+ messages in thread From: Kent Overstreet @ 2016-08-24 23:50 UTC (permalink / raw) To: marcin; +Cc: linux-bcache On Wed, Aug 24, 2016 at 11:21:02PM +0200, marcin@mejor.pl wrote: > W dniu 24.08.2016 o 08:52, Kent Overstreet pisze: > [...] > > Hi! > > Hey, sorry for the long delay, been sick past several days. > > I was afraid that I'm asking to easy question:) And I hope everything is > fine with you now! > > > > The way arguments > > were passed to bcache format was a holdover from old style make-bcache, > > and > > didn't make much sense for bcachefs - -C was used for all devices, and > > --tier > > specifies fast devices and slow devices. > > > > I finally got around to redoing the option parsing so we don't need the > > -C > > argument today - update your bcache tools, and the command you want is > > now: > > > > bcache format --tier 0 /dev/sde1 --tier 1 /dev/sdd1 > > > > (assuming sde1 is your fast device and sdd1 is your slow device). > > Does it means that cache is unavailable and only tiering will be in > bcachefs? > And... How to mount tiered FS? When I pass one device in mount I'm getting: > bcache: bch_open_as_blockdevs() register_cache_set err insufficient devices you should be able to do mount -t bcache /dev/sde1:/dev/sdd1 /mnt or, there's incremental assembly - that's going to be handled by the bcache tool eventually, but for now you can register each device via /sys/fs/bcache/register, and then mount one of the block devices. > > Replication is not yet supported (it's around 90% implemented, but I > > haven't > > worked on it in ages and the tests haven't been covering it so you'll > > likely hit > > bugs if you try it). > > If you tell me how to turn it on I'll try it. In latest bcache-tools > this option disappeared. Turning on using `echo 2 > > /sys/fs/bcache/<uid>/options/{data_replicas,metadata_replicas}` doesn't > work. Wouldn't be any point, you'd undoubtedly hit bugs that I don't have time to work on yet. Current priorities are finishing encryption and compression... > > Compression still isn't _quite_ done, I reverted the accounting until I > > fix an > > issue with copygc. You can enable it and nothing will break, but it > > won't > > actually let you store more data in your fs than if it was uncompressed. > > > What means option in /sys/fs/..../options/posix_acl? I've got zero value but > setfacl and getfacl shows that acl works (in theory, I didn't check if ACL > really works). hmm, option must be buggered. I'll have a look. > I also have feature request: it would be nice to see which file (with path) > has > checksum error, now it looks like this: > Aug 24 17:15:23 localhost kernel: [ 4692.929060] bcache (dm-10): IO > error on dm-10 for checksum error that's a good idea - though it'd take some plumbing to get a path, and we don't even always have a path (may have just an inode number). I'll make a note to have a look. > I can't help with writing code but I can try to help testing bcachefs. I'm > especially interested in using: > - cache > - tiering > - compression > - erasure encoding (I know that it isn't started) I am thinking about working on replication and erasure coding after I finish what's currently lined up, erasure coding should be a fun project. Lots to do... ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-24 21:21 ` marcin 2016-08-24 23:12 ` Eric Wheeler 2016-08-24 23:50 ` Kent Overstreet @ 2016-08-25 0:03 ` Christopher James Halse Rogers 2016-08-25 9:21 ` Marcin Mirosław 2 siblings, 1 reply; 13+ messages in thread From: Christopher James Halse Rogers @ 2016-08-25 0:03 UTC (permalink / raw) To: marcin; +Cc: linux-bcache On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: > W dniu 24.08.2016 o 08:52, Kent Overstreet pisze: > [...] > > Hi! >> Hey, sorry for the long delay, been sick past several days. > > I was afraid that I'm asking to easy question:) And I hope everything > is > fine with you now! > > >> The way arguments >> were passed to bcache format was a holdover from old style >> make-bcache, \x7fand >> didn't make much sense for bcachefs - -C was used for all devices, >> and \x7f--tier >> specifies fast devices and slow devices. >> >> I finally got around to redoing the option parsing so we don't need >> the \x7f-C >> argument today - update your bcache tools, and the command you want >> is \x7fnow: >> >> bcache format --tier 0 /dev/sde1 --tier 1 /dev/sdd1 >> >> (assuming sde1 is your fast device and sdd1 is your slow device). > > Does it means that cache is unavailable and only tiering will be in > bcachefs? > And... How to mount tiered FS? When I pass one device in mount I'm > getting: > bcache: bch_open_as_blockdevs() register_cache_set err insufficient > devices Tiering gets you all the advantages of caching, plus you can (with some effort) have the combined filesystem size be the sum of the SSD + HDD capacities, rather than the capacity be determined solely by the capacity of the slow tier (this is not currently the case for bcachefs). To mount a tiered FS you can either pass all the relevant devices to mount, like: mount -t bcache /dev/sda:/dev/sdb:/dev/sdc /mountpoint (I haven't tested this recently) or ensure that all the relevant devices have been registered with bcache, like so: echo /dev/sda | sudo tee /sys/fs/bcache/register_quiet echo /dev/sdb | sudo tee /sys/fs/bcache/register_quiet echo /dev/sdc | sudo tee /sys/fs/bcache/register_quiet mount -t bcache /dev/sdb /mountpoint (you can use any member of the set as the device to mount in this case). ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-25 0:03 ` Christopher James Halse Rogers @ 2016-08-25 9:21 ` Marcin Mirosław 2016-08-25 11:09 ` Christopher James Halse Rogers 0 siblings, 1 reply; 13+ messages in thread From: Marcin Mirosław @ 2016-08-25 9:21 UTC (permalink / raw) To: linux-bcache W dniu 25.08.2016 o 02:03, Christopher James Halse Rogers pisze: Hi! > On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: [...] >> Does it means that cache is unavailable and only tiering will be in >> bcachefs? >> And... How to mount tiered FS? When I pass one device in mount I'm >> getting: >> bcache: bch_open_as_blockdevs() register_cache_set err insufficient >> devices > > Tiering gets you all the advantages of caching, plus you can (with some > effort) have the combined filesystem size be the sum of the SSD + HDD > capacities, rather than the capacity be determined solely by the > capacity of the slow tier (this is not currently the case for bcachefs). I think that cacheing has at least such advantages over tiering: - allow fast read and write to files compressed with slow alghoritm (gzip) - can be optimized for using SSD drives > To mount a tiered FS you can either pass all the relevant devices to > mount, like: > mount -t bcache /dev/sda:/dev/sdb:/dev/sdc /mountpoint > (I haven't tested this recently) or ensure that all the relevant devices > have been registered with bcache, like so: > echo /dev/sda | sudo tee /sys/fs/bcache/register_quiet > echo /dev/sdb | sudo tee /sys/fs/bcache/register_quiet > echo /dev/sdc | sudo tee /sys/fs/bcache/register_quiet > mount -t bcache /dev/sdb /mountpoint > (you can use any member of the set as the device to mount in this case). Thanks for tip, good to know other way to mount. Marcin ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-25 9:21 ` Marcin Mirosław @ 2016-08-25 11:09 ` Christopher James Halse Rogers 2016-08-25 12:11 ` Marcin Mirosław 0 siblings, 1 reply; 13+ messages in thread From: Christopher James Halse Rogers @ 2016-08-25 11:09 UTC (permalink / raw) To: Marcin Mirosław; +Cc: linux-bcache On Thu, Aug 25, 2016 at 7:21 PM, Marcin MirosÅaw <marcin@mejor.pl> wrote: > W dniu 25.08.2016 o 02:03, Christopher James Halse Rogers pisze: > > Hi! > >> On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: > [...] >>> Does it means that cache is unavailable and only tiering will be in >>> bcachefs? >>> And... How to mount tiered FS? When I pass one device in mount I'm >>> getting: >>> bcache: bch_open_as_blockdevs() register_cache_set err >>> insufficient >>> devices >> >> Tiering gets you all the advantages of caching, plus you can (with >> some >> effort) have the combined filesystem size be the sum of the SSD + >> HDD >> capacities, rather than the capacity be determined solely by the >> capacity of the slow tier (this is not currently the case for >> bcachefs). > > I think that cacheing has at least such advantages over tiering: > - allow fast read and write to files compressed with slow alghoritm > (gzip) I think this is getting into āwhat should we call this thingā arguments. A naive cache is just going to promote the gzipped data to the fast storage. On the other end, there's nothing much preventing a sophisticated tiering system from compressing/decompressing as a part of tier demotion/promotion. > - can be optimized for using SSD drives It's not clear to me how? Tiering and caching are doing the same sort of things. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-25 11:09 ` Christopher James Halse Rogers @ 2016-08-25 12:11 ` Marcin Mirosław 2016-08-26 1:06 ` Christopher James Halse Rogers 0 siblings, 1 reply; 13+ messages in thread From: Marcin Mirosław @ 2016-08-25 12:11 UTC (permalink / raw) To: linux-bcache W dniu 25.08.2016 o 13:09, Christopher James Halse Rogers pisze: > > > On Thu, Aug 25, 2016 at 7:21 PM, Marcin MirosÅaw <marcin@mejor.pl> wrote: >> W dniu 25.08.2016 o 02:03, Christopher James Halse Rogers pisze: >> >> Hi! >> >>> On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: >> [...] >>>> Does it means that cache is unavailable and only tiering will be in >>>> bcachefs? >>>> And... How to mount tiered FS? When I pass one device in mount I'm >>>> getting: >>>> bcache: bch_open_as_blockdevs() register_cache_set err insufficient >>>> devices >>> >>> Tiering gets you all the advantages of caching, plus you can (with some >>> effort) have the combined filesystem size be the sum of the SSD + HDD >>> capacities, rather than the capacity be determined solely by the >>> capacity of the slow tier (this is not currently the case for >>> bcachefs). >> >> I think that cacheing has at least such advantages over tiering: >> - allow fast read and write to files compressed with slow alghoritm >> (gzip) > > I think this is getting into āwhat should we call this thingā > arguments. A naive cache is just going to promote the gzipped data to > the fast storage. On the other end, there's nothing much preventing a > sophisticated tiering system from compressing/decompressing as a part of > tier demotion/promotion. When (de|re)compressing would be part of demotion or promotion then I agree, cache device doesn't have advantages. >> - can be optimized for using SSD drives > > It's not clear to me how? Tiering and caching are doing the same sort of > things. I don't know how bcachefs handle SSD drives. SSD are faster in sequential write, I'm not sure if it could be achieved using tier device. Using cache device on disk format can be different than using in bcachefs and can turn random writes into sequential writes. Marcin ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-25 12:11 ` Marcin Mirosław @ 2016-08-26 1:06 ` Christopher James Halse Rogers 2016-08-26 1:48 ` Eric Wheeler 2016-08-26 8:23 ` Marcin Mirosław 0 siblings, 2 replies; 13+ messages in thread From: Christopher James Halse Rogers @ 2016-08-26 1:06 UTC (permalink / raw) To: Marcin Mirosław; +Cc: linux-bcache On Thu, Aug 25, 2016 at 10:11 PM, Marcin Mirosław <marcin@mejor.pl> wrote: > W dniu 25.08.2016 o 13:09, Christopher James Halse Rogers pisze: >> >> >> On Thu, Aug 25, 2016 at 7:21 PM, Marcin MirosÅ‚aw >> <marcin@mejor.pl> wrote: >>> W dniu 25.08.2016 o 02:03, Christopher James Halse Rogers pisze: >>> >>> Hi! >>> >>>> On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: >>> [...] >>>>> Does it means that cache is unavailable and only tiering will >>>>> be in >>>>> bcachefs? >>>>> And... How to mount tiered FS? When I pass one device in mount >>>>> I'm >>>>> getting: >>>>> bcache: bch_open_as_blockdevs() register_cache_set err >>>>> insufficient >>>>> devices >>>> >>>> Tiering gets you all the advantages of caching, plus you can >>>> (with some >>>> effort) have the combined filesystem size be the sum of the SSD >>>> + HDD >>>> capacities, rather than the capacity be determined solely by the >>>> capacity of the slow tier (this is not currently the case for >>>> bcachefs). >>> >>> I think that cacheing has at least such advantages over tiering: >>> - allow fast read and write to files compressed with slow alghoritm >>> (gzip) >> >> I think this is getting into ā€œwhat should we call this >> thingā€ >> arguments. A naive cache is just going to promote the gzipped data >> to >> the fast storage. On the other end, there's nothing much preventing >> a >> sophisticated tiering system from compressing/decompressing as a >> part of >> tier demotion/promotion. > > When (de|re)compressing would be part of demotion or promotion then I > agree, cache device doesn't have advantages. You also don't get (de|re)compression for free in a caching strategy depending on where you put your cache. For example, a traditional bcache cache backing a compressed filesystem will only ever see compressed data. > >>> - can be optimized for using SSD drives >> >> It's not clear to me how? Tiering and caching are doing the same >> sort of >> things. > I don't know how bcachefs handle SSD drives. SSD are faster in > sequential write, I'm not sure if it could be achieved using tier > device. Using cache device on disk format can be different than using > in > bcachefs and can turn random writes into sequential writes. I believe you mean *HDDs* are faster in sequential write? This is exactly what tiering gets you - all writes go to the fast, SSD storage¹, and then data are migrated off to the slower HDD in big, sequential chunks. ¹: Indeed, IIRC this is the only dataloss bug I've found in bcachefs - I wrote too much data too fast, the migration from tier 0 to tier 1 couldn't keep up, and tier 0 filled up to the point where it could no longer write necessary journal entries, resulting in the filesystem being unmountable. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-26 1:06 ` Christopher James Halse Rogers @ 2016-08-26 1:48 ` Eric Wheeler 2016-08-26 8:23 ` Marcin Mirosław 1 sibling, 0 replies; 13+ messages in thread From: Eric Wheeler @ 2016-08-26 1:48 UTC (permalink / raw) To: Christopher James Halse Rogers; +Cc: Marcin Mirosław, linux-bcache [-- Attachment #1: Type: TEXT/PLAIN, Size: 3348 bytes --] On Fri, 26 Aug 2016, Christopher James Halse Rogers wrote: > > > On Thu, Aug 25, 2016 at 10:11 PM, Marcin Mirosław <marcin@mejor.pl> wrote: > > W dniu 25.08.2016 o 13:09, Christopher James Halse Rogers pisze: > > > > > > > > > On Thu, Aug 25, 2016 at 7:21 PM, Marcin MirosÅ‚aw <marcin@mejor.pl> wrote: > > > > W dniu 25.08.2016 o 02:03, Christopher James Halse Rogers pisze: > > > > > > > > Hi! > > > > > > > > > On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: > > > > [...] > > > > > > Does it means that cache is unavailable and only tiering will be > > > > > > in > > > > > > bcachefs? > > > > > > And... How to mount tiered FS? When I pass one device in mount > > > > > > I'm > > > > > > getting: > > > > > > bcache: bch_open_as_blockdevs() register_cache_set err > > > > > > insufficient > > > > > > devices > > > > > > > > > > Tiering gets you all the advantages of caching, plus you can (with > > > > > some > > > > > effort) have the combined filesystem size be the sum of the SSD + > > > > > HDD > > > > > capacities, rather than the capacity be determined solely by the > > > > > capacity of the slow tier (this is not currently the case for > > > > > bcachefs). > > > > > > > > I think that cacheing has at least such advantages over tiering: > > > > - allow fast read and write to files compressed with slow alghoritm > > > > (gzip) > > > > > > I think this is getting into ā€œwhat should we call this thingā€? > > > arguments. A naive cache is just going to promote the gzipped data to > > > the fast storage. On the other end, there's nothing much preventing a > > > sophisticated tiering system from compressing/decompressing as a part of > > > tier demotion/promotion. > > > > When (de|re)compressing would be part of demotion or promotion then I > > agree, cache device doesn't have advantages. > > You also don't get (de|re)compression for free in a caching strategy depending > on where you put your cache. For example, a traditional bcache cache backing a > compressed filesystem will only ever see compressed data. Nor crypto. Best to compress->encrypt->store. -- Eric Wheeler > > > > > > > - can be optimized for using SSD drives > > > > > > It's not clear to me how? Tiering and caching are doing the same sort of > > > things. > > I don't know how bcachefs handle SSD drives. SSD are faster in > > sequential write, I'm not sure if it could be achieved using tier > > device. Using cache device on disk format can be different than using in > > bcachefs and can turn random writes into sequential writes. > > I believe you mean *HDDs* are faster in sequential write? This is exactly what > tiering gets you - all writes go to the fast, SSD storage¹, and then data are > migrated off to the slower HDD in big, sequential chunks. > > ¹: Indeed, IIRC this is the only dataloss bug I've found in bcachefs - I wrote > too much data too fast, the migration from tier 0 to tier 1 couldn't keep up, > and tier 0 filled up to the point where it could no longer write necessary > journal entries, resulting in the filesystem being unmountable. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: How to create bcachefs? 2016-08-26 1:06 ` Christopher James Halse Rogers 2016-08-26 1:48 ` Eric Wheeler @ 2016-08-26 8:23 ` Marcin Mirosław 1 sibling, 0 replies; 13+ messages in thread From: Marcin Mirosław @ 2016-08-26 8:23 UTC (permalink / raw) To: Christopher James Halse Rogers; +Cc: linux-bcache W dniu 26.08.2016 o 03:06, Christopher James Halse Rogers pisze: Hi! > On Thu, Aug 25, 2016 at 10:11 PM, Marcin Mirosław <marcin@mejor.pl> wrote: >> W dniu 25.08.2016 o 13:09, Christopher James Halse Rogers pisze: >>> >>> >>> On Thu, Aug 25, 2016 at 7:21 PM, Marcin MirosÅ‚aw <marcin@mejor.pl> >>> wrote: >>>> W dniu 25.08.2016 o 02:03, Christopher James Halse Rogers pisze: >>>> >>>> Hi! >>>> >>>>> On Thu, Aug 25, 2016 at 7:21 AM, marcin@mejor.pl wrote: >>>> [...] >>>>>> Does it means that cache is unavailable and only tiering will be in >>>>>> bcachefs? >>>>>> And... How to mount tiered FS? When I pass one device in mount I'm >>>>>> getting: >>>>>> bcache: bch_open_as_blockdevs() register_cache_set err >>>>>> insufficient >>>>>> devices >>>>> >>>>> Tiering gets you all the advantages of caching, plus you can >>>>> (with some >>>>> effort) have the combined filesystem size be the sum of the SSD + >>>>> HDD >>>>> capacities, rather than the capacity be determined solely by the >>>>> capacity of the slow tier (this is not currently the case for >>>>> bcachefs). >>>> >>>> I think that cacheing has at least such advantages over tiering: >>>> - allow fast read and write to files compressed with slow alghoritm >>>> (gzip) >>> >>> I think this is getting into ā€œwhat should we call this thingā€ >>> arguments. A naive cache is just going to promote the gzipped data to >>> the fast storage. On the other end, there's nothing much preventing a >>> sophisticated tiering system from compressing/decompressing as a >>> part of >>> tier demotion/promotion. >> >> When (de|re)compressing would be part of demotion or promotion then I >> agree, cache device doesn't have advantages. > > You also don't get (de|re)compression for free in a caching strategy > depending on where you put your cache. For example, a traditional bcache > cache backing a compressed filesystem will only ever see compressed data. > >> >>>> - can be optimized for using SSD drives >>> >>> It's not clear to me how? Tiering and caching are doing the same >>> sort of >>> things. >> I don't know how bcachefs handle SSD drives. SSD are faster in >> sequential write, I'm not sure if it could be achieved using tier >> device. Using cache device on disk format can be different than using in >> bcachefs and can turn random writes into sequential writes. > > I believe you mean *HDDs* are faster in sequential write? This is > exactly what tiering gets you - all writes go to the fast, SSD storage¹, > and then data are migrated off to the slower HDD in big, sequential chunks. SSD deals great with random reads but both SSD and HDD don't like random writes. I'm pasting a few links, you can read about problems with SSD performance. Also SSD performance is going down after a dozens minutes of intensive writes. https://github.com/akiradeveloper/dm-writeboost https://www.usenix.org/conference/fast12/sfs-random-write-considered-harmful-solid-state-drives http://codecapsule.com/2014/02/12/coding-for-ssds-part-5-access-patterns-and-system-optimizations/ http://www.tomsitpro.com/articles/enterprise-ssd-testing,2-863.html > ¹: Indeed, IIRC this is the only dataloss bug I've found in bcachefs - I > wrote too much data too fast, the migration from tier 0 to tier 1 > couldn't keep up, and tier 0 filled up to the point where it could no > longer write necessary journal entries, resulting in the filesystem > being unmountable. ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2016-08-26 8:24 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-08-19 18:07 How to create bcachefs? marcin 2016-08-24 6:52 ` Kent Overstreet 2016-08-24 21:21 ` marcin 2016-08-24 23:12 ` Eric Wheeler 2016-08-24 23:52 ` Kent Overstreet 2016-08-24 23:50 ` Kent Overstreet 2016-08-25 0:03 ` Christopher James Halse Rogers 2016-08-25 9:21 ` Marcin Mirosław 2016-08-25 11:09 ` Christopher James Halse Rogers 2016-08-25 12:11 ` Marcin Mirosław 2016-08-26 1:06 ` Christopher James Halse Rogers 2016-08-26 1:48 ` Eric Wheeler 2016-08-26 8:23 ` Marcin Mirosław
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.