* Re: how does the caching works in bcachefs [not found] <2308642.L3yuttUQlX@t460-skr> @ 2020-07-08 22:37 ` kent.overstreet 2020-07-09 13:20 ` Stefan K 0 siblings, 1 reply; 4+ messages in thread From: kent.overstreet @ 2020-07-08 22:37 UTC (permalink / raw) To: Stefan K; +Cc: linux-bcache, linux-bcachefs On Wed, Jul 08, 2020 at 11:46:00PM +0200, Stefan K wrote: > Hello, > > short question: how does the caching works with bcachefs? Is it like "first > in first out" or is it more complex like the ARC system in zfs? LRU, same as bcache. > The same with the write-cache, will be everything written to the SSD/NVMe > (Cache) and then to the HDD? When will will the filesystem say "its written to > disk"? And what happens with the data on the write cache if we have a > powerfail? Disks that are used as caches are treated no differently from other disks by the filesystem. If you want bcachefs to not rely on a specific disk, you can set its durability to 0, and then it'll basically only be used as a writethrough cache. > > And can I say have this file/folder always in the cache, while it works "normal" ? Yes. So caching is configured differently, specifically so that it can be configured on a per file/directory basis. Instead of having a notion of "cache device", there are options for - foregroud target: which device or group of devices are used for foreground writes - background target: if enabled, the rebalance thread will in the background move data to this target in the background, leaving a cached copy on the foreground target - promote target: if enabled, when data is read and it doesn't exist in this target, a cached copy will be added there So these options can be set to get you writeback mode, by setting foreground target and promote target to your SSD and background target to your HDD. And you can pin specific files/folders to a device, by setting foreground target to that device and setting background target and promote target to nothing. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: how does the caching works in bcachefs 2020-07-08 22:37 ` how does the caching works in bcachefs kent.overstreet @ 2020-07-09 13:20 ` Stefan K 2020-07-09 16:08 ` kent.overstreet 0 siblings, 1 reply; 4+ messages in thread From: Stefan K @ 2020-07-09 13:20 UTC (permalink / raw) To: kent.overstreet; +Cc: linux-bcache, linux-bcachefs Hi and thanks! > LRU, same as bcache. do you plan to change this , since LRU is not very efficient (in comparison to other), maybe 2Q or ARC[1-4] > [...] > And you can pin specific files/folders to a device, by setting foreground target > to that device and setting background target and promote target to nothing. ok thank you very much! That must be documented somewhere ;-) [1] https://en.ru.is/media/skjol-td/cache_comparison.pdf [2] https://dbs.uni-leipzig.de/file/ARC.pdf [3] http://people.cs.vt.edu/~butta/docs/sigmetrics05_kernelPrefetch.pdf [4] http://www.vldb.org/conf/1994/P439.PDF On Thursday, July 9, 2020 12:37:29 AM CEST kent.overstreet@gmail.com wrote: > On Wed, Jul 08, 2020 at 11:46:00PM +0200, Stefan K wrote: > > Hello, > > > > short question: how does the caching works with bcachefs? Is it like "first > > in first out" or is it more complex like the ARC system in zfs? > > LRU, same as bcache. > > > The same with the write-cache, will be everything written to the SSD/NVMe > > (Cache) and then to the HDD? When will will the filesystem say "its written to > > disk"? And what happens with the data on the write cache if we have a > > powerfail? > > Disks that are used as caches are treated no differently from other disks by the > filesystem. If you want bcachefs to not rely on a specific disk, you can set its > durability to 0, and then it'll basically only be used as a writethrough cache. > > > > > And can I say have this file/folder always in the cache, while it works "normal" ? > > Yes. > > So caching is configured differently, specifically so that it can be configured > on a per file/directory basis. Instead of having a notion of "cache device", > there are options for > - foregroud target: which device or group of devices are used for foreground > writes > - background target: if enabled, the rebalance thread will in the background > move data to this target in the background, leaving a cached copy on the > foreground target > - promote target: if enabled, when data is read and it doesn't exist in this > target, a cached copy will be added there > > So these options can be set to get you writeback mode, by setting foreground > target and promote target to your SSD and background target to your HDD. > > And you can pin specific files/folders to a device, by setting foreground target > to that device and setting background target and promote target to nothing. > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: how does the caching works in bcachefs 2020-07-09 13:20 ` Stefan K @ 2020-07-09 16:08 ` kent.overstreet 2020-07-16 21:07 ` Nix 0 siblings, 1 reply; 4+ messages in thread From: kent.overstreet @ 2020-07-09 16:08 UTC (permalink / raw) To: Stefan K; +Cc: linux-bcache, linux-bcachefs On Thu, Jul 09, 2020 at 03:20:14PM +0200, Stefan K wrote: > Hi and thanks! > > > LRU, same as bcache. > do you plan to change this , since LRU is not very efficient (in comparison to > other), maybe 2Q or ARC[1-4] No, but I'd be happy to help if someone else wanted to implement a new caching algorithm :) In real world mixed workloads LRU is fine, it's not that much of a difference vs. the more sophisticated algorithms. More important is the stuff like sequential_bypass or some other kind of knob to ensure your backup process doesn't blow away the entire cache. > > > [...] > > And you can pin specific files/folders to a device, by setting foreground target > > to that device and setting background target and promote target to nothing. > ok thank you very much! That must be documented somewhere ;-) I just write code, not documentation :) ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: how does the caching works in bcachefs 2020-07-09 16:08 ` kent.overstreet @ 2020-07-16 21:07 ` Nix 0 siblings, 0 replies; 4+ messages in thread From: Nix @ 2020-07-16 21:07 UTC (permalink / raw) To: kent.overstreet; +Cc: Stefan K, linux-bcache, linux-bcachefs On 9 Jul 2020, kent overstreet told this: > In real world mixed workloads LRU is fine, it's not that much of a difference > vs. the more sophisticated algorithms. More important is the stuff like > sequential_bypass or some other kind of knob to ensure your backup process > doesn't blow away the entire cache. The ioprio thing that never got integrated into bcache (but is still available as out-of-tree patches that work fine) is even better for this: yes, it might not quite be what ioprio was meant for, but it means the user has the equivalent of 'nocache' that they can apply to entire process hierarchies that they don't want to pollute the cache, just by using ionice. Run your backup with ionice -c 3 and now everything, even the metadata reads, comes from the cache if it's already in the cache but otherwise stays out of cache. (This matters if you have a huge amount of metadata that is rarely read except when the backup sweeps over it, and a not-too-huge cache device. You don't want to waste cache space on stuff that only the backup is accessing...) ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-07-16 21:25 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <2308642.L3yuttUQlX@t460-skr> 2020-07-08 22:37 ` how does the caching works in bcachefs kent.overstreet 2020-07-09 13:20 ` Stefan K 2020-07-09 16:08 ` kent.overstreet 2020-07-16 21:07 ` Nix
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).