From: Adam Manzanares <Adam.Manzanares@wdc.com> To: "lsf-pc@lists.linux-foundation.org" <lsf-pc@lists.linux-foundation.org> Cc: "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-block@vger.kernel.org" <linux-block@vger.kernel.org> Subject: [LSF/MM TOPIC] User Directed Tiered Memory Management Date: Wed, 24 Jan 2018 16:22:26 +0000 [thread overview] Message-ID: <cae10844-35cd-991c-c69d-545e774d5a50@wdc.com> (raw) V2l0aCB0aGUgaW50cm9kdWN0aW9uIG9mIGJ5dGUgYWRkcmVzc2FibGUgc3RvcmFnZSBkZXZpY2Vz IHRoYXQgaGF2ZSBsb3cgDQpsYXRlbmNpZXMsIGl0IGJlY29tZXMgZGlmZmljdWx0IHRvIGRlY2lk ZSBob3cgdG8gZXhwb3NlIHRoZXNlIGRldmljZXMgdG8gDQp1c2VyIHNwYWNlIGFwcGxpY2F0aW9u cy4gRG8gd2UgdHJlYXQgdGhlbSBhcyB0cmFkaXRpb25hbCBibG9jayBkZXZpY2VzIA0Kb3IgZXhw b3NlIHRoZW0gYXMgYSBEQVggY2FwYWJsZSBkZXZpY2U/IEEgdHJhZGl0aW9uYWwgYmxvY2sgZGV2 aWNlIA0KYWxsb3dzIHVzIHRvIHVzZSB0aGUgcGFnZSBjYWNoZSB0byB0YWtlIGFkdmFudGFnZSBv ZiBsb2NhbGl0eSBpbiBhY2Nlc3MgDQpwYXR0ZXJucywgYnV0IGNvbWVzIGF0IHRoZSBleHBlbnNl IG9mIGV4dHJhIG1lbW9yeSBjb3BpZXMgdGhhdCBhcmUgDQpleHRyZW1lbHkgY29zdGx5IGZvciBy YW5kb20gd29ya2xvYWRzLiBBIERBWCBjYXBhYmxlIGRldmljZSBzZWVtcyBncmVhdCANCmZvciB0 aGUgYWZvcmVtZW50aW9uZWQgcmFuZG9tIGFjY2VzcyB3b3JrbG9hZCwgYnV0IHN1ZmZlcnMgb25j ZSB0aGVyZSBpcyANCnNvbWUgbG9jYWxpdHkgaW4gdGhlIGFjY2VzcyBwYXR0ZXJuLg0KDQpXaGVu IERBWC1jYXBhYmxlIGRldmljZXMgYXJlIHVzZWQgYXMgc2xvd2VyL2NoZWFwZXIgdm9sYXRpbGUg bWVtb3J5LCANCnRyZWF0aW5nIHRoZW0gYXMgYSBzbG93ZXIgTlVNQSBub2RlIHdpdGggYW4gYXNz b2NpYXRlZCBOVU1BIG1pZ3JhdGlvbiANCnBvbGljeSB3b3VsZCBhbGxvdyBmb3IgdGFraW5nIGFk dmFudGFnZSBvZiBhY2Nlc3MgcGF0dGVybiBsb2NhbGl0eS4gDQpIb3dldmVyIHRoaXMgYXBwcm9h Y2ggc3VmZmVycyBmcm9tIGEgZmV3IGRyYXdiYWNrcy4gRmlyc3QsIHdoZW4gdGhvc2UgDQpkZXZp Y2VzIGFyZSBhbHNvIHBlcnNpc3RlbnQsIHRoZSB0aWVyaW5nIGFwcHJvYWNoIHVzZWQgaW4gTlVN QSBtaWdyYXRpb24gDQptYXkgbm90IGd1YXJhbnRlZSBwZXJzaXN0ZW5jZS4gU2Vjb25kbHksIGZv ciBkZXZpY2VzIHdpdGggc2lnbmlmaWNhbnRseSANCmhpZ2hlciBsYXRlbmNpZXMgdGhhbiBEUkFN LCB0aGUgY29zdCBvZiBtb3ZpbmcgY2xlYW4gcGFnZXMgbWF5IGJlIA0Kc2lnbmlmaWNhbnQuIEZp bmFsbHksIHBhZ2VzIGhhbmRsZWQgdmlhIE5VTUEgbWlncmF0aW9uIGFyZSBhIGNvbW1vbiANCnJl c291cmNlIHN1YmplY3QgdG8gdGhyYXNoaW5nIGluIGNhc2Ugb2YgbWVtb3J5IHByZXNzdXJlLg0K DQpJIHdvdWxkIGxpa2UgdG8gZGlzY3VzcyBhbiBhbHRlcm5hdGl2ZSBhcHByb2FjaCB3aGVyZSBt ZW1vcnkgaW50ZW5zaXZlIA0KYXBwbGljYXRpb25zIG1tYXAgdGhlc2Ugc3RvcmFnZSBkZXZpY2Vz IGludG8gdGhlaXIgYWRkcmVzcyBzcGFjZS4gVGhlIA0KYXBwbGljYXRpb24gY2FuIHNwZWNpZnkg aG93IG11Y2ggRFJBTSBjb3VsZCBiZSB1c2VkIGFzIGEgY2FjaGUgYW5kIGhhdmUgDQpzb21lIGlu Zmx1ZW5jZSBvbiBwcmVmZXRjaGluZyBhbmQgZXZpY3Rpb24gcG9saWNpZXMuIFRoZSBnb2FsIG9m IHN1Y2ggYW4gDQphcHByb2FjaCB3b3VsZCBiZSB0byBtaW5pbWl6ZSB0aGUgaW1wYWN0IG9mIHRo ZSBzbGlnaHRseSBzbG93ZXIgbWVtb3J5IA0KY291bGQgcG90ZW50aWFsbHkgaGF2ZSBvbiBhIHN5 c3RlbSB3aGVuIGl0IGlzIHRyZWF0ZWQgYXMga2VybmVsIG1hbmFnZWQgDQpnbG9iYWwgcmVzb3Vy Y2UsIGFzIHdlbGwgYXMgZW5hYmxlIHVzZSBvZiB0aG9zZSBkZXZpY2VzIGFzIHBlcnNpc3RlbnQg DQptZW1vcnkuIEJUVyB3ZSBjcmltaW5hbGx5IDspIHVzZWQgdGhlIHZtX2luc2VydF9wYWdlIGZ1 bmN0aW9uIGluIGEgDQpwcm90b3R5cGUgYW5kIGhhdmUgZm91bmQgdGhhdCBpdCBpcyBmYXN0ZXIg dG8gdXNlIHZzIHBhZ2UgY2FjaGUgYW5kIA0Kc3dhcHBpbmcgbWVjaGFuaXNtcyBsaW1pdGVkIHRv IHVzZSBhIHNtYWxsIGFtb3VudCBvZiBEUkFNLg==
WARNING: multiple messages have this Message-ID (diff)
From: Adam Manzanares <Adam.Manzanares@wdc.com> To: "lsf-pc@lists.linux-foundation.org" <lsf-pc@lists.linux-foundation.org> Cc: "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-block@vger.kernel.org" <linux-block@vger.kernel.org> Subject: [LSF/MM TOPIC] User Directed Tiered Memory Management Date: Wed, 24 Jan 2018 16:22:26 +0000 [thread overview] Message-ID: <cae10844-35cd-991c-c69d-545e774d5a50@wdc.com> (raw) With the introduction of byte addressable storage devices that have low latencies, it becomes difficult to decide how to expose these devices to user space applications. Do we treat them as traditional block devices or expose them as a DAX capable device? A traditional block device allows us to use the page cache to take advantage of locality in access patterns, but comes at the expense of extra memory copies that are extremely costly for random workloads. A DAX capable device seems great for the aforementioned random access workload, but suffers once there is some locality in the access pattern. When DAX-capable devices are used as slower/cheaper volatile memory, treating them as a slower NUMA node with an associated NUMA migration policy would allow for taking advantage of access pattern locality. However this approach suffers from a few drawbacks. First, when those devices are also persistent, the tiering approach used in NUMA migration may not guarantee persistence. Secondly, for devices with significantly higher latencies than DRAM, the cost of moving clean pages may be significant. Finally, pages handled via NUMA migration are a common resource subject to thrashing in case of memory pressure. I would like to discuss an alternative approach where memory intensive applications mmap these storage devices into their address space. The application can specify how much DRAM could be used as a cache and have some influence on prefetching and eviction policies. The goal of such an approach would be to minimize the impact of the slightly slower memory could potentially have on a system when it is treated as kernel managed global resource, as well as enable use of those devices as persistent memory. BTW we criminally ;) used the vm_insert_page function in a prototype and have found that it is faster to use vs page cache and swapping mechanisms limited to use a small amount of DRAM.
next reply other threads:[~2018-01-24 16:22 UTC|newest] Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-01-24 16:22 Adam Manzanares [this message] 2018-01-24 16:22 ` [LSF/MM TOPIC] User Directed Tiered Memory Management Adam Manzanares
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cae10844-35cd-991c-c69d-545e774d5a50@wdc.com \ --to=adam.manzanares@wdc.com \ --cc=linux-block@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lsf-pc@lists.linux-foundation.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.