* [LSF/MM TOPIC] plans for future swap changes @ 2016-12-28 14:57 Michal Hocko 2017-01-04 6:40 ` Johannes Weiner 0 siblings, 1 reply; 4+ messages in thread From: Michal Hocko @ 2016-12-28 14:57 UTC (permalink / raw) To: lsf-pc; +Cc: linux-mm, Johannes Weiner, Tim Chen This is something I would be interested to discuss even though I am not working on it directly. Sorry if I hijacked the topic from those who planned to post them. It seems that the time to reconsider our approach to the swap storage is come already and there are multiple areas to discuss. I would be interested at least in the following 1) anon/file balancing. Johannes has posted some work already and I am really interested in the future plans for it. 2) swap trashing detection is something that we are lacking for a long time and it would be great if we could do something to help situations when the machine is effectively out of memory but still hopelessly trying to swap in and out few pages while the machine is basically unusable. I hope that 1) will give us some bases but I am not sure how much we will need on top. 3) optimizations for the swap out paths - Tim Chen and other guys from Intel are already working on this. I didn't get time to review this closely - mostly because I am not closely familiar with the swapout code and it takes quite some time to get into all subtle details. I mainly interested in what are the plans in this area and how they should be coordinated with other swap related changes 4) Do we want the native THP swap in/out support? Other plans? -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [LSF/MM TOPIC] plans for future swap changes 2016-12-28 14:57 [LSF/MM TOPIC] plans for future swap changes Michal Hocko @ 2017-01-04 6:40 ` Johannes Weiner 2017-01-04 17:16 ` Tim Chen 2017-01-04 18:13 ` Shaohua Li 0 siblings, 2 replies; 4+ messages in thread From: Johannes Weiner @ 2017-01-04 6:40 UTC (permalink / raw) To: Michal Hocko; +Cc: lsf-pc, linux-mm, Tim Chen, Shaohua Li Hi, On Wed, Dec 28, 2016 at 03:57:32PM +0100, Michal Hocko wrote: > This is something I would be interested to discuss even though I am not > working on it directly. Sorry if I hijacked the topic from those who > planned to post them. > > It seems that the time to reconsider our approach to the swap storage is > come already and there are multiple areas to discuss. I would be > interested at least in the following > 1) anon/file balancing. Johannes has posted some work already and I am > really interested in the future plans for it. They needed some surgery to work on top of the node-LRU rewrite. I've restored performance on the benchmarks I was using and will post them after some more cleaning up and writing changelogs for the new pieces. > 2) swap trashing detection is something that we are lacking for a long > time and it would be great if we could do something to help > situations when the machine is effectively out of memory but still > hopelessly trying to swap in and out few pages while the machine is > basically unusable. I hope that 1) will give us some bases but I am > not sure how much we will need on top. Yes, this keeps biting us quite frequently. Not with swap so much as page cache, but it's the same problem: while we know all the thrashing *events*, we don't know how much they truly cost us. I've started drafting a thrashing quantification patch based on feedback from the Kernel Summit, attaching it below. It's unbelievably crude and needs more thought on sampling/decaying, as well as on filtering out swapins that happen after pressure has otherwise subsided. But it does give me a reasonable-looking thrashing ratio under memory pressure. > 3) optimizations for the swap out paths - Tim Chen and other guys from > Intel are already working on this. I didn't get time to review this > closely - mostly because I am not closely familiar with the swapout > code and it takes quite some time to get into all subtle details. > I mainly interested in what are the plans in this area and how they > should be coordinated with other swap related changes > 4) Do we want the native THP swap in/out support? Shaohua had some opinions on this, he might be interested in joining this discussion. CCing him. --- ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [LSF/MM TOPIC] plans for future swap changes 2017-01-04 6:40 ` Johannes Weiner @ 2017-01-04 17:16 ` Tim Chen 2017-01-04 18:13 ` Shaohua Li 1 sibling, 0 replies; 4+ messages in thread From: Tim Chen @ 2017-01-04 17:16 UTC (permalink / raw) To: Johannes Weiner, Michal Hocko; +Cc: lsf-pc, linux-mm, Shaohua Li > > > > > 3) optimizations for the swap out paths - Tim Chen and other guys from > > A A A Intel are already working on this. I didn't get time to review this > > A A A closely - mostly because I am not closely familiar with the swapout > > A A A code and it takes quite some time to get into all subtle details. > > A A A I mainly interested in what are the plans in this area and how they > > A A A should be coordinated with other swap related changes We are also planning on discussing this topic at the mm summit, if the patch series have not yet got into mainline, plus a couple of others swap related stuff. A I'll be sending out our proposal separately. Tim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [LSF/MM TOPIC] plans for future swap changes 2017-01-04 6:40 ` Johannes Weiner 2017-01-04 17:16 ` Tim Chen @ 2017-01-04 18:13 ` Shaohua Li 1 sibling, 0 replies; 4+ messages in thread From: Shaohua Li @ 2017-01-04 18:13 UTC (permalink / raw) To: Johannes Weiner; +Cc: Michal Hocko, lsf-pc, linux-mm, Tim Chen On Wed, Jan 04, 2017 at 01:40:24AM -0500, Johannes Weiner wrote: > Hi, > > On Wed, Dec 28, 2016 at 03:57:32PM +0100, Michal Hocko wrote: > > This is something I would be interested to discuss even though I am not > > working on it directly. Sorry if I hijacked the topic from those who > > planned to post them. > > > > It seems that the time to reconsider our approach to the swap storage is > > come already and there are multiple areas to discuss. I would be > > interested at least in the following > > 1) anon/file balancing. Johannes has posted some work already and I am > > really interested in the future plans for it. > > They needed some surgery to work on top of the node-LRU rewrite. I've > restored performance on the benchmarks I was using and will post them > after some more cleaning up and writing changelogs for the new pieces. > > > 2) swap trashing detection is something that we are lacking for a long > > time and it would be great if we could do something to help > > situations when the machine is effectively out of memory but still > > hopelessly trying to swap in and out few pages while the machine is > > basically unusable. I hope that 1) will give us some bases but I am > > not sure how much we will need on top. > > Yes, this keeps biting us quite frequently. Not with swap so much as > page cache, but it's the same problem: while we know all the thrashing > *events*, we don't know how much they truly cost us. I've started > drafting a thrashing quantification patch based on feedback from the > Kernel Summit, attaching it below. It's unbelievably crude and needs > more thought on sampling/decaying, as well as on filtering out swapins > that happen after pressure has otherwise subsided. But it does give me > a reasonable-looking thrashing ratio under memory pressure. > > > 3) optimizations for the swap out paths - Tim Chen and other guys from > > Intel are already working on this. I didn't get time to review this > > closely - mostly because I am not closely familiar with the swapout > > code and it takes quite some time to get into all subtle details. > > I mainly interested in what are the plans in this area and how they > > should be coordinated with other swap related changes > > 4) Do we want the native THP swap in/out support? > > Shaohua had some opinions on this, he might be interested in joining > this discussion. CCing him. Sure, I'm very interested in the topic. I'm pretty interested in reducing swap latency and making the latency more predictable. I had some random ideas and did some experiments before (no real patch though, sorry), like io poll for swapin, reduce reclaim batch count in direct page reclaim, and a flush-like proactive swapout and so on. Thanks, Shaohua -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-01-04 18:13 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-12-28 14:57 [LSF/MM TOPIC] plans for future swap changes Michal Hocko 2017-01-04 6:40 ` Johannes Weiner 2017-01-04 17:16 ` Tim Chen 2017-01-04 18:13 ` Shaohua Li
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.