linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Dominique Martinet <asmadeus@codewreck.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Matthew Wilcox <willy@infradead.org>,
	Jann Horn <jannh@google.com>, Jiri Kosina <jikos@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Greg KH <gregkh@linuxfoundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Michal Hocko <mhocko@suse.com>, Linux-MM <linux-mm@kvack.org>,
	kernel list <linux-kernel@vger.kernel.org>,
	Linux API <linux-api@vger.kernel.org>
Subject: Re: [PATCH] mm/mincore: allow for making sys_mincore() privileged
Date: Mon, 7 Jan 2019 05:32:27 +0100	[thread overview]
Message-ID: <20190107043227.GA3325@nautica> (raw)
In-Reply-To: <CAHk-=wiT=ov+6zYcnw_64ihYf74Amzqs67iVGtJMQq65PxiVYw@mail.gmail.com>

Linus Torvalds wrote on Sat, Jan 05, 2019:
> But I think my patch to just rip out all that page lookup, and just
> base it on the page table state has the fundamental advantage that it
> gets rid of code. Maybe I should jst commit it, and see if anything
> breaks? We do have options in case things break, and then we'd at
> least know who cares (and perhaps a lot more information of _why_ they
> care).

There actually are many tools like fincore which depend on mincore to
try to tell whether a file is "loaded in cache" or not (I personally use
vmtouch[1], but I know of at least nocache[2] uses it as well to only
try to evict used pages)

[1] https://hoytech.com/vmtouch/
[2] https://github.com/Feh/nocache


I mostly use these to either fadvise(POSIX_FADV_DONTNEED) or
prefetch/lock whole files so my "production" use-cases don't actually
rely on the mincore part of them; but when playing with these actions
it's actually fairly useful to be able to visualize which part of a file
ended in cache or monitor how a file's content evolve in cache...

There are various non-obvious behaviours where being able to poke around
is enlightening (e.g. fadvise dontneed is actually a hint, so even if
nothing uses the file linux sometimes keep the data around if it thinks
that would be useful and nocache has a mode to call fadvise multiple
times and things like that...)


Anyway, I agree the use of mincore for this is rather ugly, and
frankly some "cache management API" might be better in the long run if
only for performance reason (don't try these tools on a hundred TB
sparse file...), but until that pipe dream comes true I think mincore as
it was is useful for system admins.



Linus Torvalds wrote on Sun, Jan 06, 2019:
> I decided to just apply that patch. It is *not* marked for stable,
> very intentionally, because I expect that we will need to wait and see
> if there are issues with it, and whether we might have to do something
> entirely different (more like the traditional behavior with some extra
> "only for owner" logic).

FWIW I personally don't care much about "only for owner" or depending on
mmap options; I don't understand much of the security implications
honestly so I'm not sure how these limitations actually help.
On the other hand, a simple CAP_SYS_ADMIN check making the call take
either behaviour should be safe and would cover what I described above.


(by the way, while we are discussing permissions, a regular user can use
fadvise dontneed on files it doesn't own as well as long as it can open
them for reading; I'm not sure if that would need restricting as well in
the context of the security issue. Frankly even with mincore someone
could likely tell the difference through timing, if they just do it a
few times. Do magic, probe, flush out, repeat until satisfied.)


Thanks,
-- 
Dominique Martinet | Asmadeus

  parent reply	other threads:[~2019-01-07  4:32 UTC|newest]

Thread overview: 215+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-05 17:27 [PATCH] mm/mincore: allow for making sys_mincore() privileged Jiri Kosina
2019-01-05 17:27 ` Jiri Kosina
2019-01-05 19:14 ` Vlastimil Babka
2019-01-05 19:24   ` Jiri Kosina
2019-01-05 19:24     ` Jiri Kosina
2019-01-05 19:38     ` Vlastimil Babka
2019-01-08  9:14       ` Bernd Petrovitsch
2019-01-08 11:37         ` Jiri Kosina
2019-01-08 11:37           ` Jiri Kosina
2019-01-08 13:53           ` Bernd Petrovitsch
2019-01-08 14:08             ` Kirill A. Shutemov
2019-01-05 19:44 ` kbuild test robot
2019-01-05 19:46 ` Linus Torvalds
2019-01-05 19:46   ` Linus Torvalds
2019-01-05 20:12   ` Jiri Kosina
2019-01-05 20:12     ` Jiri Kosina
2019-01-05 20:17     ` Linus Torvalds
2019-01-05 20:17       ` Linus Torvalds
2019-01-05 20:43       ` Jiri Kosina
2019-01-05 20:43         ` Jiri Kosina
2019-01-05 21:54         ` Linus Torvalds
2019-01-05 21:54           ` Linus Torvalds
2019-01-06 11:33           ` Kevin Easton
2019-01-08  8:50           ` Kevin Easton
2019-01-18 14:23           ` Tejun Heo
2019-01-05 20:13   ` Linus Torvalds
2019-01-05 20:13     ` Linus Torvalds
2019-01-05 19:56 ` kbuild test robot
2019-01-05 22:54 ` Jann Horn
2019-01-05 22:54   ` Jann Horn
2019-01-05 23:05   ` Linus Torvalds
2019-01-05 23:05     ` Linus Torvalds
2019-01-05 23:16     ` Linus Torvalds
2019-01-05 23:16       ` Linus Torvalds
2019-01-05 23:28       ` Linus Torvalds
2019-01-05 23:28         ` Linus Torvalds
2019-01-05 23:39       ` Linus Torvalds
2019-01-05 23:39         ` Linus Torvalds
2019-01-06  0:11         ` Matthew Wilcox
2019-01-06  0:22           ` Linus Torvalds
2019-01-06  0:22             ` Linus Torvalds
2019-01-06  1:50             ` Linus Torvalds
2019-01-06  1:50               ` Linus Torvalds
2019-01-06 21:46               ` Linus Torvalds
2019-01-06 21:46                 ` Linus Torvalds
2019-01-08  4:43                 ` Dave Chinner
2019-01-08 17:57                   ` Linus Torvalds
2019-01-08 17:57                     ` Linus Torvalds
2019-01-09  2:24                     ` Dave Chinner
2019-01-09  2:31                       ` Jiri Kosina
2019-01-09  2:31                         ` Jiri Kosina
2019-01-09  4:39                         ` Dave Chinner
2019-01-09 10:08                           ` Jiri Kosina
2019-01-09 10:08                             ` Jiri Kosina
2019-01-10  1:15                             ` Dave Chinner
2019-01-10  7:54                               ` Jiri Kosina
2019-01-10  7:54                                 ` Jiri Kosina
2019-01-09 18:25                           ` Linus Torvalds
2019-01-09 18:25                             ` Linus Torvalds
2019-01-10  0:44                             ` Dave Chinner
2019-01-10  1:18                               ` Linus Torvalds
2019-01-10  1:18                                 ` Linus Torvalds
2019-01-10  5:26                                 ` Andy Lutomirski
2019-01-10  5:26                                   ` Andy Lutomirski
2019-01-10 14:47                                   ` Matthew Wilcox
2019-01-10 21:44                                     ` Dave Chinner
2019-01-10 21:59                                       ` Linus Torvalds
2019-01-10 21:59                                         ` Linus Torvalds
2019-01-11  1:47                                   ` Dave Chinner
2019-01-10  7:03                                 ` Dave Chinner
2019-01-10 11:47                                   ` Linus Torvalds
2019-01-10 11:47                                     ` Linus Torvalds
2019-01-10 12:24                                     ` Dominique Martinet
2019-01-10 12:24                                       ` Dominique Martinet
2019-01-10 22:11                                       ` Linus Torvalds
2019-01-10 22:11                                         ` Linus Torvalds
2019-01-11  2:03                                         ` Dave Chinner
2019-01-11  2:18                                           ` Linus Torvalds
2019-01-11  2:18                                             ` Linus Torvalds
2019-01-11  4:04                                             ` Dave Chinner
2019-01-11  4:08                                               ` Andy Lutomirski
2019-01-11  7:20                                                 ` Dave Chinner
2019-01-11  7:08                                               ` Linus Torvalds
2019-01-11  7:08                                                 ` Linus Torvalds
2019-01-11  7:36                                                 ` Dave Chinner
2019-01-11 16:26                                                   ` Linus Torvalds
2019-01-11 16:26                                                     ` Linus Torvalds
2019-01-15 23:45                                                     ` Dave Chinner
2019-01-16  4:54                                                       ` Linus Torvalds
2019-01-16  4:54                                                         ` Linus Torvalds
2019-01-16  5:49                                                         ` Linus Torvalds
2019-01-16  5:49                                                           ` Linus Torvalds
2019-01-17  1:26                                                         ` Dave Chinner
2019-02-20 15:49                                                     ` Nicolai Stange
2019-01-11  4:57                                         ` Dominique Martinet
2019-01-11  7:11                                           ` Linus Torvalds
2019-01-11  7:11                                             ` Linus Torvalds
2019-01-11  7:32                                             ` Dominique Martinet
2019-01-16  0:42                                         ` Josh Snyder
2019-01-16  5:00                                           ` Linus Torvalds
2019-01-16  5:00                                             ` Linus Torvalds
2019-01-16  5:25                                             ` Andy Lutomirski
2019-01-16  5:34                                               ` Linus Torvalds
2019-01-16  5:34                                                 ` Linus Torvalds
2019-01-16  5:46                                                 ` Dominique Martinet
2019-01-16  5:58                                                   ` Linus Torvalds
2019-01-16  5:58                                                     ` Linus Torvalds
2019-01-16  6:34                                                     ` Dominique Martinet
2019-01-16  7:52                                                       ` Josh Snyder
2019-01-16  7:52                                                         ` Josh Snyder
2019-01-16 12:18                                                         ` Kevin Easton
2019-01-17 21:45                                                         ` Vlastimil Babka
2019-01-18  4:49                                                           ` Linus Torvalds
2019-01-18  4:49                                                             ` Linus Torvalds
2019-01-18 18:58                                                             ` Vlastimil Babka
2019-01-16 16:12                                                     ` Jiri Kosina
2019-01-16 16:12                                                       ` Jiri Kosina
2019-01-16 17:48                                                       ` Linus Torvalds
2019-01-16 17:48                                                         ` Linus Torvalds
2019-01-16 20:23                                                         ` Jiri Kosina
2019-01-16 20:23                                                           ` Jiri Kosina
2019-01-16 21:37                                                           ` Matthew Wilcox
2019-01-16 21:41                                                             ` Jiri Kosina
2019-01-16 21:41                                                               ` Jiri Kosina
2019-01-17  9:52                                                               ` Cyril Hrubis
2019-01-28 13:49                                                               ` Cyril Hrubis
2019-01-17  4:51                                                             ` Linus Torvalds
2019-01-17  4:51                                                               ` Linus Torvalds
2019-01-18  4:54                                                               ` Linus Torvalds
2019-01-18  4:54                                                                 ` Linus Torvalds
2019-01-17  1:49                                                           ` Dominique Martinet
2019-01-23 20:27                                                           ` Linus Torvalds
2019-01-23 20:27                                                             ` Linus Torvalds
2019-01-23 20:35                                                             ` Linus Torvalds
2019-01-23 20:35                                                               ` Linus Torvalds
2019-01-23 23:12                                                               ` Jiri Kosina
2019-01-23 23:12                                                                 ` Jiri Kosina
2019-01-24  0:20                                                                 ` Linus Torvalds
2019-01-24  0:20                                                                   ` Linus Torvalds
2019-01-24  0:24                                                             ` Dominique Martinet
2019-01-24 12:45                                                               ` Dominique Martinet
2019-01-24 14:25                                                                 ` Jiri Kosina
2019-01-24 14:25                                                                   ` Jiri Kosina
2019-01-27 22:35                                                                   ` Jiri Kosina
2019-01-27 22:35                                                                     ` Jiri Kosina
2019-01-28  0:05                                                                     ` Dominique Martinet
2019-01-29 23:52                                                                       ` Jiri Kosina
2019-01-30  9:09                                                                         ` Michal Hocko
2019-01-30 12:29                                                                           ` Jiri Kosina
2019-01-16 12:36                                             ` Matthew Wilcox
2019-01-10 14:50                               ` Matthew Wilcox
2019-01-11  7:36                               ` Jiri Kosina
2019-01-11  7:36                                 ` Jiri Kosina
2019-01-17  2:22                                 ` Dave Chinner
2019-01-17  8:18                                   ` Jiri Kosina
2019-01-17  8:18                                     ` Jiri Kosina
2019-01-17 21:06                                     ` Dave Chinner
2019-01-07  4:32             ` Dominique Martinet [this message]
2019-01-07 10:33               ` Vlastimil Babka
2019-01-07 11:08                 ` Dominique Martinet
2019-01-07 11:59                   ` Vlastimil Babka
2019-01-07 13:29                   ` Daniel Gruss
2019-01-07 10:10         ` Michael Ellerman
2019-01-05 23:09   ` Jiri Kosina
2019-01-05 23:09     ` Jiri Kosina
2019-01-30 12:44 ` [PATCH 0/3] mincore() and IOCB_NOWAIT adjustments Vlastimil Babka
2019-01-30 12:44   ` [PATCH 1/3] mm/mincore: make mincore() more conservative Vlastimil Babka
2019-01-31  9:43     ` Michal Hocko
2019-01-31  9:51       ` Dominique Martinet
2019-01-31 17:46       ` Josh Snyder
2019-02-01  8:56     ` Vlastimil Babka
2019-03-06 23:13     ` Andrew Morton
2019-03-07  0:01       ` Jiri Kosina
2019-03-07  0:40         ` Dominique Martinet
2019-03-07  5:46           ` Jiri Kosina
2019-01-30 12:44   ` [PATCH 2/3] mm/filemap: initiate readahead even if IOCB_NOWAIT is set for the I/O Vlastimil Babka
2019-01-30 15:04     ` Florian Weimer
2019-01-30 15:15       ` Jiri Kosina
2019-01-31 10:47         ` Florian Weimer
2019-01-31 11:34           ` Jiri Kosina
2019-01-31  9:56     ` Michal Hocko
2019-01-31 10:15       ` Jiri Kosina
2019-01-31 10:23         ` Michal Hocko
2019-01-31 10:30           ` Jiri Kosina
2019-01-31 11:32             ` Michal Hocko
2019-01-31 17:54           ` Linus Torvalds
2019-02-01  5:13             ` Dave Chinner
2019-02-01  7:05               ` Linus Torvalds
2019-02-01  7:21                 ` Linus Torvalds
2019-02-01  1:44       ` Dave Chinner
2019-02-12 15:48         ` Jiri Kosina
2019-01-31 12:04     ` Daniel Gruss
2019-01-31 12:06       ` Vlastimil Babka
2019-01-31 12:08       ` Jiri Kosina
2019-01-31 12:57         ` Daniel Gruss
2019-01-30 12:44   ` [PATCH 3/3] mm/mincore: provide mapped status when cached status is not allowed Vlastimil Babka
2019-01-31 10:09     ` Michal Hocko
2019-02-01  9:04       ` Vlastimil Babka
2019-02-01  9:11         ` Michal Hocko
2019-02-01  9:27           ` Vlastimil Babka
2019-02-06 20:14             ` Jiri Kosina
2019-02-12  3:44         ` Jiri Kosina
2019-02-12  6:36           ` Michal Hocko
2019-02-12 13:09             ` Jiri Kosina
2019-02-12 14:01               ` Michal Hocko
2019-03-06 12:11   ` [PATCH 0/3] mincore() and IOCB_NOWAIT adjustments Jiri Kosina
2019-03-06 22:35     ` Andrew Morton
2019-03-06 22:48       ` Jiri Kosina
2019-03-06 23:23         ` Andrew Morton
2019-03-06 23:32           ` Dominique Martinet
2019-03-06 23:38             ` Andrew Morton
2019-03-09 16:53               ` Linus Torvalds
2019-03-12 14:17   ` [PATCH v2 0/2] prevent mincore() page cache leaks Vlastimil Babka
2019-03-12 14:17     ` [PATCH v2 1/2] mm/mincore: make mincore() more conservative Vlastimil Babka
2019-03-12 14:17     ` [PATCH v2 2/2] mm/mincore: provide mapped status when cached status is not allowed Vlastimil Babka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190107043227.GA3325@nautica \
    --to=asmadeus@codewreck.org \
    --cc=akpm@linux-foundation.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=jannh@google.com \
    --cc=jikos@kernel.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=peterz@infradead.org \
    --cc=torvalds@linux-foundation.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).