From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: * X-Spam-Status: No, score=1.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FSL_HELO_FAKE,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, T_DKIMWL_WL_HIGH,USER_AGENT_MUTT autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFB9BC04AB6 for ; Tue, 28 May 2019 11:44:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9B4572070D for ; Tue, 28 May 2019 11:44:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1559043884; bh=Bw+1krKs9tiBaVfLtA1HVktTszDvJ2R/ptDZHg605MA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=yqqGMi01n83Ypl+gSG19/7yEhsMGSmYTCo8+pVD75vRw3EmBE1wYP/KVO3VaSUicQ M10jAntAtQlfTRP7qgAxleDUROtwMi1AiEUQEoc+3qSo4l9Zk++5jfTS7+mPBGBfhw 84XwHbkmpEcFikP17olgMReYTOacBKDw2HhlJA90= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726806AbfE1Loo (ORCPT ); Tue, 28 May 2019 07:44:44 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:34569 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726580AbfE1Lon (ORCPT ); Tue, 28 May 2019 07:44:43 -0400 Received: by mail-pg1-f194.google.com with SMTP id h2so7782444pgg.1; Tue, 28 May 2019 04:44:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=Na+J22i77Z4lW2CgfdPS10rXWxVdK/zIAtW3TAGkfc0=; b=OXrfO0e3RlFaNQy3MzONif4phiuHMCrfpnn6bbGelKBttsjDb7M44ZI+FJjgbUxw6f 7vXbGVdyuku1hUT89ULnYttsb2u5kz+/BF/U8tcBofrdT60RhDF7fwu4ghS6H7ZfsAJa FZ50GaZ9uRRD7wN2LYuLslXoDyELhO8unuwv8Xwg3UAse7oMSN4FPEALNexObmpuMFW/ 5tA/FWmvQH7GohrjdijN5eSt6qT+A7WGKgpjeoFVUudB1fizEiT481PdyYQ3Z5oxZlmg byTjYxHrYSLJKGsm3NO5UVkJkblMmBrgaHfU+8U+83lHzKF6n4q9it0LigqhyMDCjINY getA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=Na+J22i77Z4lW2CgfdPS10rXWxVdK/zIAtW3TAGkfc0=; b=WcCTaBc2Ay/4fBa8IHBQYy7FFZmce9fzsSG5T1AYrqMY+CpV5ffyQMdsIcxuk6AAQK 1SiNcrG6oKNa5pyABEkvkGWN13f5XIYU5pd1L0ruRbKEGoo5dG0St3i8y4XRB1IsRJ52 Qzj12eWEO25n0wdlIfc+pFNrPsy5tieH71MH8FwGOPnZEHgHhBR8AGIk7nJ8YRTltRzL cN20Y5jPVFGAWeSPgE6tjsUYRKz+I2Pm15Yzzw7SMAnUL7RARBPgPJ05GQyl2SQHaY5D A8fDIll3cSrDhgP+B34HtF+/DNydpNrkyq91gVWm0Nq3eGqCVVi4wWAtQfi+jZ/vJ+J7 gRdg== X-Gm-Message-State: APjAAAUQxYXIAxOvK2A7CU+t0bx0JjUUf6UkkQGeG9JyV66V9bykMPHA CHsZ40S4o+yI2NFI+vl93Ja+Jtby X-Google-Smtp-Source: APXvYqxKdDbXQTCQ2wTlaKWGq9gz76gsK2o3fQBBvCu+KCLAp37410V79lyMGOqtSpKnkp3QPk1hFQ== X-Received: by 2002:a65:5206:: with SMTP id o6mr1937077pgp.248.1559043882565; Tue, 28 May 2019 04:44:42 -0700 (PDT) Received: from google.com ([2401:fa00:d:0:98f1:8b3d:1f37:3e8]) by smtp.gmail.com with ESMTPSA id v4sm8549594pfe.180.2019.05.28.04.44.38 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 28 May 2019 04:44:41 -0700 (PDT) Date: Tue, 28 May 2019 20:44:36 +0900 From: Minchan Kim To: Michal Hocko Cc: Daniel Colascione , Andrew Morton , LKML , linux-mm , Johannes Weiner , Tim Murray , Joel Fernandes , Suren Baghdasaryan , Shakeel Butt , Sonny Rao , Brian Geffon , Linux API Subject: Re: [RFC 7/7] mm: madvise support MADV_ANONYMOUS_FILTER and MADV_FILE_FILTER Message-ID: <20190528114436.GB30365@google.com> References: <20190528032632.GF6879@google.com> <20190528062947.GL1658@dhcp22.suse.cz> <20190528081351.GA159710@google.com> <20190528084927.GB159710@google.com> <20190528090821.GU1658@dhcp22.suse.cz> <20190528103256.GA9199@google.com> <20190528104117.GW1658@dhcp22.suse.cz> <20190528111208.GA30365@google.com> <20190528112840.GY1658@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190528112840.GY1658@dhcp22.suse.cz> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, May 28, 2019 at 01:28:40PM +0200, Michal Hocko wrote: > On Tue 28-05-19 20:12:08, Minchan Kim wrote: > > On Tue, May 28, 2019 at 12:41:17PM +0200, Michal Hocko wrote: > > > On Tue 28-05-19 19:32:56, Minchan Kim wrote: > > > > On Tue, May 28, 2019 at 11:08:21AM +0200, Michal Hocko wrote: > > > > > On Tue 28-05-19 17:49:27, Minchan Kim wrote: > > > > > > On Tue, May 28, 2019 at 01:31:13AM -0700, Daniel Colascione wrote: > > > > > > > On Tue, May 28, 2019 at 1:14 AM Minchan Kim wrote: > > > > > > > > if we went with the per vma fd approach then you would get this > > > > > > > > > feature automatically because map_files would refer to file backed > > > > > > > > > mappings while map_anon could refer only to anonymous mappings. > > > > > > > > > > > > > > > > The reason to add such filter option is to avoid the parsing overhead > > > > > > > > so map_anon wouldn't be helpful. > > > > > > > > > > > > > > Without chiming on whether the filter option is a good idea, I'd like > > > > > > > to suggest that providing an efficient binary interfaces for pulling > > > > > > > memory map information out of processes. Some single-system-call > > > > > > > method for retrieving a binary snapshot of a process's address space > > > > > > > complete with attributes (selectable, like statx?) for each VMA would > > > > > > > reduce complexity and increase performance in a variety of areas, > > > > > > > e.g., Android memory map debugging commands. > > > > > > > > > > > > I agree it's the best we can get *generally*. > > > > > > Michal, any opinion? > > > > > > > > > > I am not really sure this is directly related. I think the primary > > > > > question that we have to sort out first is whether we want to have > > > > > the remote madvise call process or vma fd based. This is an important > > > > > distinction wrt. usability. I have only seen pid vs. pidfd discussions > > > > > so far unfortunately. > > > > > > > > With current usecase, it's per-process API with distinguishable anon/file > > > > but thought it could be easily extended later for each address range > > > > operation as userspace getting smarter with more information. > > > > > > Never design user API based on a single usecase, please. The "easily > > > extended" part is by far not clear to me TBH. As I've already mentioned > > > several times, the synchronization model has to be thought through > > > carefuly before a remote process address range operation can be > > > implemented. > > > > I agree with you that we shouldn't design API on single usecase but what > > you are concerning is actually not our usecase because we are resilient > > with the race since MADV_COLD|PAGEOUT is not destruptive. > > Actually, many hints are already racy in that the upcoming pattern would > > be different with the behavior you thought at the moment. > > How come they are racy wrt address ranges? You would have to be in > multithreaded environment and then the onus of synchronization is on > threads. That model is quite clear. But we are talking about separate Think about MADV_FREE. Allocator would think the chunk is worth to mark "freeable" but soon, user of the allocator asked the chunk - ie, it's not freeable any longer once user start to use it. My point is that kinds of *hints* are always racy so any synchronization couldn't help a lot. That's why I want to restrict hints process_madvise supports as such kinds of non-destruptive one at next respin. > processes and some of them might be even not aware of an external entity > tweaking their address space. > > > If you are still concerning of address range synchronization, how about > > moving such hints to per-process level like prctl? > > Does it make sense to you? > > No it doesn't. How is prctl any relevant to any address range > operations. "whether we want to have the remote madvise call process or vma fd based." You asked the above question and I answered we are using process level hints but anon/vma filter at this moment. That's why I told you prctl to make forward progress on discussion.