From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A74DAC433E2 for ; Fri, 4 Sep 2020 14:25:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 52405206B8 for ; Fri, 4 Sep 2020 14:25:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="Rn5qthIa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 52405206B8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D32AC900003; Fri, 4 Sep 2020 10:25:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE3D76B0068; Fri, 4 Sep 2020 10:25:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD2C4900003; Fri, 4 Sep 2020 10:25:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id A388D6B005D for ; Fri, 4 Sep 2020 10:25:44 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5DA6C180AD81F for ; Fri, 4 Sep 2020 14:25:44 +0000 (UTC) X-FDA: 77225602608.24.pain17_0e10576270b2 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id BB46B1A4AF for ; Fri, 4 Sep 2020 14:25:40 +0000 (UTC) X-HE-Tag: pain17_0e10576270b2 X-Filterd-Recvd-Size: 5230 Received: from mail-ed1-f68.google.com (mail-ed1-f68.google.com [209.85.208.68]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Fri, 4 Sep 2020 14:25:39 +0000 (UTC) Received: by mail-ed1-f68.google.com with SMTP id ay8so6288360edb.8 for ; Fri, 04 Sep 2020 07:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jyk7CHt5bTCTjZ7EhyIfL9NYG87K4gl68GKaA7iV2V8=; b=Rn5qthIaDvzwFjhhXEEk9a0ckRXKPetzFqHyFCMSmJ8zIntPq6DN7Qn7EtPUyngR88 P37FqKENw04smcg+k9HBl5GLKyWbcT/iAUwsfDhYSu7W7AtDe1TZun8rfsn8cG+bt5sv 1i7gjuvUAy1R2X4bP9OfI81neqhAnC0xzBFiwUWDP24/RsAmAv1JbpEaRiGfgotjcXSj xR4WwaDN/ER7jmjn/h7goGDrIZPJyaY/dpwrNAz+mBp4B5YtxP3NgUn2ryJydxuq23Lz NgKdCMrPRcpkychU0PnC3S9UTrm6aDG7cE287ILgfjMDaZBy1AGTqyZhsUhxKyWnGapx BsCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jyk7CHt5bTCTjZ7EhyIfL9NYG87K4gl68GKaA7iV2V8=; b=gUA1OwHe34TqQ2J21b62n294ZvX1KwjQ+w6tcNZWVM5gn7/GqpJN/fIQKniotH/P4T DXWnqK6V1jrLaMgF7uttwFGyxrCusQMSJSF2MThM7TC9dF83EETk6AwQBb+SpQCf8x4F Mx0YUCGrlNaM1qWjl1DVwgLGM3nRVvKZ2FlggOdOadyOXYmymv1BTzpHlL86YyNThi2/ BZoHNR1NbcCBN/nc9bnjK03oLfsQp8PSdewqW+DzO4KURqSDJk+oMawlaWcgwvHn8o3G iniMvTb6PnY/6kJMpSouj0/rA7tHDaRZHmM5zrAX/pFjM1H9n3+nzFikYdCk0MFq1/Oi ETwA== X-Gm-Message-State: AOAM533AuQKaVJBn1dyzA1vICpxQkT9s2WDNMg3W7+PVFwCfvmrTtLLr QU+SLPFm/n2ESB1jqwOXjBlW9BRpFNIVy8U6fDiiTQ== X-Google-Smtp-Source: ABdhPJysfnaP8HEjzDvtjD+Y2aV+Obj8YyKPtfutRyAPfLvQQAnyIsL4ylug10Xad5ayI7NRXdv4tos0CdWBQRLsepY= X-Received: by 2002:a05:6402:1544:: with SMTP id p4mr8572340edx.346.1599229538615; Fri, 04 Sep 2020 07:25:38 -0700 (PDT) MIME-Version: 1.0 References: <20200901124615.137200-1-pasha.tatashin@soleen.com> <20200902140851.GJ4617@dhcp22.suse.cz> <74f2341a-7834-3e37-0346-7fbc48d74df3@suse.cz> <20200902151306.GL4617@dhcp22.suse.cz> <20200903063806.GM4617@dhcp22.suse.cz> <20200904070235.GA15277@dhcp22.suse.cz> In-Reply-To: <20200904070235.GA15277@dhcp22.suse.cz> From: Pavel Tatashin Date: Fri, 4 Sep 2020 10:25:02 -0400 Message-ID: Subject: Re: [PATCH] mm/memory_hotplug: drain per-cpu pages again during memory offline To: Michal Hocko Cc: David Hildenbrand , Vlastimil Babka , LKML , Andrew Morton , linux-mm Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: BB46B1A4AF X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > Another alternative would be to enable/disable static branch only from > users who really care but this is quite tricky because how do you tell > you need or not? It seems that alloc_contig_range would be just fine > with a weaker semantic because it would "only" to a spurious failure. > Memory hotplug on the other hand really needs to have a point where > nobody interferes with the offlined memory so it could ask for a > stronger semantic. > > Yet another option would be to make draining stronger and actually > guarantee there are no in-flight pages to be freed to the pcp list. > One way would be to tweak pcp->high and implement a strong barrier > (IPI?) to sync with all CPUs. Quite expensive, especially when there are > many draining requests (read cma users because hotplug doesn't really > matter much as it happens seldom). > > So no nice&cheap solution I can think of... I think start_isolate_page_range() should not be doing page draining at all. It should isolate ranges, meaning set appropriate flags, but draining should be performed by the users when appropriate: next to lru_add_drain_all() calls both in CMA and hotplug. Currently, the way start_isolate_page_range() drains pages is very inefficient. It calls drain_all_pages() for every valid page block, which is a slow call as it starts a thread per cpu, and waits for those threads to finish before returning. We could optimize by moving the drain_all_pages() calls from set_migratetype_isolate() to start_isolate_page_range() and call it once for every different zone, but both current users of this interface guarantee that all pfns [start_pfn, end_pfn] are within the same zone, and I think we should keep it this way, so again the extra traversal is going to be overhead overhead. This way we will have on average only a single drain per hot-remove (instead of one per block), and also it is going to be symmetric only in one place. Faster hot-remove and cma alloc, and no race, imo win-win. Pasha