From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E4B4C6FD1D for ; Thu, 30 Mar 2023 13:28:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DBCA6B0074; Thu, 30 Mar 2023 09:28:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 065FD6B007D; Thu, 30 Mar 2023 09:28:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E20C96B007E; Thu, 30 Mar 2023 09:28:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CF97C6B0074 for ; Thu, 30 Mar 2023 09:28:57 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 906911C6154 for ; Thu, 30 Mar 2023 13:28:57 +0000 (UTC) X-FDA: 80625645114.04.DF113A7 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf13.hostedemail.com (Postfix) with ESMTP id A8EE420013 for ; Thu, 30 Mar 2023 13:28:54 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=gvETP0ew; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf13.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680182935; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3wzkcoa7Mihtj9UNRxQh22SXDR3QABl+wSSlCghODgg=; b=kOMezxav+3TM01f7LCkRgqm7fgcvqopHHYBAdGYwi0sHBpGjHBpW+Cw2VbJ+RiLNmAIGoS X2krTS8ovMIumeBWELsAjETPrDQTuQ8FleFK9c8ax4n+aMVQ/EtiitJ8XvOY3T4AgrpkJ7 FfCaZbCg0gTGoQdEoRXktm8Cmj2WfRk= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=gvETP0ew; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf13.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680182935; a=rsa-sha256; cv=none; b=nJ9HJUlkNlvduKIl1cqntRkqMPF7H1g+pDI+aOSB0cJ4y+V2IVeXJ8sj/t6pDCGkCQOToU WItJ3BMVZEOeXb/MmjZAUkH4Gu5FmaJXInHsryM3yrktasgF425VG09xZyCZpvEIo3v/74 HPR9agZgUZXTB5IN7DBKcdQpl0bhwoE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 15FC321A4E; Thu, 30 Mar 2023 13:28:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1680182933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3wzkcoa7Mihtj9UNRxQh22SXDR3QABl+wSSlCghODgg=; b=gvETP0ewtc0Ul0ZpGOqUt9Yp3FS7/2Q0XqMw9CaX97ZRMz3tveUL7AOHe7jahtIcVRoeHP oXO228EQnaPtNCFbTD8sXDAbdHwiMtiyH+9O5eM9Mah6lg6Evz4V+m+g4blsQm2+V/Lr9s 4IG5CQxIeBNlxSc5enE6unLkLQU9nQs= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E4FF01348E; Thu, 30 Mar 2023 13:28:52 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id b3m5NZSOJWTuCQAAMHmgww (envelope-from ); Thu, 30 Mar 2023 13:28:52 +0000 Date: Thu, 30 Mar 2023 15:28:52 +0200 From: Michal Hocko To: Marcelo Tosatti Cc: Frederic Weisbecker , Frederic Weisbecker , Andrew Morton , Leonardo Bras , Peter Zijlstra , Thomas Gleixner , Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , LKML , linux-mm@kvack.org Subject: Re: [PATCH 1/2] sched/isolation: Add cpu_is_isolated() API Message-ID: References: <20230317134448.11082-1-mhocko@kernel.org> <20230317134448.11082-2-mhocko@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: A8EE420013 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: wcyi34954pxbpzzhehkn9merri36c365 X-HE-Tag: 1680182934-709649 X-HE-Meta: U2FsdGVkX1+JfocucfCXZ3tOkeaxN04KInsRB+c8n8R78uj00cGGGYDUSJdtrZKpiteqCp1ZI2XLK5INdww35riPG4ZKV7h5l+m2rjfWZOVdk7gWwtQlywIq9gUPBMcMSiCUJNy9c2atXJJT6QfUS/0rJO7FRL+NEuiZwSidYz//aBBsGRZtHGOgX3yGUiZ7Da/pvlZwnNVQU6aAQmcjeEs4ToeuPzitME65NyqcSn2pRGs5ahMZdfsw4uaEO8Q6vmPkN1MhmmggCoYH2R6qUVFzZz8xGYH/DiXBzTRpjmxoIf413TnLlSFJcbv/tLTKuhEIfHPaF/R2Mu/xA+hfwLBy0xGlTOFxGj1fKv8qw19UXDI0wGiCljaj++M+7GjRLZSHjam56Go/BzNcSJg0a89FepBCe7bEoQ1uCtuBAGbIiMYPL9cCZPniypJufE5aDJiSBff30yZshUVJwGqANtY3gWByohKxAJwFX+ABeVzLnAqtt0N0Qt+SvvikzioMDQk7ddq9pL56FVQJiJnqC8sVjI5rwX1K/i2aA1lg0cf+2LPp8L1KyomM9NQVu1IsIH1rpDbMyONuavi3BlkuQ8pY3B6cB8yTLFgwE+3YaSjMisPSK/tHXpVlpN8t4KHkcpeb2XqLDutwCIMj2oe7gqtxLFozB0jzQyBqb52ZpKDFGNHiQsdXKVQsaL9CvIk38VSXB+XIlFhpw3XY7fkQ6pYvVazzv3VCSGbi5bblrYrzs4aT1Vh+zeVPX76CK/ushgt1t/ol4kJT87p+ECI9WbnRIDGieSu4tmTA+C7Zi0rIynJFfNG9IGU5lgMBbEN9Oxg+Te6WrB4a80QHtUIQPT4QWpUe6t7DJyHzknK+yCSZK+IkKAMyo79TOajLkpeWCas7sNDMu3JPAonk8h92n/4C37rkV8lC6SfulVLmAl7lafV0lytOKLBdtWXRnOoz1AUlfdaJGdCdJ5WJkrN IW1UF2xx MnccL8j01QtFId+5zqV+PY+eVZqsWGPvBw6kBDzpLO/mbMZoviBSBuDSeBzAI0esnXc6hcBdwSCN7VJnVzt99Xt8WsrfQZ3E0Dijlbxzro5B7qM9Q02JriyXhzLAuAsNNz1g2ZnmjiUcFosVbx/3Px0D4nBxU7EhPioINZZlCFEnGb2qIa/F4Pg9tKr+YRKbUxtklr9DQMKaip4cKJJCOmT/ESo9dUIZVGovkBpRaa+s7b9rmFJkwpZ4GF8jR39O3fDHica9MfnOveZT2w9U6rOy6zd7n5lwl3PSZTqfeXYuBvNo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 29-03-23 11:20:21, Marcelo Tosatti wrote: > On Tue, Mar 28, 2023 at 01:48:02PM +0200, Michal Hocko wrote: > > On Mon 27-03-23 07:24:54, Marcelo Tosatti wrote: > > > On Fri, Mar 24, 2023 at 11:35:35PM +0100, Frederic Weisbecker wrote: > > > > Le Sat, Mar 18, 2023 at 09:04:38AM +0100, Michal Hocko a écrit : > > > > > On Fri 17-03-23 15:35:05, Marcelo Tosatti wrote: > > [...] > > > > > > Actually introducing cpu_is_isolated() seems fine, but it can call > > > > > > housekeeping_test_cpu(cpu, HK_TYPE_TICK) AFAICS. > > > > > > > > > > This is not really my area. Frederic, could you have a look please? > > > > > > > > The point is to have a function that tells if either nohz_full= or > > > > isolcpus=[domain] has been passed for the given CPU. > > > > > > > > Because I assumed that both would be interested in avoiding that flush > > > > noise, wouldn't it be the case? > > > > > > Yes, that is the case. But as a note: for the two main types of > > > configuration performed (one uses isolcpus=[domain] and the other > > > cgroups, for isolating processes) nohz_full= is always set. > > > > > > So just testing for nohz_full= would be sufficient (which perhaps would > > > make the code simpler). > > > > I do not see any mention about that assumption under Documentation/. > > Documentation/admin-guide/kernel-per-CPU-kthreads.rst > > SCHED_SOFTIRQ > ------------- > > Do all of the following: > > 1. Avoid sending scheduler IPIs to the CPU to be de-jittered, > for example, ensure that at most one runnable kthread is present > on that CPU. If a thread that expects to run on the de-jittered > CPU awakens, the scheduler will send an IPI that can result in > a subsequent SCHED_SOFTIRQ. > 2. CONFIG_NO_HZ_FULL=y and ensure that the CPU to be de-jittered > is marked as an adaptive-ticks CPU using the "nohz_full=" > boot parameter. This reduces the number of scheduler-clock > interrupts that the de-jittered CPU receives, minimizing its > chances of being selected to do the load balancing work that > runs in SCHED_SOFTIRQ context. Quite hidden and easy to miss if you are only aware of isolcpus. > > Is this a best practice documented anywhere or it just happens to be > > the case with workloads you deal with? > > Option 2. However Frederic seems interested in matching the exported > toggles with the known use-cases classes. > > For example, for this guide: > http://www.comfilewiki.co.kr/en/doku.php?id=comfilepi:improving_real-time_performance:index > > Using nohz_full= would be a benefit (and its not being currently set, > perhaps due to not knowing all the options?). > > http://www.comfilewiki.co.kr/en/doku.php?id=comfilepi:improving_real-time_performance:index > > > AFAIU the workloads for which disabling nohz_full= is a benefit are those > where the switching between nohz full mode and sched tick enabled mode > and vice-versa (which involve programming the local timer) happens > often and is therefore avoidable? For example switching between 1 > runnable task and more than 1 runnable task (and vice versa). The patch from Frederic is testing for both. You seem to be arguing to reduce the test and I still do not understand why. Sure some workloads (following the above) will likely use nohz_full= as well but does it make sense to build that expectation into the higher level logic? What is an actual benefit? -- Michal Hocko SUSE Labs