From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F224C433FE for ; Mon, 17 Oct 2022 16:06:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CAB5B6B0074; Mon, 17 Oct 2022 12:06:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5C786B0078; Mon, 17 Oct 2022 12:06:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B26BD6B007B; Mon, 17 Oct 2022 12:06:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A2BB66B0074 for ; Mon, 17 Oct 2022 12:06:11 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 699BD1A0F05 for ; Mon, 17 Oct 2022 16:06:11 +0000 (UTC) X-FDA: 80030918142.07.1D20764 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf08.hostedemail.com (Postfix) with ESMTP id 07F0216003A for ; Mon, 17 Oct 2022 16:06:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666022769; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=KG02hPJbNg6J8TWd2/QPe4puTPwoDSB7QBpWIeJjsZI=; b=VCb3RE0HpDkoWccmuzx6GH/Hl6sG/PkX7kmnCpzNww3DFv5JEf+Zg3eRPkUFsbDPbaI+19 5/mT5kR2WsNXfEOWLSO55zlsXgh626neIYfhid2mH7NehP/ZmR6Oi+/OUNjK1bzG9sI7W/ jNeIZL3NL9p8yJlJGzuviQ3qGKtY6os= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-25-7MzJootMPyCO-tr1JF1MDQ-1; Mon, 17 Oct 2022 12:06:03 -0400 X-MC-Unique: 7MzJootMPyCO-tr1JF1MDQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 57D7286C042; Mon, 17 Oct 2022 16:06:02 +0000 (UTC) Received: from fuller.cnet (ovpn-112-2.gru2.redhat.com [10.97.112.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D2ADC10197; Mon, 17 Oct 2022 16:06:01 +0000 (UTC) Received: by fuller.cnet (Postfix, from userid 1000) id A17D5416D5CB; Mon, 17 Oct 2022 13:04:47 -0300 (-03) Date: Mon, 17 Oct 2022 13:04:47 -0300 From: Marcelo Tosatti To: Hillf Danton Cc: Aaron Tomlin , frederic@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v8 5/5] tick/sched: Ensure quiet_vmstat() is called when the idle tick was stopped too Message-ID: References: <20220924152227.819815-1-atomlin@redhat.com> <20220925010511.1482-1-hdanton@sina.com> <20221003124435.1769-1-hdanton@sina.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221003124435.1769-1-hdanton@sina.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VCb3RE0H; spf=pass (imf08.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666022770; a=rsa-sha256; cv=none; b=X5f4Vtm6wZXItrK1S3n05tgAyIx6mC9yHqE4v39FTE6ugmdbL3oMUsVSJFIK9wqRRKuWIG 0hw19TFPP0gzcbBHAc6/G18PCOIya3W7u5Ap+4Ly/gHweap5mlpXSWvop6IEAo1vWPpp7v vlDX4lJi2Y6pGqAhzJNKYg/PU6FcLak= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666022770; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KG02hPJbNg6J8TWd2/QPe4puTPwoDSB7QBpWIeJjsZI=; b=QkxLzQRGu9Fbesrqs+6z0ieHYG27TMfza3oBZCGZWBNABCQ6AwkluuFwp85Az3nlrjKqi1 HNhYonty+rrCWPAbuOXTlJpjNX/jGRSa/BJKl6Um8Ii9eOX8wdxr9kIdfxLHzSocWI7/hk ur1hdIwG6sCP8pKlNOdBz6lTIA9T7UI= X-Stat-Signature: yu4ebrnsk7kqrabiabftkp8u7xoesx4n X-Rspamd-Queue-Id: 07F0216003A Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VCb3RE0H; spf=pass (imf08.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1666022769-241412 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Oct 03, 2022 at 08:44:35PM +0800, Hillf Danton wrote: > On 26 Sep 2022 10:20:04 +0100 Aaron Tomlin wrote: > > On Sun 2022-09-25 09:05 +0800, Hillf Danton wrote: > > > On 24 Sep 2022 16:24:41 +0100 Aaron Tomlin wrote: > > > > > > > > In the context of the idle task and an adaptive-tick mode/or a nohz_full > > > > CPU, quiet_vmstat() can be called: before stopping the idle tick, > > > > entering an idle state and on exit. In particular, for the latter case, > > > > when the idle task is required to reschedule, the idle tick can remain > > > > stopped and the timer expiration time endless i.e., KTIME_MAX. Now, > > > > indeed before a nohz_full CPU enters an idle state, CPU-specific vmstat > > > > counters should be processed to ensure the respective values have been > > > > reset and folded into the zone specific 'vm_stat[]'. That being said, it > > > > can only occur when: the idle tick was previously stopped, and > > > > reprogramming of the timer is not required. > > > > > > > > A customer provided some evidence which indicates that the idle tick was > > > > stopped; albeit, CPU-specific vmstat counters still remained populated. > > > > Thus one can only assume quiet_vmstat() was not invoked on return to the > > > > idle loop. > > > > > > Why did housekeeping CPUs fail to do their works, with this assumption > > > put aside? > > > > Hi Hillf, > > > > I'm not sure I understand your question. > > > > In this context, when tick processing is stopped, delayed work is not going > > to be handled until the CPU exits idle. > > Given work canceled because per-CPU pages can be freed remotely from > housekeeping CPUs (see patch 3/5), what is added here is not needed. > > IOW which one is incorrect? > > BTW given delayed work is not going to be handled until the CPU exits idle, Hi Hilf, The comment on the codebase now is: void quiet_vmstat(void) { if (system_state != SYSTEM_RUNNING) return; if (!delayed_work_pending(this_cpu_ptr(&vmstat_work))) return; if (!need_update(smp_processor_id())) return; /* * Just refresh counters and do not care about the pending delayed * vmstat_update. It doesn't fire that often to matter and canceling * it would be too expensive from this path. * vmstat_shepherd will take care about that for us. */ refresh_cpu_vm_stats(false); } However this is incorrect. The pending delayed work is only cancelled when executed and not requeued from: static void vmstat_update(struct work_struct *w) { if (refresh_cpu_vm_stats(true)) { /* * Counters were updated so we expect more updates * to occur in the future. Keep on running the * update worker thread. */ queue_delayed_work_on(smp_processor_id(), mm_percpu_wq, this_cpu_ptr(&vmstat_work), round_jiffies_relative(sysctl_stat_interval)); } } Since this patchset changes the synchronization to happen at return to userspace or entering idle, we do want to cancel that work (which, after synchronization, is not necessary). > canceling work is noop in 3/5, despite what the vmstat shepherd does depends > not on tick. Canceling work is a not a noop in 3/5: If the work is not cancelled (if 3/5 is dropped), there will be a pending work to be executed, from the kworker thread on an isolated CPU. Which is undesired for a fully isolated CPU, with no interruptions.