From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC706C433EF for ; Wed, 4 May 2022 01:31:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BD206B0071; Tue, 3 May 2022 21:31:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 346676B0075; Tue, 3 May 2022 21:31:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E5F26B0078; Tue, 3 May 2022 21:31:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0C6636B0071 for ; Tue, 3 May 2022 21:31:43 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id CCFFF608BC for ; Wed, 4 May 2022 01:31:42 +0000 (UTC) X-FDA: 79426333644.14.E45A045 Received: from mail-vs1-f46.google.com (mail-vs1-f46.google.com [209.85.217.46]) by imf11.hostedemail.com (Postfix) with ESMTP id 8A2674007E for ; Wed, 4 May 2022 01:31:38 +0000 (UTC) Received: by mail-vs1-f46.google.com with SMTP id e10so8453436vsr.1 for ; Tue, 03 May 2022 18:31:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=SGRn2T0VMRPgqL8N4asL2R4bnXM/z5CuvCoeNoDGm/0=; b=n8tgslGE/cIbNMc/8xWjtECQvrKEb1JENIFN0173WK6xmffIwoXSaAiBK3AbQVGNNc IVZlqizaa9fCMkailAm/+NMr3vL4vr0hPzVQkUIAZwtAsDZbi3BtPzXaSbmAX5mHl/OP U9O+CueUAI2caDAN9uvDTMk5LZ4aVZcLVXnWd1dhZcVRP821y4YZ8XQCuBCSzUtAo55h iWd9qLHiyI0Il5piLCYCFbyZo38jK7IEvlJ0VK3lMxvkQVDwFQUGpNCeEznBlQ2bsWiB EEYUC4/gcPWzjegopgaLFWnbezK9GvQOUL52XJEmzwnEE4ZrVRYRhuHxd3Ci81rKImj5 MnEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=SGRn2T0VMRPgqL8N4asL2R4bnXM/z5CuvCoeNoDGm/0=; b=kzbzhlGTvT5tkfs89OhXa9nxL+41WMD6Wcb/oTkxq8UvnY5ZxF3lh92OtPwNUg8yFC 4Q3hYXIM9Vqv9TweJcFqnheQmv3ilUqH/qIbpqbQns0/Pd6o9aT7E3D6HYl5x3+qxae+ 3xPqJ0rNem0sLLrltmhe5Ms5s+C/0TaxN1mXjulNIYygY1TYXWaU9fdNIoJ3E5o1LtQv 3niT7fz63ivcOe5hpoFBcwi0D4HnpB47g6yNhFwXp1zmwkxhhHECi/SAoAwh35nm1ULr zIwFnMgMHSq8OHb9f0gYc7JZVoBrNeMl/h2ip8qtilVMn0/SQwD3zqsazbtbbOC+n0V1 1AeA== X-Gm-Message-State: AOAM533Uc0qafPssfBcs079E2vZaqHzMinS+6CAvvFZx0/wOpSb0w10u CFxGrnYGCORU0+okiLV1jLq0qD9V6xYvFUa3qeQzvA== X-Google-Smtp-Source: ABdhPJwsf3b9O/caCqzX/9BgtkrFdL2f5WcP7kRbQugM77uDVVCBf5+E7hT0BZj+8MC2zgvTCXycs7Nrg/wht/5kns8= X-Received: by 2002:a67:fd0b:0:b0:31b:e36d:31b1 with SMTP id f11-20020a67fd0b000000b0031be36d31b1mr5768157vsr.44.1651627901488; Tue, 03 May 2022 18:31:41 -0700 (PDT) MIME-Version: 1.0 References: <20220501175813.tvytoosygtqlh3nn@offworld> <87o80eh65f.fsf@nvdebian.thelocal> <87mtfygoxs.fsf@nvdebian.thelocal> <9fb22767-54de-d316-7e6b-5aac375c9c49@intel.com> In-Reply-To: <9fb22767-54de-d316-7e6b-5aac375c9c49@intel.com> From: Wei Xu Date: Tue, 3 May 2022 18:31:30 -0700 Message-ID: Subject: Re: RFC: Memory Tiering Kernel Interfaces To: Dave Hansen Cc: Alistair Popple , Davidlohr Bueso , Andrew Morton , Dave Hansen , Huang Ying , Dan Williams , Yang Shi , Linux MM , Greg Thelen , "Aneesh Kumar K.V" , Jagdish Gediya , Linux Kernel Mailing List , Michal Hocko , Baolin Wang , Brice Goglin , Feng Tang , Jonathan Cameron Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 8A2674007E Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=n8tgslGE; spf=pass (imf11.hostedemail.com: domain of weixugc@google.com designates 209.85.217.46 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Stat-Signature: x51otoak4tpbcus938dcdx6oq997pn3t X-HE-Tag: 1651627898-728536 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 3, 2022 at 4:54 PM Dave Hansen wrote: > > On 5/3/22 15:35, Alistair Popple wrote: > > Not entirely true. The GPUs on POWER9 have performance counters capable of > > collecting this kind of information for memory accessed from the GPU. I will > > admit though that sadly most people probably don't have a P9 sitting under their > > desk :) > > Well, x86 CPUs have performance monitoring hardware that can > theoretically collect physical access information too. But, this > performance monitoring hardware wasn't designed for this specific use > case in mind. So, in practice, these events (PEBS) weren't very useful > for driving memory tiering. The PEBS events without any filtering might not be useful for memory tiering, but the PEBS events with hardware-based data source filtering can be useful in driving promotions in memory tiering. Certainly, because these events are not designed for this specific use case in mind, there are inefficiencies using them for memory tiering, e.g. instead of just getting a heat counter for each hot page, we can get events repeatedly on the hot pages. > Are you saying that the GPUs on POWER9 have performance counters that > can drive memory tiering in practice? I'd be curious if there's working > code to show how they get used. Maybe the hardware is better than the > x86 PMU or the software consuming it is more clever than what we did. > But, I'd love to see it either way.