From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9F50C433EF for ; Wed, 25 May 2022 11:37:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243141AbiEYLhM (ORCPT ); Wed, 25 May 2022 07:37:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243002AbiEYLhJ (ORCPT ); Wed, 25 May 2022 07:37:09 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7C9DCA005A for ; Wed, 25 May 2022 04:37:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653478626; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M3hAOXoo6D5ZjzfUZf6OBWb/9SBl/lq9MIa99qLJlp4=; b=TYRYfCai+rfP+hTGRyZohTUZ838I3jPEhF4RmtGN2WVTu7/BIo3b6WMHQyQqcIDOQdHE1H I2kZ2rrG8WxjcDBjsMdl6a3h3ptzk8z95Sz2j6to7DOojCsK4e0G8/Pk9+rH/R7RHDVK3K ukpUPyzXDuacnlGL7kGk+dNPJQxlU/8= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-44-7nBBaDZ6Ng6jV-ghjZOH_A-1; Wed, 25 May 2022 07:37:03 -0400 X-MC-Unique: 7nBBaDZ6Ng6jV-ghjZOH_A-1 Received: by mail-lj1-f200.google.com with SMTP id a14-20020a2e88ce000000b00254078ad384so599165ljk.7 for ; Wed, 25 May 2022 04:37:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=M3hAOXoo6D5ZjzfUZf6OBWb/9SBl/lq9MIa99qLJlp4=; b=wmoJGPJ5m1G+7PBVzUXUpUHmJ8+TN8kqRB3n+BObHzDe3DcfkzWZw8hJblRNNzW3Jz naxFgc2jSOFgxKRcQnRwHuv/qIeOW8MxX5xKXD3Zwkas0n3Fv3nAdTMti+k3XurI0TL7 VtLBY6tCqP9YoI1BMqPDZ+mtXoFLgv5tZ/rpsmA8J5dIY1viOsYOf9/GLwP2v5Bk/5xX k5AjhOeZCBYgSAWuNCOjpI13HA1No9wocpCnom3jxPQCVJaAzezgK+2/GIK8XIySPRSB OpmJcvlCnJA+NY8/ddYZPabYV8mZuK5dFNwnQqEKwVznaBmcdvXJl7VyU0MDCv5Dj7Tm ep7A== X-Gm-Message-State: AOAM533cOTEFNXhzFn94PxrJCDdXvJvR2GB/psD/FNpmMXLETb+goqVT VIh7iu/2nfdEyrMRppPxtSY260dIA62PH8i20jFbKdEWQFXLcdLzn097EIURZtY5prIFRSrnpAg 13uY2LYqsvDp1t9N8yZfduBY= X-Received: by 2002:a05:6512:b96:b0:478:5d7b:2e21 with SMTP id b22-20020a0565120b9600b004785d7b2e21mr15160521lfv.305.1653478621389; Wed, 25 May 2022 04:37:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwriGoreU0E3V6nOEG0nIMHCpIDDL6uaEAL1ptQBVTwiAybYB4y/6yMJsflVcf0aO7Cn3SYrg== X-Received: by 2002:a05:6512:b96:b0:478:5d7b:2e21 with SMTP id b22-20020a0565120b9600b004785d7b2e21mr15160495lfv.305.1653478621052; Wed, 25 May 2022 04:37:01 -0700 (PDT) Received: from [192.168.1.121] (91-145-109-188.bb.dnainternet.fi. [91.145.109.188]) by smtp.gmail.com with ESMTPSA id m8-20020ac24288000000b0047255d211ecsm3057974lfh.283.2022.05.25.04.36.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 25 May 2022 04:37:00 -0700 (PDT) Message-ID: <2d06932c-7d80-94c7-89ab-78375e58d3cc@redhat.com> Date: Wed, 25 May 2022 14:36:59 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: RFC: Memory Tiering Kernel Interfaces (v2) Content-Language: en-US To: Aneesh Kumar K V , Ying Huang , Wei Xu Cc: Jonathan Cameron , Andrew Morton , Greg Thelen , Yang Shi , Linux Kernel Mailing List , Jagdish Gediya , Michal Hocko , Tim C Chen , Dave Hansen , Alistair Popple , Baolin Wang , Feng Tang , Davidlohr Bueso , Dan Williams , David Rientjes , Linux MM , Brice Goglin , Hesham Almatary References: <20220512160010.00005bc4@Huawei.com> <6b7c472b50049592cde912f04ca47c696caa2227.camel@intel.com> <6ce724e5c67d4f7530457897fa08d0a8ba5dd6d0.camel@intel.com> <594046f8-9ab3-786a-fc48-8a61f1238f52@linux.ibm.com> From: =?UTF-8?Q?Mika_Penttil=c3=a4?= In-Reply-To: <594046f8-9ab3-786a-fc48-8a61f1238f52@linux.ibm.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 25.5.2022 13.01, Aneesh Kumar K V wrote: > On 5/25/22 2:33 PM, Ying Huang wrote: >> On Tue, 2022-05-24 at 22:32 -0700, Wei Xu wrote: >>> On Tue, May 24, 2022 at 1:24 AM Ying Huang wrote: >>>> >>>> On Tue, 2022-05-24 at 00:04 -0700, Wei Xu wrote: >>>>> On Thu, May 19, 2022 at 8:06 PM Ying Huang >>>>> wrote: >>>>>> > > ... > >> >> OK.  Just to confirm.  Does this mean that we will have fixed device ID, >> for example, >> >> GPU            memtier255 >> DRAM (with CPU)        memtier0 >> PMEM            memtier1 >> >> When we add a new memtier, it can be memtier254, or memter2?  The rank >> value will determine the real demotion order. >> >> I think you may need to send v3 to make sure everyone is at the same >> page. >> > > What we have implemented which we will send as RFC shortly is below. > > cd /sys/dekvaneesh@ubuntu-guest:~$ cd /sys/devices/system/ > kvaneesh@ubuntu-guest:/sys/devices/system$ pwd > /sys/devices/system > kvaneesh@ubuntu-guest:/sys/devices/system$ ls > clockevents  clocksource  container  cpu  edac  memory  memtier  mpic > node  power > kvaneesh@ubuntu-guest:/sys/devices/system$ cd memtier/ > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ pwd > /sys/devices/system/memtier > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ ls > default_rank  max_rank  memtier1  power  uevent > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat default_rank > 1 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat max_rank > 3 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cd memtier1/ > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ ls > nodelist  power  rank  subsystem  uevent > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat nodelist > 0-3 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat rank > 1 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cd > ../../node/node1/ > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ cat memtier > 1 > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ > root@ubuntu-guest:/sys/devices/system/node/node1# echo 0 > memtier > root@ubuntu-guest:/sys/devices/system/node/node1# cat memtier > 0 > root@ubuntu-guest:/sys/devices/system/node/node1# cd ../../memtier/ > root@ubuntu-guest:/sys/devices/system/memtier# ls > default_rank  max_rank  memtier0  memtier1  power  uevent > root@ubuntu-guest:/sys/devices/system/memtier# cd memtier0/ > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat nodelist > 1 > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat rank > 0 > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# echo 4 > rank > bash: rank: Permission denied > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# > Just to confirm, unlike today's demotion code, the demotion target allocation is planned to honor mempolicies? --Mika