From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2848EC433DB for ; Thu, 18 Mar 2021 15:16:38 +0000 (UTC) Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67BCA64E90 for ; Thu, 18 Mar 2021 15:16:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67BCA64E90 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kroah.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernelnewbies-bounces@kernelnewbies.org Received: from localhost ([::1] helo=shelob.surriel.com) by shelob.surriel.com with esmtp (Exim 4.94) (envelope-from ) id 1lMuNo-00071p-7w; Thu, 18 Mar 2021 11:16:20 -0400 Received: from wout3-smtp.messagingengine.com ([64.147.123.19]) by shelob.surriel.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lMuNk-00071d-CR for kernelnewbies@kernelnewbies.org; Thu, 18 Mar 2021 11:16:17 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id EC45AA6F; Thu, 18 Mar 2021 11:16:11 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 18 Mar 2021 11:16:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kroah.com; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=RuRKpE0iMsgpXx+0PBOG51v3wDA BqEjvkVpcZu7wKpM=; b=dOosPNx3a1T+p8yz8XPFXk+iLiMP7k+Sw+bt2GNT904 PAtVDFq1sNXtzoX0kxEyrWByqBkL4KLO4PCxjMnIFrCvm+5YsnYOxxcW9egRSdAr zWA5mnjmYtnyqjcFabz0KvvmliRzkxYf6ttE0UcAKTzh1rndOjVfrIqH6caT4iLC Xf8Ko62q/kj+JLYmRSO2S4kh6jHatuOtuuM7OgHNbqaITXAAEnUL7XTFqJ26JCoU 5vLGsiZqeDpwuuLluKcr2Vu0+J35d66w/Lr822NXH1MTK2d50Wv/PptuXNOMryXX 8MGf5DfOW8CjANfV1EfeSUSRVC01Nt+HNpiZywo96Zg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=RuRKpE 0iMsgpXx+0PBOG51v3wDABqEjvkVpcZu7wKpM=; b=eQCt5+EGkciFKjLUILapPr QVptK6CLeFfRt8W9z2N385O31mt6VcbcJirwGnFn0rlht1ZE2idCNJ+azQm2dI/X SlaVfAD9kJ1gokOXODoqY8bH3oeA5C+rrPJkQcxJyVEkZY0kxWDfeATNR1c7n5Io 2uqRyA8+0fVAOttESTUwkNEar55n2EXoyyG7Mu6T5xRreW0pQqHZ+0LH0Og3VNMd yihIzjGjXyZqUeo4LssxHQmRGDyONq256/DRWbzt1b8iC47wBS7jSkYKvo78Z6fV npFDDC9BqqfbxSFE/7hYoO9tacPZdtdpwieOJW5Qzlki+fgRqHPLegp4zDhTvVoQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudefiedgjeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehttdertddttddvnecuhfhrohhmpefirhgvghcu mffjuceoghhrvghgsehkrhhorghhrdgtohhmqeenucggtffrrghtthgvrhhnpeeffeeuud ehgedtvdeujeeuieevleehtdfhjeduheetjefggeduffegtddtteehkeenucffohhmrghi nhepsghoohhtlhhinhdrtghomhenucfkphepkeefrdekiedrjeegrdeigeenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhgvgheskhhrohgr hhdrtghomh X-ME-Proxy: Received: from localhost (83-86-74-64.cable.dynamic.v4.ziggo.nl [83.86.74.64]) by mail.messagingengine.com (Postfix) with ESMTPA id BF150108005C; Thu, 18 Mar 2021 11:16:10 -0400 (EDT) Date: Thu, 18 Mar 2021 16:16:08 +0100 From: Greg KH To: Junyeong Jeong Subject: Re: /sys/devices/system/cpu/possible is immutable? Message-ID: References: <87a6r01pfs.fsf@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87a6r01pfs.fsf@gmail.com> Cc: kernelnewbies@kernelnewbies.org X-BeenThere: kernelnewbies@kernelnewbies.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Learn about the Linux kernel List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kernelnewbies-bounces@kernelnewbies.org On Thu, Mar 18, 2021 at 11:10:18PM +0900, Junyeong Jeong wrote: > Hello everyone :) > > I hope that kernelnewbies mailing list is a suitable place for asking my > question. > > I wonder that possible-CPU-mask(/sys/devices/system/cpu/possible) can be > changed after boot in some way or other. I read that it is fixed at boot > time (https://elixir.bootlin.com/linux/v5.8/source/include/linux/cpumask.h#L50). > But I am not convinced that it is really immutable even if some cgroup > or virtualization magic is used. > > Let me account for why I am curious about it. > Nowadays I am developing BPF library written in rust language. > In order to call `bpf_lookup_elem()` to get values from > BPF_MAP_TYPE_PERCPU_ARRAY in userspace, we need to know the correct > number of per-cpu areas before calling it. Because an out-buffer for > multiple per-cpu values should be allocated and passed to the > `bpf_lookup_elem()`. But this process is strongly based on the > assumption that the number of per-cpu area is always immutable. > > I am referring to /sys/devices/system/cpu/possible file to get to know > the number of per-cpu areas. I don't know the better way for figuring > out the number. What I am anxious about is that the number of per-cpu > areas varies from time to time under some circumstances with cgroup or > virtualization magic. > > So I checked some cgroup and virtualization ordinary use-cases which did > not affect the possible-CPU-mask. > > -- > 1. > $ docker run --cpuset-cpus=0-3 -it ubuntu:20.10 bash > > This does not affect /sys/devices/system/cpu/possible at all. The value > it contains is the same with the value of the host machine. > > 2. > $ virsh setvcpus --current ubuntu20.10 5 > > Before starting guest OS, the number of maximum vCPU was set to 8 and > current vCPU was set to 4. While guest OS is running, I changed the > number of vCPU to 5. And _inside guest OS_, I enabled the new CPU by > setting /sys/devices/system/cpu/cpu4/online to 1. But > /sys/devices/system/cpu/possible of guest OS did not change as expected. > -- > > While I was conducting some tests, I realized that it's not possible to > prove the immutability of possible-CPU-mask using inductive > method. Because there must be some corner cases that I can never > imagine. > > > Can anyone explain that possible-CPU-mask and the number of per-cpu > areas never change after boot-time even by cgroup magic or some tricks > from outside of hypervisors? That sysfs value itself will not change for the single system while the kernel is running, but your program could be moved from a system with one value for that file, to another system with a different value while it is running without knowing that you were migrated. But that being said, that file does not show you how many cpus you actually have access to at that moment in time, it's just a max-value for that specific kernel build. CPUs can come and go while your program is running, so be aware of that. Your program can also be forbidden to run on specific cpus if the admin decides to do so. So be careful here, and use the "normal" api to deal with cpu values and numbers, never assume that once you start, that the number of cpus your program has access to will not change. good luck! greg k-h _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies