From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3B6E19915 for ; Tue, 6 Jun 2023 18:22:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B748CC433B3 for ; Tue, 6 Jun 2023 18:22:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686075733; bh=08bgDTu6K2r14VqmkEIMfCPJQ/ZZrnAx7Zh60inB9tQ=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=PvVbLJcYr9sEPc4KvXnz7da5hnzjz64447Y1vyViBiCb+fl59qj0pvbLDlkg7Wuob rzrnBjSOFtiQecuq+LZ/SRRxmzgXLdbIbqyyYrcYGhY29RsiV56QlreCUSnVm+stVt UoVEGKhBSdT54YRh1gXdKSmvd62cZB1711kSzDIW8D76YUK3gBLlfz43UtqujOAa4M WNIOBkfcw2WKv0IUzEzdSsuoiqNT0iAk+aytnzF0p5Ex5Qs+2o2gd4vHHiQKTwquZM rFReB/rrpz7owlp/MdxR2Mp3uAIDm8MwdFNdtC8Gq4wlspi7VJYfgsGIf9GhxSZs3/ 0v9cbIwE3cmiA== Received: by mail-lj1-f182.google.com with SMTP id 38308e7fff4ca-2b1a6a8e851so72152031fa.2 for ; Tue, 06 Jun 2023 11:22:13 -0700 (PDT) X-Gm-Message-State: AC+VfDy/LmlOPH1uVDyJOBVh8feRRVePysgjhfeCB5upnwhV6/XSUQ6j DKS7FwhJw2JPcblW79i2+6/wyhcKfhR3uFkRhqY= X-Google-Smtp-Source: ACHHUZ5bLSr3tA3IB8xCjeC0rEYdnEDeq7vmc1y/QuS5oHjyhnnuqZDpSw2PwmODz4CMTnRxT8zG59Qp1zLSa+hdG70= X-Received: by 2002:a2e:82d0:0:b0:2b0:297c:cbdf with SMTP id n16-20020a2e82d0000000b002b0297ccbdfmr1546782ljh.1.1686075731505; Tue, 06 Jun 2023 11:22:11 -0700 (PDT) Precedence: bulk X-Mailing-List: loongarch@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20230601101257.530867-1-rppt@kernel.org> <20230605092040.GB3460@kernel.org> In-Reply-To: From: Song Liu Date: Tue, 6 Jun 2023 11:21:59 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 00/13] mm: jit/text allocator To: Mark Rutland Cc: Mike Rapoport , Kent Overstreet , linux-kernel@vger.kernel.org, Andrew Morton , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Luis Chamberlain , Michael Ellerman , "Naveen N. Rao" , Palmer Dabbelt , Russell King , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Jun 5, 2023 at 3:09=E2=80=AFAM Mark Rutland = wrote: [...] > > > > Can you give more detail on what parameters you need? If the only e= xtra > > > > parameter is just "does this allocation need to live close to kerne= l > > > > text", that's not that big of a deal. > > > > > > My thinking was that we at least need the start + end for each caller= . That > > > might be it, tbh. > > > > Do you mean that modules will have something like > > > > jit_text_alloc(size, MODULES_START, MODULES_END); > > > > and kprobes will have > > > > jit_text_alloc(size, KPROBES_START, KPROBES_END); > > ? > > Yes. How about we start with two APIs: jit_text_alloc(size); jit_text_alloc_range(size, start, end); AFAICT, arm64 is the only arch that requires the latter API. And TBH, I am not quite convinced it is needed. > > > It sill can be achieved with a single jit_alloc_arch_params(), just by > > adding enum jit_type parameter to jit_text_alloc(). > > That feels backwards to me; it centralizes a bunch of information about > distinct users to be able to shove that into a static array, when the cal= lsites > can pass that information. I think we only two type of users: module and everything else (ftrace, kpro= be, bpf stuff). The key differences are: 1. module uses text and data; while everything else only uses text. 2. module code is generated by the compiler, and thus has stronger requirements in address ranges; everything else are generated via some JIT or manual written assembly, so they are more flexible with address ranges (in JIT, we can avoid using instructions that requires a specific address range). The next question is, can we have the two types of users share the same address ranges? If not, we can reserve the preferred range for modules, and let everything else use the other range. I don't see reasons to further separate users in the "everything else" group. > > What's *actually* common after separating out the ranges? Is it just the > permissions? I believe permission is the key, as we need the hardware to enforce permission. > > If we want this to be able to share allocations and so on, why can't we d= o this > like a kmem_cache, and have the callsite pass a pointer to the allocator = data? > That would make it easy for callsites to share an allocator or use a dist= inct > one. Sharing among different call sites will give us more benefit (in TLB misses rate, etc.). For example, a 2MB page may host text of two kernel modules, 4 kprob= es, 6 ftrace trampolines, and 10 BPF programs. All of these only require one en= try in the iTLB. Thanks, Song From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDC3AC7EE2A for ; Tue, 6 Jun 2023 18:22:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Lv6GQJV4Sr4qXRQEgcdSwrI0EuorRVtbJud7X192Kx4=; b=PSRiLLfN14eNJV IjgGci0QuiZSE76qdbGSmOSmzfYOHL0qLmioPUc8cN2p5CbV85NdTokiqZOWAFLwCePw/0OOgQty7 og38W5dEFiCYwaYjrmqoJAbXHNRiMTWLPtS+Ds26+wrgsLcgldxoACeDM/fGezgC813JQe1u+9kr9 DOcSiYNZ1l6wz1/+FUZLTcvZhiBtsloziQyvZB4XdjoVUN4uBgJyFJeaYKR8MxxIItWMiRcRWbNw2 8dRH6vvHG/DcDaKn3LZLXa7hmhMktV6YYvbJ/BDbutrgy18O3G60Bcz7n0e8QN3bg6uUr9z4JfVPr vvjjtUGecwtqepQM+yJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6bJz-002jtk-1r; Tue, 06 Jun 2023 18:22:19 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6bJv-002js0-2o; Tue, 06 Jun 2023 18:22:17 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E461D636A7; Tue, 6 Jun 2023 18:22:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DCA9C4331F; Tue, 6 Jun 2023 18:22:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686075734; bh=08bgDTu6K2r14VqmkEIMfCPJQ/ZZrnAx7Zh60inB9tQ=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=ZWPc2Ay9qV+46W5EIl1Hat/oWhCmkf2atoI7Djyx7M9K8nTcPWM51zO0JgXQR/xJA t6kkVflNQu8aEchQ0gWokejvbn0lD4N+mkDVLz/vpv4NfXuw4Ue4w5/ggGi9dBfKiZ FbyGdgANCc28tlTGlHG3sjNhVLTPxaSnMiPYhxWksxnkaN6w7Z8t8XpDCqJBTwrewz 7miB72dJ8VC2moDFlWb2Dg+2pHaD8TB88qpU65gh5ZEErvyTm+QOEYn4FI4IN5NnX+ vSaQ/C5uDNk1W0XxM90aSXxJjvBdgk4T++XB8N2/4BPrh3DvP9IK4m5Qlazjdmj/3m EHdGa+FT3FpmQ== Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2b1a6a8e851so72152051fa.2; Tue, 06 Jun 2023 11:22:13 -0700 (PDT) X-Gm-Message-State: AC+VfDx8vJxF4wXBEewLNUOaMWBRkggbwgiRpmweafg8qAgCL6pefZKz 2LHYUqaxGCKjXZz1oMNWACCgRaRvWepena+gN3s= X-Google-Smtp-Source: ACHHUZ5bLSr3tA3IB8xCjeC0rEYdnEDeq7vmc1y/QuS5oHjyhnnuqZDpSw2PwmODz4CMTnRxT8zG59Qp1zLSa+hdG70= X-Received: by 2002:a2e:82d0:0:b0:2b0:297c:cbdf with SMTP id n16-20020a2e82d0000000b002b0297ccbdfmr1546782ljh.1.1686075731505; Tue, 06 Jun 2023 11:22:11 -0700 (PDT) MIME-Version: 1.0 References: <20230601101257.530867-1-rppt@kernel.org> <20230605092040.GB3460@kernel.org> In-Reply-To: From: Song Liu Date: Tue, 6 Jun 2023 11:21:59 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 00/13] mm: jit/text allocator To: Mark Rutland Cc: Mike Rapoport , Kent Overstreet , linux-kernel@vger.kernel.org, Andrew Morton , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Luis Chamberlain , Michael Ellerman , "Naveen N. Rao" , Palmer Dabbelt , Russell King , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_112216_002578_0AF28623 X-CRM114-Status: GOOD ( 26.48 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gTW9uLCBKdW4gNSwgMjAyMyBhdCAzOjA54oCvQU0gTWFyayBSdXRsYW5kIDxtYXJrLnJ1dGxh bmRAYXJtLmNvbT4gd3JvdGU6CgpbLi4uXQoKPiA+ID4gPiBDYW4geW91IGdpdmUgbW9yZSBkZXRh aWwgb24gd2hhdCBwYXJhbWV0ZXJzIHlvdSBuZWVkPyBJZiB0aGUgb25seSBleHRyYQo+ID4gPiA+ IHBhcmFtZXRlciBpcyBqdXN0ICJkb2VzIHRoaXMgYWxsb2NhdGlvbiBuZWVkIHRvIGxpdmUgY2xv c2UgdG8ga2VybmVsCj4gPiA+ID4gdGV4dCIsIHRoYXQncyBub3QgdGhhdCBiaWcgb2YgYSBkZWFs Lgo+ID4gPgo+ID4gPiBNeSB0aGlua2luZyB3YXMgdGhhdCB3ZSBhdCBsZWFzdCBuZWVkIHRoZSBz dGFydCArIGVuZCBmb3IgZWFjaCBjYWxsZXIuIFRoYXQKPiA+ID4gbWlnaHQgYmUgaXQsIHRiaC4K PiA+Cj4gPiBEbyB5b3UgbWVhbiB0aGF0IG1vZHVsZXMgd2lsbCBoYXZlIHNvbWV0aGluZyBsaWtl Cj4gPgo+ID4gICAgICAgaml0X3RleHRfYWxsb2Moc2l6ZSwgTU9EVUxFU19TVEFSVCwgTU9EVUxF U19FTkQpOwo+ID4KPiA+IGFuZCBrcHJvYmVzIHdpbGwgaGF2ZQo+ID4KPiA+ICAgICAgIGppdF90 ZXh0X2FsbG9jKHNpemUsIEtQUk9CRVNfU1RBUlQsIEtQUk9CRVNfRU5EKTsKPiA+ID8KPgo+IFll cy4KCkhvdyBhYm91dCB3ZSBzdGFydCB3aXRoIHR3byBBUElzOgogICAgIGppdF90ZXh0X2FsbG9j KHNpemUpOwogICAgIGppdF90ZXh0X2FsbG9jX3JhbmdlKHNpemUsIHN0YXJ0LCBlbmQpOwoKQUZB SUNULCBhcm02NCBpcyB0aGUgb25seSBhcmNoIHRoYXQgcmVxdWlyZXMgdGhlIGxhdHRlciBBUEku IEFuZCBUQkgsIEkgYW0Kbm90IHF1aXRlIGNvbnZpbmNlZCBpdCBpcyBuZWVkZWQuCgo+Cj4gPiBJ dCBzaWxsIGNhbiBiZSBhY2hpZXZlZCB3aXRoIGEgc2luZ2xlIGppdF9hbGxvY19hcmNoX3BhcmFt cygpLCBqdXN0IGJ5Cj4gPiBhZGRpbmcgZW51bSBqaXRfdHlwZSBwYXJhbWV0ZXIgdG8gaml0X3Rl eHRfYWxsb2MoKS4KPgo+IFRoYXQgZmVlbHMgYmFja3dhcmRzIHRvIG1lOyBpdCBjZW50cmFsaXpl cyBhIGJ1bmNoIG9mIGluZm9ybWF0aW9uIGFib3V0Cj4gZGlzdGluY3QgdXNlcnMgdG8gYmUgYWJs ZSB0byBzaG92ZSB0aGF0IGludG8gYSBzdGF0aWMgYXJyYXksIHdoZW4gdGhlIGNhbGxzaXRlcwo+ IGNhbiBwYXNzIHRoYXQgaW5mb3JtYXRpb24uCgpJIHRoaW5rIHdlIG9ubHkgdHdvIHR5cGUgb2Yg dXNlcnM6IG1vZHVsZSBhbmQgZXZlcnl0aGluZyBlbHNlIChmdHJhY2UsIGtwcm9iZSwKYnBmIHN0 dWZmKS4gVGhlIGtleSBkaWZmZXJlbmNlcyBhcmU6CgogIDEuIG1vZHVsZSB1c2VzIHRleHQgYW5k IGRhdGE7IHdoaWxlIGV2ZXJ5dGhpbmcgZWxzZSBvbmx5IHVzZXMgdGV4dC4KICAyLiBtb2R1bGUg Y29kZSBpcyBnZW5lcmF0ZWQgYnkgdGhlIGNvbXBpbGVyLCBhbmQgdGh1cyBoYXMgc3Ryb25nZXIK ICByZXF1aXJlbWVudHMgaW4gYWRkcmVzcyByYW5nZXM7IGV2ZXJ5dGhpbmcgZWxzZSBhcmUgZ2Vu ZXJhdGVkIHZpYSBzb21lCiAgSklUIG9yIG1hbnVhbCB3cml0dGVuIGFzc2VtYmx5LCBzbyB0aGV5 IGFyZSBtb3JlIGZsZXhpYmxlIHdpdGggYWRkcmVzcwogIHJhbmdlcyAoaW4gSklULCB3ZSBjYW4g YXZvaWQgdXNpbmcgaW5zdHJ1Y3Rpb25zIHRoYXQgcmVxdWlyZXMgYSBzcGVjaWZpYwogIGFkZHJl c3MgcmFuZ2UpLgoKVGhlIG5leHQgcXVlc3Rpb24gaXMsIGNhbiB3ZSBoYXZlIHRoZSB0d28gdHlw ZXMgb2YgdXNlcnMgc2hhcmUgdGhlIHNhbWUKYWRkcmVzcyByYW5nZXM/IElmIG5vdCwgd2UgY2Fu IHJlc2VydmUgdGhlIHByZWZlcnJlZCByYW5nZSBmb3IgbW9kdWxlcywKYW5kIGxldCBldmVyeXRo aW5nIGVsc2UgdXNlIHRoZSBvdGhlciByYW5nZS4gSSBkb24ndCBzZWUgcmVhc29ucyB0byBmdXJ0 aGVyCnNlcGFyYXRlIHVzZXJzIGluIHRoZSAiZXZlcnl0aGluZyBlbHNlIiBncm91cC4KCj4KPiBX aGF0J3MgKmFjdHVhbGx5KiBjb21tb24gYWZ0ZXIgc2VwYXJhdGluZyBvdXQgdGhlIHJhbmdlcz8g SXMgaXQganVzdCB0aGUKPiBwZXJtaXNzaW9ucz8KCkkgYmVsaWV2ZSBwZXJtaXNzaW9uIGlzIHRo ZSBrZXksIGFzIHdlIG5lZWQgdGhlIGhhcmR3YXJlIHRvIGVuZm9yY2UKcGVybWlzc2lvbi4KCj4K PiBJZiB3ZSB3YW50IHRoaXMgdG8gYmUgYWJsZSB0byBzaGFyZSBhbGxvY2F0aW9ucyBhbmQgc28g b24sIHdoeSBjYW4ndCB3ZSBkbyB0aGlzCj4gbGlrZSBhIGttZW1fY2FjaGUsIGFuZCBoYXZlIHRo ZSBjYWxsc2l0ZSBwYXNzIGEgcG9pbnRlciB0byB0aGUgYWxsb2NhdG9yIGRhdGE/Cj4gVGhhdCB3 b3VsZCBtYWtlIGl0IGVhc3kgZm9yIGNhbGxzaXRlcyB0byBzaGFyZSBhbiBhbGxvY2F0b3Igb3Ig dXNlIGEgZGlzdGluY3QKPiBvbmUuCgpTaGFyaW5nIGFtb25nIGRpZmZlcmVudCBjYWxsIHNpdGVz IHdpbGwgZ2l2ZSB1cyBtb3JlIGJlbmVmaXQgKGluIFRMQgptaXNzZXMgcmF0ZSwKZXRjLikuIEZv ciBleGFtcGxlLCBhIDJNQiBwYWdlIG1heSBob3N0IHRleHQgb2YgdHdvIGtlcm5lbCBtb2R1bGVz LCA0IGtwcm9iZXMsCjYgZnRyYWNlIHRyYW1wb2xpbmVzLCBhbmQgMTAgQlBGIHByb2dyYW1zLiBB bGwgb2YgdGhlc2Ugb25seSByZXF1aXJlIG9uZSBlbnRyeQppbiB0aGUgaVRMQi4KClRoYW5rcywK U29uZwoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlu dXgtcmlzY3YgbWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0 cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3FA4BC77B73 for ; Tue, 6 Jun 2023 18:22:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YrK6NPgi72f7gbFBPpXisgN8PorrKa1HaPr8aGo08MM=; b=UDWPBAO95MQPt9 zXuowNE9VnDa90oysD/1KBtHIMCRExFMEQqX+rvIIGTK9VOWgQHcUH3NB6d4hZK4uXHh4OLRvBSor yhxayKjv9ZeU53vR5gfMNPEN8u3d/41Ws0wCX+id8mf5beM2jmxoRfUZAZABCot6pd9xHGXI9EQFJ EfFqMOXt2IlmGqKDVij0hlJTZgisn4ZO/OaYlNtJAZ+k1gG5GOEXuLm77S84mRV7uqE5zOdanmtm5 +p9rkjpXvQm69lMe/TzRn5IBshoWRgWPU6nagRNI8j23xokwIY6J+kG9bfodZT6AYqPcnB0mkWqMe GuAy1JAPi8VAZ9KyxZ2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q6bJx-002jtF-32; Tue, 06 Jun 2023 18:22:17 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q6bJv-002js0-2o; Tue, 06 Jun 2023 18:22:17 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E461D636A7; Tue, 6 Jun 2023 18:22:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DCA9C4331F; Tue, 6 Jun 2023 18:22:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686075734; bh=08bgDTu6K2r14VqmkEIMfCPJQ/ZZrnAx7Zh60inB9tQ=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=ZWPc2Ay9qV+46W5EIl1Hat/oWhCmkf2atoI7Djyx7M9K8nTcPWM51zO0JgXQR/xJA t6kkVflNQu8aEchQ0gWokejvbn0lD4N+mkDVLz/vpv4NfXuw4Ue4w5/ggGi9dBfKiZ FbyGdgANCc28tlTGlHG3sjNhVLTPxaSnMiPYhxWksxnkaN6w7Z8t8XpDCqJBTwrewz 7miB72dJ8VC2moDFlWb2Dg+2pHaD8TB88qpU65gh5ZEErvyTm+QOEYn4FI4IN5NnX+ vSaQ/C5uDNk1W0XxM90aSXxJjvBdgk4T++XB8N2/4BPrh3DvP9IK4m5Qlazjdmj/3m EHdGa+FT3FpmQ== Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2b1a6a8e851so72152051fa.2; Tue, 06 Jun 2023 11:22:13 -0700 (PDT) X-Gm-Message-State: AC+VfDx8vJxF4wXBEewLNUOaMWBRkggbwgiRpmweafg8qAgCL6pefZKz 2LHYUqaxGCKjXZz1oMNWACCgRaRvWepena+gN3s= X-Google-Smtp-Source: ACHHUZ5bLSr3tA3IB8xCjeC0rEYdnEDeq7vmc1y/QuS5oHjyhnnuqZDpSw2PwmODz4CMTnRxT8zG59Qp1zLSa+hdG70= X-Received: by 2002:a2e:82d0:0:b0:2b0:297c:cbdf with SMTP id n16-20020a2e82d0000000b002b0297ccbdfmr1546782ljh.1.1686075731505; Tue, 06 Jun 2023 11:22:11 -0700 (PDT) MIME-Version: 1.0 References: <20230601101257.530867-1-rppt@kernel.org> <20230605092040.GB3460@kernel.org> In-Reply-To: From: Song Liu Date: Tue, 6 Jun 2023 11:21:59 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 00/13] mm: jit/text allocator To: Mark Rutland Cc: Mike Rapoport , Kent Overstreet , linux-kernel@vger.kernel.org, Andrew Morton , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Luis Chamberlain , Michael Ellerman , "Naveen N. Rao" , Palmer Dabbelt , Russell King , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230606_112216_002578_0AF28623 X-CRM114-Status: GOOD ( 26.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCBKdW4gNSwgMjAyMyBhdCAzOjA54oCvQU0gTWFyayBSdXRsYW5kIDxtYXJrLnJ1dGxh bmRAYXJtLmNvbT4gd3JvdGU6CgpbLi4uXQoKPiA+ID4gPiBDYW4geW91IGdpdmUgbW9yZSBkZXRh aWwgb24gd2hhdCBwYXJhbWV0ZXJzIHlvdSBuZWVkPyBJZiB0aGUgb25seSBleHRyYQo+ID4gPiA+ IHBhcmFtZXRlciBpcyBqdXN0ICJkb2VzIHRoaXMgYWxsb2NhdGlvbiBuZWVkIHRvIGxpdmUgY2xv c2UgdG8ga2VybmVsCj4gPiA+ID4gdGV4dCIsIHRoYXQncyBub3QgdGhhdCBiaWcgb2YgYSBkZWFs Lgo+ID4gPgo+ID4gPiBNeSB0aGlua2luZyB3YXMgdGhhdCB3ZSBhdCBsZWFzdCBuZWVkIHRoZSBz dGFydCArIGVuZCBmb3IgZWFjaCBjYWxsZXIuIFRoYXQKPiA+ID4gbWlnaHQgYmUgaXQsIHRiaC4K PiA+Cj4gPiBEbyB5b3UgbWVhbiB0aGF0IG1vZHVsZXMgd2lsbCBoYXZlIHNvbWV0aGluZyBsaWtl Cj4gPgo+ID4gICAgICAgaml0X3RleHRfYWxsb2Moc2l6ZSwgTU9EVUxFU19TVEFSVCwgTU9EVUxF U19FTkQpOwo+ID4KPiA+IGFuZCBrcHJvYmVzIHdpbGwgaGF2ZQo+ID4KPiA+ICAgICAgIGppdF90 ZXh0X2FsbG9jKHNpemUsIEtQUk9CRVNfU1RBUlQsIEtQUk9CRVNfRU5EKTsKPiA+ID8KPgo+IFll cy4KCkhvdyBhYm91dCB3ZSBzdGFydCB3aXRoIHR3byBBUElzOgogICAgIGppdF90ZXh0X2FsbG9j KHNpemUpOwogICAgIGppdF90ZXh0X2FsbG9jX3JhbmdlKHNpemUsIHN0YXJ0LCBlbmQpOwoKQUZB SUNULCBhcm02NCBpcyB0aGUgb25seSBhcmNoIHRoYXQgcmVxdWlyZXMgdGhlIGxhdHRlciBBUEku IEFuZCBUQkgsIEkgYW0Kbm90IHF1aXRlIGNvbnZpbmNlZCBpdCBpcyBuZWVkZWQuCgo+Cj4gPiBJ dCBzaWxsIGNhbiBiZSBhY2hpZXZlZCB3aXRoIGEgc2luZ2xlIGppdF9hbGxvY19hcmNoX3BhcmFt cygpLCBqdXN0IGJ5Cj4gPiBhZGRpbmcgZW51bSBqaXRfdHlwZSBwYXJhbWV0ZXIgdG8gaml0X3Rl eHRfYWxsb2MoKS4KPgo+IFRoYXQgZmVlbHMgYmFja3dhcmRzIHRvIG1lOyBpdCBjZW50cmFsaXpl cyBhIGJ1bmNoIG9mIGluZm9ybWF0aW9uIGFib3V0Cj4gZGlzdGluY3QgdXNlcnMgdG8gYmUgYWJs ZSB0byBzaG92ZSB0aGF0IGludG8gYSBzdGF0aWMgYXJyYXksIHdoZW4gdGhlIGNhbGxzaXRlcwo+ IGNhbiBwYXNzIHRoYXQgaW5mb3JtYXRpb24uCgpJIHRoaW5rIHdlIG9ubHkgdHdvIHR5cGUgb2Yg dXNlcnM6IG1vZHVsZSBhbmQgZXZlcnl0aGluZyBlbHNlIChmdHJhY2UsIGtwcm9iZSwKYnBmIHN0 dWZmKS4gVGhlIGtleSBkaWZmZXJlbmNlcyBhcmU6CgogIDEuIG1vZHVsZSB1c2VzIHRleHQgYW5k IGRhdGE7IHdoaWxlIGV2ZXJ5dGhpbmcgZWxzZSBvbmx5IHVzZXMgdGV4dC4KICAyLiBtb2R1bGUg Y29kZSBpcyBnZW5lcmF0ZWQgYnkgdGhlIGNvbXBpbGVyLCBhbmQgdGh1cyBoYXMgc3Ryb25nZXIK ICByZXF1aXJlbWVudHMgaW4gYWRkcmVzcyByYW5nZXM7IGV2ZXJ5dGhpbmcgZWxzZSBhcmUgZ2Vu ZXJhdGVkIHZpYSBzb21lCiAgSklUIG9yIG1hbnVhbCB3cml0dGVuIGFzc2VtYmx5LCBzbyB0aGV5 IGFyZSBtb3JlIGZsZXhpYmxlIHdpdGggYWRkcmVzcwogIHJhbmdlcyAoaW4gSklULCB3ZSBjYW4g YXZvaWQgdXNpbmcgaW5zdHJ1Y3Rpb25zIHRoYXQgcmVxdWlyZXMgYSBzcGVjaWZpYwogIGFkZHJl c3MgcmFuZ2UpLgoKVGhlIG5leHQgcXVlc3Rpb24gaXMsIGNhbiB3ZSBoYXZlIHRoZSB0d28gdHlw ZXMgb2YgdXNlcnMgc2hhcmUgdGhlIHNhbWUKYWRkcmVzcyByYW5nZXM/IElmIG5vdCwgd2UgY2Fu IHJlc2VydmUgdGhlIHByZWZlcnJlZCByYW5nZSBmb3IgbW9kdWxlcywKYW5kIGxldCBldmVyeXRo aW5nIGVsc2UgdXNlIHRoZSBvdGhlciByYW5nZS4gSSBkb24ndCBzZWUgcmVhc29ucyB0byBmdXJ0 aGVyCnNlcGFyYXRlIHVzZXJzIGluIHRoZSAiZXZlcnl0aGluZyBlbHNlIiBncm91cC4KCj4KPiBX aGF0J3MgKmFjdHVhbGx5KiBjb21tb24gYWZ0ZXIgc2VwYXJhdGluZyBvdXQgdGhlIHJhbmdlcz8g SXMgaXQganVzdCB0aGUKPiBwZXJtaXNzaW9ucz8KCkkgYmVsaWV2ZSBwZXJtaXNzaW9uIGlzIHRo ZSBrZXksIGFzIHdlIG5lZWQgdGhlIGhhcmR3YXJlIHRvIGVuZm9yY2UKcGVybWlzc2lvbi4KCj4K PiBJZiB3ZSB3YW50IHRoaXMgdG8gYmUgYWJsZSB0byBzaGFyZSBhbGxvY2F0aW9ucyBhbmQgc28g b24sIHdoeSBjYW4ndCB3ZSBkbyB0aGlzCj4gbGlrZSBhIGttZW1fY2FjaGUsIGFuZCBoYXZlIHRo ZSBjYWxsc2l0ZSBwYXNzIGEgcG9pbnRlciB0byB0aGUgYWxsb2NhdG9yIGRhdGE/Cj4gVGhhdCB3 b3VsZCBtYWtlIGl0IGVhc3kgZm9yIGNhbGxzaXRlcyB0byBzaGFyZSBhbiBhbGxvY2F0b3Igb3Ig dXNlIGEgZGlzdGluY3QKPiBvbmUuCgpTaGFyaW5nIGFtb25nIGRpZmZlcmVudCBjYWxsIHNpdGVz IHdpbGwgZ2l2ZSB1cyBtb3JlIGJlbmVmaXQgKGluIFRMQgptaXNzZXMgcmF0ZSwKZXRjLikuIEZv ciBleGFtcGxlLCBhIDJNQiBwYWdlIG1heSBob3N0IHRleHQgb2YgdHdvIGtlcm5lbCBtb2R1bGVz LCA0IGtwcm9iZXMsCjYgZnRyYWNlIHRyYW1wb2xpbmVzLCBhbmQgMTAgQlBGIHByb2dyYW1zLiBB bGwgb2YgdGhlc2Ugb25seSByZXF1aXJlIG9uZSBlbnRyeQppbiB0aGUgaVRMQi4KClRoYW5rcywK U29uZwoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlu dXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRl YWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgt YXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C50D2C77B73 for ; Tue, 6 Jun 2023 18:23:10 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4QbJkP0b8xz3f04 for ; Wed, 7 Jun 2023 04:23:09 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=ZWPc2Ay9; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=song@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=ZWPc2Ay9; dkim-atps=neutral Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4QbJjQ08g4z2xHb for ; Wed, 7 Jun 2023 04:22:17 +1000 (AEST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E3E26636A6 for ; Tue, 6 Jun 2023 18:22:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 278B9C43321 for ; Tue, 6 Jun 2023 18:22:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686075734; bh=08bgDTu6K2r14VqmkEIMfCPJQ/ZZrnAx7Zh60inB9tQ=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=ZWPc2Ay9qV+46W5EIl1Hat/oWhCmkf2atoI7Djyx7M9K8nTcPWM51zO0JgXQR/xJA t6kkVflNQu8aEchQ0gWokejvbn0lD4N+mkDVLz/vpv4NfXuw4Ue4w5/ggGi9dBfKiZ FbyGdgANCc28tlTGlHG3sjNhVLTPxaSnMiPYhxWksxnkaN6w7Z8t8XpDCqJBTwrewz 7miB72dJ8VC2moDFlWb2Dg+2pHaD8TB88qpU65gh5ZEErvyTm+QOEYn4FI4IN5NnX+ vSaQ/C5uDNk1W0XxM90aSXxJjvBdgk4T++XB8N2/4BPrh3DvP9IK4m5Qlazjdmj/3m EHdGa+FT3FpmQ== Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-2b1a4250b07so75710931fa.3 for ; Tue, 06 Jun 2023 11:22:14 -0700 (PDT) X-Gm-Message-State: AC+VfDxg9qRj/HOyLl3qwJdCWVqYWS+Rh1Lg9dmVcvVPLozSQ+K4k+Nr tjHnrH2UeXNPgqjC7Vg09p0kOE2K6imbnl2yaA4= X-Google-Smtp-Source: ACHHUZ5bLSr3tA3IB8xCjeC0rEYdnEDeq7vmc1y/QuS5oHjyhnnuqZDpSw2PwmODz4CMTnRxT8zG59Qp1zLSa+hdG70= X-Received: by 2002:a2e:82d0:0:b0:2b0:297c:cbdf with SMTP id n16-20020a2e82d0000000b002b0297ccbdfmr1546782ljh.1.1686075731505; Tue, 06 Jun 2023 11:22:11 -0700 (PDT) MIME-Version: 1.0 References: <20230601101257.530867-1-rppt@kernel.org> <20230605092040.GB3460@kernel.org> In-Reply-To: From: Song Liu Date: Tue, 6 Jun 2023 11:21:59 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 00/13] mm: jit/text allocator To: Mark Rutland Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: x86@kernel.org, Catalin Marinas , linux-mips@vger.kernel.org, linux-mm@kvack.org, sparclinux@vger.kernel.org, linux-riscv@lists.infradead.org, Will Deacon , linux-s390@vger.kernel.org, Helge Deller , Huacai Chen , Russell King , "Naveen N. Rao" , linux-trace-kernel@vger.kernel.org, Heiko Carstens , Steven Rostedt , loongarch@lists.linux.dev, Thomas Gleixner , Andrew Morton , linux-arm-kernel@lists.infradead.org, Thomas Bogendoerfer , linux-parisc@vger.kernel.org, netdev@vger.kernel.org, Kent Overstreet , linux-kernel@vger.kernel.org, Dinh Nguyen , Luis Chamberlain , Palmer Dabbelt , linux-modules@vger.kernel.org, bpf@vger.kernel.org, linuxppc-dev@lists.o zlabs.org, "David S. Miller" , Mike Rapoport Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, Jun 5, 2023 at 3:09=E2=80=AFAM Mark Rutland = wrote: [...] > > > > Can you give more detail on what parameters you need? If the only e= xtra > > > > parameter is just "does this allocation need to live close to kerne= l > > > > text", that's not that big of a deal. > > > > > > My thinking was that we at least need the start + end for each caller= . That > > > might be it, tbh. > > > > Do you mean that modules will have something like > > > > jit_text_alloc(size, MODULES_START, MODULES_END); > > > > and kprobes will have > > > > jit_text_alloc(size, KPROBES_START, KPROBES_END); > > ? > > Yes. How about we start with two APIs: jit_text_alloc(size); jit_text_alloc_range(size, start, end); AFAICT, arm64 is the only arch that requires the latter API. And TBH, I am not quite convinced it is needed. > > > It sill can be achieved with a single jit_alloc_arch_params(), just by > > adding enum jit_type parameter to jit_text_alloc(). > > That feels backwards to me; it centralizes a bunch of information about > distinct users to be able to shove that into a static array, when the cal= lsites > can pass that information. I think we only two type of users: module and everything else (ftrace, kpro= be, bpf stuff). The key differences are: 1. module uses text and data; while everything else only uses text. 2. module code is generated by the compiler, and thus has stronger requirements in address ranges; everything else are generated via some JIT or manual written assembly, so they are more flexible with address ranges (in JIT, we can avoid using instructions that requires a specific address range). The next question is, can we have the two types of users share the same address ranges? If not, we can reserve the preferred range for modules, and let everything else use the other range. I don't see reasons to further separate users in the "everything else" group. > > What's *actually* common after separating out the ranges? Is it just the > permissions? I believe permission is the key, as we need the hardware to enforce permission. > > If we want this to be able to share allocations and so on, why can't we d= o this > like a kmem_cache, and have the callsite pass a pointer to the allocator = data? > That would make it easy for callsites to share an allocator or use a dist= inct > one. Sharing among different call sites will give us more benefit (in TLB misses rate, etc.). For example, a 2MB page may host text of two kernel modules, 4 kprob= es, 6 ftrace trampolines, and 10 BPF programs. All of these only require one en= try in the iTLB. Thanks, Song