From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AEC7C6FA8E for ; Thu, 22 Sep 2022 03:10:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CEEF80008; Wed, 21 Sep 2022 23:10:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7588580007; Wed, 21 Sep 2022 23:10:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 582AF80008; Wed, 21 Sep 2022 23:10:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4424D80007 for ; Wed, 21 Sep 2022 23:10:26 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 24C914054B for ; Thu, 22 Sep 2022 03:10:26 +0000 (UTC) X-FDA: 79938243252.17.1E68FB2 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf07.hostedemail.com (Postfix) with ESMTP id C063340014 for ; Thu, 22 Sep 2022 03:10:24 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id rt12so5960039pjb.1 for ; Wed, 21 Sep 2022 20:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=e/QakfBu8OdcLOaNtNAlOZZbARPmNgFJAYuG6V5fVNI=; b=KID4+fNPZ/+EeETfIhlZodisrk94VxxslzZcsqTK2+SnDvmzAz6LYMBzVDpytU/nEq EIXxg57qTfuTgbSUKdc6GxhGn+QXLKP5XVcWuZOc8i9lduGT5GmK9QPkOg0atE9jam9x IA1gpnZyHuwdwIlZEHKE4Y17orFlGUFf4xU0Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=e/QakfBu8OdcLOaNtNAlOZZbARPmNgFJAYuG6V5fVNI=; b=wjUw7F/LRyFx3383qGqHPpyVWQXvms5U0zGeMalXFykny0RJERP8HNzzJtYG6gihTD tpGId5Syov8f6Zycm9OOCVKCO19SfS66zdOUjbWjOylShgpdojSWs1vrC/2c1Gol7mh8 0Re4QbiF0COrv5DoqtqNzNl5mi4TrktLDqaZyoKr8uNgUhQj8DRiSjzKwzeSUHCqr7tL M7tYXiRnYdhF6J5ks1oGkqqzyIZxSXL0wg7pBdbT59bqFNkDs3YtJs+08Bple3WQQCuj VPYm34riOmE61Anl1HOZ9GudkRdoODtn0ltKApYAVDIMHKW5XfulDjnL1qJIZI3tDS+i tVbw== X-Gm-Message-State: ACrzQf2ZPA/vbyiubVsdvZKbUL6036P/P1EpMTGer8G0UaTF+t6THwGc ACgbPzYTvWlaxtKGztbn2XCqIA== X-Google-Smtp-Source: AMsMyM40uPN5zeKYkv78Anmrprlj6jSzqrJtHp0Hc07l03jt/qUE59pMSphisPX0a2uGMaa6VRKIFQ== X-Received: by 2002:a17:90b:3809:b0:202:b482:b7d6 with SMTP id mq9-20020a17090b380900b00202b482b7d6mr12697142pjb.209.1663816223696; Wed, 21 Sep 2022 20:10:23 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id z19-20020a17090abd9300b001fab208523esm2641836pjr.3.2022.09.21.20.10.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Sep 2022 20:10:22 -0700 (PDT) From: Kees Cook To: Vlastimil Babka Cc: Kees Cook , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , netdev@vger.kernel.org, Greg Kroah-Hartman , Nick Desaulniers , David Rientjes , Pekka Enberg , Joonsoo Kim , Andrew Morton , Alex Elder , Josef Bacik , David Sterba , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jesse Brandeburg , Daniel Micay , Yonghong Song , Marco Elver , Miguel Ojeda , Jacob Shin , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-btrfs@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-fsdevel@vger.kernel.org, intel-wired-lan@lists.osuosl.org, dev@openvswitch.org, x86@kernel.org, linux-wireless@vger.kernel.org, llvm@lists.linux.dev, linux-hardening@vger.kernel.org Subject: [PATCH 02/12] skbuff: Proactively round up to kmalloc bucket size Date: Wed, 21 Sep 2022 20:10:03 -0700 Message-Id: <20220922031013.2150682-3-keescook@chromium.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220922031013.2150682-1-keescook@chromium.org> References: <20220922031013.2150682-1-keescook@chromium.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4645; h=from:subject; bh=3WJZlXdhsWxnvdirTV8FcTTBEg/t69yXm1UnFNwV9iU=; b=owEBbQKS/ZANAwAKAYly9N/cbcAmAcsmYgBjK9ISG+mldB4B+ZQUmvF386D3yd+qSk6StyPf1824 8ETszD+JAjMEAAEKAB0WIQSlw/aPIp3WD3I+bhOJcvTf3G3AJgUCYyvSEgAKCRCJcvTf3G3AJs8VD/ 9DiU++7/Ux07qrQmKcm2ezAAgh30rdqxKBsQKDrEL5EOX43CK7LgkmIf5KQ4kOjg1Ex2RxD6Yv+Dfu IDElH/EKSC/tRs1m4xC3h+q+7SbPjgQtmT9a1i7nnOlBSJqcbBqgQZ+Hespu169Fv7Hx8O+weK7bWC muvsyI9vTnW0JpI1rPfLOOuKjJ9sJYhk/W394UUvBZ4qMgXFE8aIbG8xv2R7+NixPSkSyQ6b4Is/W/ uWoVPoWJ3Ls+LdnmTnJk6b/E3AYpCLPbBzVShs8FTktYCaTY3VT44AcqfR9kRa2oc56ACWEfbkOpGV 9Gdqub12EZ8F5YMvtWlggw8p51Y10wSTZMspf3mp9LrJ+nx5uZjuzI1PVjfwxQXMRJgEhTkhCDDSuv m+dzcrKowiJxgMAkwOeKwQ8qujiIBEiZUXYyh+XMBd6lv3eqaKtmbfVjheVmcOKPdIHpDXFeAi2TKg 8ifxHByFdWEpvyMobgH17zpUhVJO9SadJqpr6eeqxOjE1ZoxhVgQ9O5qvP+z593Jj9IcW1wAszLbTk gH9xXpGnAHbvxSgL9fcZE1eSRiJ5TlYNZ/k5Pg37qUzVRuuybsmEFKzLhX1Uy/JT/L3Q8EfXaLuynn pjk6/2IqlrRV1hBXe63De0ndNRPpfun/+iA/10deHRQwZNQRHdL3HARlGZMA== X-Developer-Key: i=keescook@chromium.org; a=openpgp; fpr=A5C3F68F229DD60F723E6E138972F4DFDC6DC026 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663816224; a=rsa-sha256; cv=none; b=klW+vyBaDdj2rPddOuBkkTXGDb/5QMWM5aZMG98Iqv12AjBia4bvw2LAk6/YM4hGmmu/Yn F3+j0QmqHmK/ycN8HIt7DO58OS1qCizrI2SrSXvtvK16P5CZyfyKAU+Rt6btPTJI0Nm9Gq R06+u7Fdpg26CRCw9tAnDscIie6wnpU= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=KID4+fNP; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf07.hostedemail.com: domain of keescook@chromium.org designates 209.85.216.43 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663816224; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e/QakfBu8OdcLOaNtNAlOZZbARPmNgFJAYuG6V5fVNI=; b=CfisahEzVvu8xt8Pi1z6BpAvH3CR7m2eh6v/qu4xnuDsGC1ZWeU5MzwVpdb43aWUOQOrC/ dhMrCm6LT1di6b2mXZqaJmaz5MqZocw2wvgs4Ra+hMWSH4ICF/Ca1YduCrdN0+u0t6LaEe xOsC1JFOE10GySbQ7iywvptLCU/nD/w= X-Stat-Signature: axh89wjmys44ydh8bnkousd53up44t5w X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=KID4+fNP; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf07.hostedemail.com: domain of keescook@chromium.org designates 209.85.216.43 as permitted sender) smtp.mailfrom=keescook@chromium.org X-Rspamd-Queue-Id: C063340014 X-Rspamd-Server: rspam04 X-HE-Tag: 1663816224-336295 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instead of discovering the kmalloc bucket size _after_ allocation, round up proactively so the allocation is explicitly made for the full size, allowing the compiler to correctly reason about the resulting size of the buffer through the existing __alloc_size() hint. This will allow for kernels built with CONFIG_UBSAN_BOUNDS or the coming dynamic bounds checking under CONFIG_FORTIFY_SOURCE to gain back the __alloc_size() hints that were temporarily reverted in commit 93dd04ab0b2b ("slab: remove __alloc_size attribute from __kmalloc_track_caller") Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: netdev@vger.kernel.org Cc: Greg Kroah-Hartman Cc: Nick Desaulniers Cc: David Rientjes Cc: Vlastimil Babka Signed-off-by: Kees Cook --- net/core/skbuff.c | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 974bbbbe7138..4fe4c7544c1d 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -427,14 +427,15 @@ struct sk_buff *__alloc_skb(unsigned int size, gfp_t gfp_mask, */ size = SKB_DATA_ALIGN(size); size += SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); - data = kmalloc_reserve(size, gfp_mask, node, &pfmemalloc); - if (unlikely(!data)) - goto nodata; - /* kmalloc(size) might give us more room than requested. + /* kmalloc(size) might give us more room than requested, so + * allocate the true bucket size up front. * Put skb_shared_info exactly at the end of allocated zone, * to allow max possible filling before reallocation. */ - osize = ksize(data); + osize = kmalloc_size_roundup(size); + data = kmalloc_reserve(osize, gfp_mask, node, &pfmemalloc); + if (unlikely(!data)) + goto nodata; size = SKB_WITH_OVERHEAD(osize); prefetchw(data + size); @@ -1709,6 +1710,7 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, { int i, osize = skb_end_offset(skb); int size = osize + nhead + ntail; + int alloc_size; long off; u8 *data; @@ -1722,11 +1724,11 @@ int pskb_expand_head(struct sk_buff *skb, int nhead, int ntail, if (skb_pfmemalloc(skb)) gfp_mask |= __GFP_MEMALLOC; - data = kmalloc_reserve(size + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), - gfp_mask, NUMA_NO_NODE, NULL); + alloc_size = kmalloc_size_roundup(size + SKB_DATA_ALIGN(sizeof(struct skb_shared_info))); + data = kmalloc_reserve(alloc_size, gfp_mask, NUMA_NO_NODE, NULL); if (!data) goto nodata; - size = SKB_WITH_OVERHEAD(ksize(data)); + size = SKB_WITH_OVERHEAD(alloc_size); /* Copy only real data... and, alas, header. This should be * optimized for the cases when header is void. @@ -6063,19 +6065,19 @@ static int pskb_carve_inside_header(struct sk_buff *skb, const u32 off, int i; int size = skb_end_offset(skb); int new_hlen = headlen - off; + int alloc_size; u8 *data; size = SKB_DATA_ALIGN(size); if (skb_pfmemalloc(skb)) gfp_mask |= __GFP_MEMALLOC; - data = kmalloc_reserve(size + - SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), - gfp_mask, NUMA_NO_NODE, NULL); + alloc_size = kmalloc_size_roundup(size + SKB_DATA_ALIGN(sizeof(struct skb_shared_info))); + data = kmalloc_reserve(alloc_size, gfp_mask, NUMA_NO_NODE, NULL); if (!data) return -ENOMEM; - size = SKB_WITH_OVERHEAD(ksize(data)); + size = SKB_WITH_OVERHEAD(alloc_size); /* Copy real data, and all frags */ skb_copy_from_linear_data_offset(skb, off, data, new_hlen); @@ -6184,18 +6186,18 @@ static int pskb_carve_inside_nonlinear(struct sk_buff *skb, const u32 off, u8 *data; const int nfrags = skb_shinfo(skb)->nr_frags; struct skb_shared_info *shinfo; + int alloc_size; size = SKB_DATA_ALIGN(size); if (skb_pfmemalloc(skb)) gfp_mask |= __GFP_MEMALLOC; - data = kmalloc_reserve(size + - SKB_DATA_ALIGN(sizeof(struct skb_shared_info)), - gfp_mask, NUMA_NO_NODE, NULL); + alloc_size = kmalloc_size_roundup(size + SKB_DATA_ALIGN(sizeof(struct skb_shared_info))); + data = kmalloc_reserve(alloc_size, gfp_mask, NUMA_NO_NODE, NULL); if (!data) return -ENOMEM; - size = SKB_WITH_OVERHEAD(ksize(data)); + size = SKB_WITH_OVERHEAD(alloc_size); memcpy((struct skb_shared_info *)(data + size), skb_shinfo(skb), offsetof(struct skb_shared_info, frags[0])); -- 2.34.1