From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4BEBC433EF for ; Wed, 23 Mar 2022 11:51:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28B586B0072; Wed, 23 Mar 2022 07:51:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 239B56B0073; Wed, 23 Mar 2022 07:51:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08DB36B0074; Wed, 23 Mar 2022 07:51:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id E9FAC6B0072 for ; Wed, 23 Mar 2022 07:51:46 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 94EDF8249980 for ; Wed, 23 Mar 2022 11:51:46 +0000 (UTC) X-FDA: 79275486612.24.2C381CD Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150049.outbound.protection.outlook.com [40.107.15.49]) by imf15.hostedemail.com (Postfix) with ESMTP id 7A26FA002C for ; Wed, 23 Mar 2022 11:51:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cODtzpkO3uO5BNXI1yKypjrO0MTH+I5c1NCmnALPLBE=; b=zwvHH4+TKT+HYgkL7Lnlto+Un2KUUMkLLG+0i8DfQSa9bh5TNF8cVrQIEz+efYx2yIxqJtFq6LSp2IYTyi4RK9KUvQdt3aL+1Bx0yywXXoX8rxJlWCDfPSRsIM0mKnX5rxb0XJXz7tSScok2GfUzx4kAaTUNm8yr1i7qVXk+Bx4= Received: from AS9PR06CA0187.eurprd06.prod.outlook.com (2603:10a6:20b:45d::21) by DBBPR08MB6186.eurprd08.prod.outlook.com (2603:10a6:10:204::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Wed, 23 Mar 2022 11:51:42 +0000 Received: from AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:45d:cafe::61) by AS9PR06CA0187.outlook.office365.com (2603:10a6:20b:45d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.21 via Frontend Transport; Wed, 23 Mar 2022 11:51:42 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT063.mail.protection.outlook.com (10.152.16.226) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.18 via Frontend Transport; Wed, 23 Mar 2022 11:51:42 +0000 Received: ("Tessian outbound 826a6d8e58c3:v113"); Wed, 23 Mar 2022 11:51:42 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: b40cfa086f05ca44 X-CR-MTA-TID: 64aa7808 Received: from 90f41b31d6d7.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E0EC43A9-1CF4-4313-B405-F8A886303DB1.1; Wed, 23 Mar 2022 11:51:40 +0000 Received: from EUR01-VE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 90f41b31d6d7.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 23 Mar 2022 11:51:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kpUkaEF7wFgyEl84GXpFlRkfCzCGPNQKUk+HyyFKitZkHOF/177WYkGX9TDmgh01C9KgO0Ov5cwRpi++rFGFKbgEMKb7yemnGftbXfWkYS3BppNQaYa8x2qlZy/kYX91t16vb5f1rA5uVq0Ou/KZXnRZOsuWrNBp/zgzWWE/Lt/UxPXe4KJ5q8JCNBPXCpwnJbNSK4LWXraMlH3ppOrmDkJipePYDyolQc8NXrDmu9ravc9EukH4rQvuwJiKGJ9XTLZFChpCL1bx2aXrexr5U2qMB/1dpMdrZiayDrK0PBXxW+FkQncbXr/ygEFnTkGk9sc1EcPbuLrtlZxTH4ID/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cODtzpkO3uO5BNXI1yKypjrO0MTH+I5c1NCmnALPLBE=; b=hnUwiqepx6OyjXi0Zaj1854fhJ7hPNtJbrzZqjHSGYY2+HsZ8Kf9W9T3kU9yC90J6xOrs7UTcYQI9JLnD5RFTLGarenSEu1i/+PHYaIF08XxElcOuCivU8lucIa4afz1PrNjjSyGLRq73VmEssCuKg71Moed36K+1BvvzX5OURsmBapqru/H6z8msH/ZLK+zOtJjVCrxSKA2T/gw/7LeyCGq+4HPCK9KQTISL9XFa8mZvVGilqDzF/Tk/7Rdhh9r2tfLDrFI9LUuSmwm9GTObHaiGFZAeNyl60zB3P+SH4ZDypQXcnxStYXhLPo313CelW/1vhiZI84tXtQBrZiW3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cODtzpkO3uO5BNXI1yKypjrO0MTH+I5c1NCmnALPLBE=; b=zwvHH4+TKT+HYgkL7Lnlto+Un2KUUMkLLG+0i8DfQSa9bh5TNF8cVrQIEz+efYx2yIxqJtFq6LSp2IYTyi4RK9KUvQdt3aL+1Bx0yywXXoX8rxJlWCDfPSRsIM0mKnX5rxb0XJXz7tSScok2GfUzx4kAaTUNm8yr1i7qVXk+Bx4= Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; Received: from AS8PR08MB7028.eurprd08.prod.outlook.com (2603:10a6:20b:34f::8) by DB6PR0801MB1766.eurprd08.prod.outlook.com (2603:10a6:4:3c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.17; Wed, 23 Mar 2022 11:51:29 +0000 Received: from AS8PR08MB7028.eurprd08.prod.outlook.com ([fe80::bc91:3925:e9dc:a851]) by AS8PR08MB7028.eurprd08.prod.outlook.com ([fe80::bc91:3925:e9dc:a851%8]) with mapi id 15.20.5102.017; Wed, 23 Mar 2022 11:51:28 +0000 Message-ID: Date: Wed, 23 Mar 2022 11:51:25 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: VM_BUG_ON(!tlb->end) on munmap() with CONT hugetlb pages To: Catalin Marinas , David Hildenbrand Cc: Mike Kravetz , "linux-mm@kvack.org" , anshuman.khandual@arm.com, Will Deacon , "Aneesh Kumar K . V" , Peter Zijlstra , nd@arm.com References: <811c5c8e-b3a2-85d2-049c-717f17c3a03a@redhat.com> <993f1258-6550-e5d7-1e6f-72e2a24b60f0@oracle.com> <3ba18a1d-d5d8-558f-9576-8119c210e98a@oracle.com> From: Steve Capper In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P123CA0477.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a8::14) To AS8PR08MB7028.eurprd08.prod.outlook.com (2603:10a6:20b:34f::8) MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 06ff9f59-fe7e-4bc1-b945-08da0cc37f79 X-MS-TrafficTypeDiagnostic: DB6PR0801MB1766:EE_|AM5EUR03FT063:EE_|DBBPR08MB6186:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: qkG1c+qvO12HqH5VcyKamTI5WNjpsTRylQpZ8VWd9HLyuUglfi8HPC9Pj0UafhLPprY3efJHsiAw4KjtgQavKGbvyvaAgYMpfv31pTP8McDMz5qKlxLieXG88DdVJGxqsx8GmPdBLeKp9J7YwQcfYRDmVFfSanZmRSOpz0mVwW56Qq2QpeYCLu0ymtFuBTKgVSCGgVPd+wizEZ5NHJ7jNEkZnWRljscU2+wbuIoIWyrmNbiWsLCVDVOhsQpviqr027Tr0zc0gD1VHgbef1bIA0YB1i68gSqQ4gKemIQLtYkaKZf0dsAUSuQgOUQB3h640usTfsqDWDkkMjYP5Yy8UJepnf0SA5RhE76YCs4i60duwYndoJpYXGZAmFIiJluwseGYt5q7vUhVu2JZpckh3s+jpBPYS2XinRpe9ZmSn+tpHD3a8S2KclXbZpCvLEWf+g/8H+aHZUpn5U70P675cbPB8eXvX7x0viqm/C4KnalYYgGmkK+TAqD2vLZ5rc+ftjGVq8VajY3xMCWpqECtAtQnQE0LFcO83/6h6Usoe45Ip7GtDha7lnw4Db3q7L7uFgsmabPF58nZGRQRHboh5KaGVrXuuAO7+ObGmQktvm4iVTout8bqc4SjAsb5J6Krz4Uuop+EX3hTtiJql8/pgStDNpYPMc77LI/wg+5z/7wXVItK9prigu5mFVMbBYv40dw6Ji3Yr8xX4FC0zd9o0K3vktdBLRwT0d+DFaERcuM= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:AS8PR08MB7028.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(66476007)(4326008)(6486002)(66946007)(45080400002)(186003)(26005)(2616005)(38100700002)(8676002)(5660300002)(31686004)(6666004)(66556008)(44832011)(508600001)(36756003)(2906002)(8936002)(83380400001)(6512007)(54906003)(316002)(53546011)(31696002)(6506007)(86362001)(110136005)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1766 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 9a7737a9-7f58-44a5-e232-08da0cc3770b X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7nrciAyto5JDWWgEg9begLlewM45ly2HeGCWMPYKMxZPw7Z5+y+PU9Lra7lCC4xUQ3oyyV9qN1354NuX+iduIgzcwxemruY7Emxjgyq9s3qJPlutZlu/MuNN+FZyxuFRPouldOen5ZNQHPMPYQjx9XaeMxs1+GGCztJ8SP7oFDCyGxiAM/Y7rrGi5mZ/2VI4JnFuzgZAPErKdd6LeH3s2w2FdYjKOXOBGCv1MaeeXIsc4CpaHQPYw+j8QIUOv9z/pM328urtBOz13naG3LF0gcUuKRgOjQLavdsTMlsdRa1TZ85henhqHNfua0JdfERf5Wb9/9LFyKkohzas9GoiH/t2ibDspn9ruiDdZZOJl3N8feVFgVLxr8qjSOK5/Ccof4LBZVT87PFL4oB+4vvze0TLtxdKGs74qSdyBPqJHk3I6jIQcZ7NReGRQ+d09g5uZlEbqUeirVfcn8mjD+HV9LiAiKCYP6F1nz+m6Ry0pDAxgRlOWkk9yGlAIGXetl/Rqck0LqBZpy98t0ApeGY6FcMTxUbxJg3T+pCzFx4CPFtQwWNg47t9QVM8taR8MdCXWsoBWgqtiyWn2IX2shoB3kCaSqKmcrBRrk31rmS2AUD81IV3H+WLn7/1VAAE6XR0KMaXxKIPBEuHPe8z3BEKBZ7J4tQtYNNWa5fyzy8jIEj/nHdRGzd88UxX2nkVSp4O2LWtA9HAdWUVkyaB1hWe0g== X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(47076005)(8936002)(356005)(4326008)(5660300002)(86362001)(82310400004)(83380400001)(6666004)(26005)(6512007)(186003)(53546011)(40460700003)(81166007)(336012)(6506007)(2616005)(44832011)(110136005)(508600001)(70586007)(54906003)(70206006)(31686004)(8676002)(45080400002)(316002)(31696002)(6486002)(2906002)(36756003)(36860700001)(43740500002);DIR:OUT;SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2022 11:51:42.2447 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 06ff9f59-fe7e-4bc1-b945-08da0cc37f79 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT063.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6186 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: abfiaip1p86ow9trsekecntkjr4xmwac Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=zwvHH4+T; dkim=pass header.d=armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=zwvHH4+T; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of Steve.Capper@arm.com designates 40.107.15.49 as permitted sender) smtp.mailfrom=Steve.Capper@arm.com X-Rspamd-Queue-Id: 7A26FA002C X-HE-Tag: 1648036305-671663 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 22/03/2022 17:56, Catalin Marinas wrote: > Adding Steve as well who wrote the initial hugetlb code for arm64 (and > not trimming the quoted text). > Hey Catalin, > On Mon, Mar 21, 2022 at 05:34:18PM +0100, David Hildenbrand wrote: >> On 08.03.22 00:06, Mike Kravetz wrote: >>> On 2/28/22 16:26, Mike Kravetz wrote: >>>> On 2/28/22 07:39, David Hildenbrand wrote: >>>>> playing with anonymous CONT hugetlb pages on aarch64, I stumbled over the following VM_BUG_ON: >>>>> >>>>> [ 124.770288] ------------[ cut here ]------------ >>>>> [ 124.774899] kernel BUG at mm/mmu_gather.c:70! >>>>> [ 124.779244] Internal error: Oops - BUG: 0 [#1] SMP >>>>> [ 124.784022] Modules linked in: mlx4_ib ib_uverbs ib_core mlx4_en rfkill vfat fat acpi_ipmi joydev ipmi_ssif igb mlx4_core ipmi_devintf ipmi_msghandler cppc_cpufreq fuse zram ip_tables xfs uas usb_storage dwc3 ulpi ast udc_core i2c_algo_bit drm_vram_helper drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec drm_ttm_helper ttm crct10dif_ce drm ghash_ce sbsa_gwdt i2c_xgene_slimpro xgene_hwmon ahci_platform gpio_dwapb xhci_plat_hcd >>>>> [ 124.823045] CPU: 16 PID: 1160 Comm: test Not tainted 5.16.11-200.fc35.aarch64 #1 >>>>> [ 124.830428] Hardware name: Lenovo HR350A 7X35CTO1WW /HR350A , BIOS hve104r-1.15 02/26/2021 >>>>> [ 124.840240] pstate: 40400005 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) >>>>> [ 124.847189] pc : __tlb_remove_page_size+0x88/0xe4 >>>>> [ 124.851885] lr : __unmap_hugepage_range+0x260/0x504 >>>>> [ 124.856751] sp : ffff80000f6f3ae0 >>>>> [ 124.860053] x29: ffff80000f6f3ae0 x28: ffff00080b639d24 x27: ffff000802504080 >>>>> [ 124.867176] x26: fffffc00210f8000 x25: 0000000000000000 x24: ffff80000a9e8750 >>>>> [ 124.874299] x23: 0000ffff8da20000 x22: ffff000804f0c190 x21: 0000000000010000 >>>>> [ 124.881423] x20: ffff80000f6f3cb0 x19: ffff80000f6f3cb0 x18: 0000000000000000 >>>>> [ 124.888545] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 >>>>> [ 124.895668] x14: 0000000000000000 x13: 0008000000000000 x12: 0008000000000080 >>>>> [ 124.902791] x11: 0008000000000000 x10: 00f80008c3e00f43 x9 : ffff800008404e60 >>>>> [ 124.909914] x8 : 0846000000000000 x7 : 0000000000000000 x6 : ffff80000a8a4000 >>>>> [ 124.917038] x5 : 0000000000000040 x4 : 0000000000000000 x3 : 0000000000001000 >>>>> [ 124.924161] x2 : 0000000000010000 x1 : fffffc00210f8000 x0 : 0000000000000000 >>>>> [ 124.931284] Call trace: >>>>> [ 124.933718] __tlb_remove_page_size+0x88/0xe4 >>>>> [ 124.938062] __unmap_hugepage_range+0x260/0x504 >>>>> [ 124.942580] __unmap_hugepage_range_final+0x24/0x40 >>>>> [ 124.947445] unmap_single_vma+0x100/0x11c >>>>> [ 124.951443] unmap_vmas+0x7c/0xf4 >>>>> [ 124.954746] unmap_region+0xa4/0xf0 >>>>> [ 124.958222] __do_munmap+0x1b8/0x50c >>>>> [ 124.961785] __vm_munmap+0x74/0x120 >>>>> [ 124.965261] __arm64_sys_munmap+0x40/0x54 >>>>> [ 124.969257] invoke_syscall+0x50/0x120 >>>>> [ 124.972995] el0_svc_common.constprop.0+0x4c/0x100 >>>>> [ 124.977774] do_el0_svc+0x34/0xa0 >>>>> [ 124.981077] el0_svc+0x30/0xd0 >>>>> [ 124.984120] el0t_64_sync_handler+0xa4/0x130 >>>>> [ 124.988377] el0t_64_sync+0x1a4/0x1a8 >>>>> [ 124.992028] Code: b4000140 f9001660 29410402 17fffff4 (d4210000) >>>>> [ 124.998109] ---[ end trace a74a76b89c9f2d88 ]--- >>>>> [ 125.002900] ------------[ cut here ]------------ >>>>> >>>>> >>>>> I'm running with 64k hugetlb on 4k aarch64. Reproducer: >>>>> >>>>> #define _GNU_SOURCE >>>>> #include >>>>> #include >>>>> #include >>>>> #include >>>>> >>>>> void main(void) >>>>> { >>>>> const size_t size = 64*1024; >>>>> unsigned long cur; >>>>> char *area; >>>>> int fd; >>>>> >>>>> fd = memfd_create("test", MFD_HUGETLB | MFD_HUGE_64KB); >>>>> ftruncate(fd, size); >>>>> area = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0); >>>>> >>>>> memset(area, 0, size); >>>>> >>>>> munmap(area, size); >>>>> } >>>>> >>>>> >>>>> >>>>> I assume __unmap_hugepage_range() does a >>>>> >>>>> a) tlb_remove_huge_tlb_entry() >>>>> >>>>> -> for sz != PMD_SIZE and sz != PUD_SIZE, this calls __tlb_remove_tlb_entry() >>>>> >>>>> -> __tlb_remove_tlb_entry() is a NOP on aarch64. __tlb_adjust_range() isn't called. >>>>> >>>>> b) tlb_remove_page_size() >>>>> >>>>> -> __tlb_remove_page_size() runs into VM_BUG_ON(!tlb->end); >>>>> >>>>> >>>>> Not sure if this is just "ok" and we don't have to adjust the range or if there is >>>>> some tlb range adjustment missing. >>>>> >>>> >>>> To me, it looks like we are missing range adjustment in the case where >>>> hugetlb page size != PMD_SIZE and != PUD_SIZE. Not sure how those ranges >>>> are being flushed because as you note tlb_remove_huge_tlb_entry is pretty >>>> much a NOP in this case on aarch64. >>>> >>>> Cc'ing Will and Peter as they most recently changed this code. Commit >>>> 2631ed00b049 "tlb: mmu_gather: add tlb_flush_*_range APIs" removed an >>>> unconditional call to __tlb_adjust_range() in tlb_remove_huge_tlb_entry. >>>> That might have taken care of range adjustments in earlier versions of >>>> the code. Not exactly sure what is needed now. >>> >>> I verified that commit 2631ed00b049 caused the VM_BUG when it removed the >>> unconditional call to __tlb_adjust_range(). However, I need some assistance >>> on the proper solution. >>> >>> Just adding the __tlb_adjust_range() call to tlb_remove_huge_tlb_entry in >>> the case where size != PMD_SIZE and != PUD_SIZE will silence the BUG. >>> However, one outcome of 2631ed00b049 is that cleared_p* is set if >>> __tlb_adjust_range is ever called. >>> >>> It 'seems' that tlb_flush_pte_range() should be called for the CONT PTE case >>> on arm64, and tlb_flush_pmd_range() should be called for CONT PMD. But, this >>> would require an arch specific version of tlb_remove_huge_tlb_entry. >>> >>> FYI - This same issue should exist on other architectures that support >>> hugetlb page sizes != PMD_SIZE and != PUD_SIZE. >>> >>> Suggestions on how to proceed? >> >> Unfortunately, I have absolutely no clue what would be the right thing >> to do. Any aarch64 CONT experts? > > At a quick look, we wouldn't have a problem with missing TLB flushing > since huge_ptep_get_and_clear() does this for contiguous PTEs. Not sure > why it needs this though, Steve added it in commit d8bdcff28764. I think > we can defer this flushing to tlb_remove_page_size(). The TLB flush in huge_ptep_get_and_clear() was added because it was called by hugetlb_change_protection() without any flushing. The concern was that, without the flush, it would be possible to get to different views of the same contiguous huge page. (Being contiguous they were not changed en masse atomically). > > As Mike noted, tlb_remove_huge_tlb_entry() could call > tlb_flush_pte_range() and I think this would work even when dealing with > a CONT PMD case (just more flushing at page granularity). I need to > check the range TLBI ops in the arm64 __flush_tlb_range() but if they > are fine, we can do this as a quick fix. > > A better solution is probably to allow arch-specific > tlb_remove_huge_tlb_entry() that understands whether it's a contiguous > pmd or pte and sets the clear_p* accordingly. > Yeah an arch specific tlb_remove_huge_tlb_entry() would make sense. The current one assumes huge ptes can either be PMD_SIZE or PUD_SIZE. Cheers, -- Steve