From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E916C54EE9 for ; Tue, 27 Sep 2022 16:45:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A3808E00E5; Tue, 27 Sep 2022 12:45:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97A368E00C1; Tue, 27 Sep 2022 12:45:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 841108E00E5; Tue, 27 Sep 2022 12:45:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 74B858E00C1 for ; Tue, 27 Sep 2022 12:45:19 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 48E4A16023E for ; Tue, 27 Sep 2022 16:45:19 +0000 (UTC) X-FDA: 79958440758.10.6E0424E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id E66CE40008 for ; Tue, 27 Sep 2022 16:45:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664297118; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WTa5rbKShIyGoJknw92BfG1x+s2qAF7vnnJsM/DzVR4=; b=LpP7nOlxuNbJNFyptJwYymEJT9Q8mYfck6+SypPhex+kCPgxIw7gGDks45ulj581Chv+26 RnnDjQ/mCx1CF5oMM5EXIfaJGN2xNGbFAomZbYpSaGzLM42Zi2AnYn5RocSQ4qZrIqPBQ5 F3rnQFTA8LonNS97DHeL08AVnf4SCqc= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-261-JTNtLAECMAumDHGXQYzNnA-1; Tue, 27 Sep 2022 12:45:17 -0400 X-MC-Unique: JTNtLAECMAumDHGXQYzNnA-1 Received: by mail-qt1-f199.google.com with SMTP id v9-20020a05622a188900b0035cc030ca25so7108195qtc.1 for ; Tue, 27 Sep 2022 09:45:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date; bh=WTa5rbKShIyGoJknw92BfG1x+s2qAF7vnnJsM/DzVR4=; b=aXEdhVFUfezopxUGA+JZYwcPyyp+/TpDo+LeJmSNCg1GR2VrwPX+4d0hX6EDRujeoi pLC91w6E/pOp8jdffxpJVa2abvkTr8j75q878cJuYjgPUJ/IDW/HyrgGRl30QDiPCYgw jxrKUHuDxZXIHwCNc+0Pt8hk7kKLWtmuSe0fLtd/7VO69OtNPE/ytxrqKdL2RWS9yw/5 LyIGhjVEqJb3AMn7KYEHbc7q5zYGzFOuBpP3gLnawsvkrhEfvUr9IJ5h/nBG7dVXWdgh AIfWIrz2gPGTi2i2h7RGnHQarKfhWfg0ik43ztntWc8uGEKcnRKVPE1zdV4yCqmTC0gG 7HZw== X-Gm-Message-State: ACrzQf0MxRr/m9QtnmwWEsHSjJVKsYMJNYSPVkBuC8OU1NPrUt/RnlNP /n6oYuy9PNxgbgtuy1Y78ksjfVm3ZGdd7IwbB1uBjC8VWk3GxnavZVJAzf2CtpktGIDOVntULKL KrV6hiRrGwa0= X-Received: by 2002:a05:620a:2601:b0:6bc:70bb:c56b with SMTP id z1-20020a05620a260100b006bc70bbc56bmr18970166qko.416.1664297116783; Tue, 27 Sep 2022 09:45:16 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5AgLFJmBdANYEOVOVCK9odrG/eAgjCtDCyK2CtvVPq8D05QAsmkYI+yJDhMCoTJ31v1Bm/4A== X-Received: by 2002:a05:620a:2601:b0:6bc:70bb:c56b with SMTP id z1-20020a05620a260100b006bc70bbc56bmr18970133qko.416.1664297116560; Tue, 27 Sep 2022 09:45:16 -0700 (PDT) Received: from x1n (bras-base-aurron9127w-grc-46-70-31-27-79.dsl.bell.ca. [70.31.27.79]) by smtp.gmail.com with ESMTPSA id a19-20020ac84d93000000b00342fc6a8e25sm1082638qtw.50.2022.09.27.09.45.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Sep 2022 09:45:16 -0700 (PDT) Date: Tue, 27 Sep 2022 12:45:13 -0400 From: Peter Xu To: Mike Kravetz Cc: Hugh Dickins , Axel Rasmussen , Yang Shi , Matthew Wilcox , syzbot , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, nathan@kernel.org, ndesaulniers@google.com, songmuchun@bytedance.com, syzkaller-bugs@googlegroups.com, trix@redhat.com Subject: Re: [syzbot] general protection fault in PageHeadHuge Message-ID: References: <0000000000006c300705e95a59db@google.com> <7693a84-bdc2-27b5-2695-d0fe8566571f@google.com> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664297119; a=rsa-sha256; cv=none; b=TVVyxP4VIla4ZtXjWt+viJCobaJbhlO4+4zjJ5VTwU5TXthJirUqzhkx8/syz6m4LcmeR4 SxrwFM0zvuv23w1aP7/vEHGVN8RmlvSVlTp6AZ958bSzqrXT4pYNsAcjQ3VFA75w3leJm0 dD4ZiOV2UOZZac3S31H677TMTg+P8hY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LpP7nOlx; spf=pass (imf12.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664297119; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WTa5rbKShIyGoJknw92BfG1x+s2qAF7vnnJsM/DzVR4=; b=qJUCxB+Lxztz62e0f+f4f/TAJ85ypqEBeeJoqoXQeUbRTrKhrwrip5zqfRNpUgSqRlHsy1 H4BL9qm/YIzj3aqeJltMxbBkOLRmhquiKd91rUddMSqmMXojXjjRM3s5vf5QkRoDr/AW/l +6POtZOUgeI3/SYVayRVYabh+fhG2AE= X-Stat-Signature: f9fy1w9zsfj1u8yhg81obbf7kwtoo97b X-Rspamd-Queue-Id: E66CE40008 X-Rspam-User: Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LpP7nOlx; spf=pass (imf12.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam01 X-HE-Tag: 1664297118-86303 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Sep 27, 2022 at 09:24:37AM -0700, Mike Kravetz wrote: > This should guarantee a read fault independent of what pthread_mutex_lock > does. However, it still results in the occasional "ERROR: unexpected write > fault". So, something else if happening. I will continue to experiment > and think about this. Thanks for verifying this, Mike. I didn't yet reply but I did have some update on my side too. I plan to look deeper and wanted to reply until that, because I do think there's something special on hugetlb and I still don't know. I just keep getting distracted by other things.. but maybe I should still share out what I have already. I think I already know what had guaranteed the read faults - the NPTL pthread lib implemented mutex in different types, and the 1st instruction of lock() is to fetch the mutex type (at offset 0x10) then we know how we should move on: (gdb) disas pthread_mutex_lock Dump of assembler code for function ___pthread_mutex_lock: 0x00007ffff7e3b0d0 <+0>: endbr64 0x00007ffff7e3b0d4 <+4>: mov 0x10(%rdi),%eax <---- read 0x10 of &mutex 0x00007ffff7e3b0d7 <+7>: mov %eax,%edx 0x00007ffff7e3b0d9 <+9>: and $0x17f,%edx (gdb) ptype pthread_mutex_t type = union { struct __pthread_mutex_s __data; char __size[40]; long __align; } (gdb) ptype struct __pthread_mutex_s type = struct __pthread_mutex_s { int __lock; unsigned int __count; int __owner; unsigned int __nusers; int __kind; <--- 0x10 offset here short __spins; short __elision; __pthread_list_t __list; } I looked directly at asm dumped from the binary I tested to make sure it's accurate. So it means with current uffd selftest logically there should never be a write missing fault (I still don't think it's good to have the write check though.. but that's separate question to ask). It also means hugetlb does something special here. It smells really like for some reason the hugetlb pgtable got evicted after UFFDIO_COPY during locking_thread running, then any further lock() (e.g. cmpxchg) or modifying the counter could trigger a write fault. OTOH this also explained why futex code was never tested on userfaultfd selftest, simply because futex() will always to be after that "read the type of mutex" thing.. which is something I want to rework a bit, so as to have uffd selftest to cover gup as you used to rightfully suggest. But that'll be nothing urgent, and be something after we know what's special with hugetlb new code. I'll also keep update if I figured something more out of it. -- Peter Xu