From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09E71C432BE for ; Fri, 20 Aug 2021 18:59:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9F74160F5E for ; Fri, 20 Aug 2021 18:59:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9F74160F5E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id F3EF26B0071; Fri, 20 Aug 2021 14:59:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEF4C8D0001; Fri, 20 Aug 2021 14:59:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDD5E6B0073; Fri, 20 Aug 2021 14:59:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id C29E66B0071 for ; Fri, 20 Aug 2021 14:59:53 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 79125182701A4 for ; Fri, 20 Aug 2021 18:59:53 +0000 (UTC) X-FDA: 78496373466.28.C95A3D5 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf09.hostedemail.com (Postfix) with ESMTP id B800D3000105 for ; Fri, 20 Aug 2021 18:59:52 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10082"; a="216860368" X-IronPort-AV: E=Sophos;i="5.84,338,1620716400"; d="scan'208";a="216860368" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Aug 2021 11:59:48 -0700 X-IronPort-AV: E=Sophos;i="5.84,338,1620716400"; d="scan'208";a="533086069" Received: from agluck-desk2.sc.intel.com (HELO agluck-desk2.amr.corp.intel.com) ([10.3.52.146]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Aug 2021 11:59:46 -0700 Date: Fri, 20 Aug 2021 11:59:45 -0700 From: "Luck, Tony" To: Borislav Petkov Cc: Jue Wang , Ding Hui , naoya.horiguchi@nec.com, osalvador@suse.de, Youquan Song , huangcun@sangfor.com.cn, x86@kernel.org, linux-edac@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 1/3] x86/mce: Avoid infinite loop for copy from user recovery Message-ID: <20210820185945.GA1623421@agluck-desk2.amr.corp.intel.com> References: <20210706190620.1290391-1-tony.luck@intel.com> <20210818002942.1607544-1-tony.luck@intel.com> <20210818002942.1607544-2-tony.luck@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: B800D3000105 Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf09.hostedemail.com: domain of tony.luck@intel.com has no SPF policy when checking 134.134.136.65) smtp.mailfrom=tony.luck@intel.com X-Rspamd-Server: rspam04 X-Stat-Signature: po3mbp86tsyccqox44bwi8f9rddnztqf X-HE-Tag: 1629485992-243524 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 20, 2021 at 07:31:43PM +0200, Borislav Petkov wrote: > On Tue, Aug 17, 2021 at 05:29:40PM -0700, Tony Luck wrote: > > + /* Ten is likley overkill. Don't expect more than two faults before task_work() */ > > "likely" Oops. > > > + if (count > 10) > > + mce_panic("Too many machine checks while accessing user data", m, msg); > > Ok, aren't we too nasty here? Why should we panic the whole box even > with 10 MCEs? It is still user memory... > > IOW, why not: > > if (count > 10) > current->mce_kill_me.func = kill_me_now; > > and when we return, that user process dies immediately. It's the "when we return" part that is the problem here. Logical trace looks like: user-syscall: kernel does get_user() or copyin(), hits user poison address machine check sees that this was kernel get_user()/copyin() and uses extable to "return" to exception path still in kernel, see that get_user() or copyin() failed Kernel does another get_user() or copyin() (maybe the first was inside a pagefault_disable() region, and kernel is trying again to see if the error was a fixable page fault. But that wasn't the problem so ... machine check sees that this was kernel get_user()/copyin() and uses extable to "return" to exception path still in kernel ... but persistently thinks that just trying again might fix it. machine check sees that this was kernel get_user()/copyin() and uses extable to "return" to exception path still in kernel ... this time for sure! get_user() machine check sees that this was kernel get_user()/copyin() and uses extable to "return" to exception path still in kernel ... but you may see the pattern get_user() machine check sees that this was kernel get_user()/copyin() and uses extable to "return" to exception path I'm bored typing this, but the kernel may not ever give up machine check sees that this was kernel get_user()/copyin() and uses extable to "return" to exception path I.e. the kernel doesn't ever get to call current->mce_kill_me.func() I do have tests that show as many as 4 consecutive machine checks before the kernel gives up trying and returns to the user to complete recovery. Maybe the message could be clearer? mce_panic("Too many consecutive machine checks in kernel while accessing user data", m, msg); > > > + /* Second or later call, make sure page address matches the one from first call */ > > + if (count > 1 && (current->mce_addr >> PAGE_SHIFT) != (m->addr >> PAGE_SHIFT)) > > + mce_panic("Machine checks to different user pages", m, msg); > > Same question here. Not quite the same answer ... but similar. We could in theory handle multiple different machine check addresses by turning the "mce_addr" field in the task structure into an array and saving each address so that when the kernel eventually gives up poking at poison and tries to return to user kill_me_maybe() could loop through them and deal with each poison page. I don't think this can happen. Jue Wang suggested that multiple poisoned pages passed to a single write(2) syscall might trigger this panic (and because of a bug in my earlier version, he managed to trigger this "different user pages" panic). But this fixed up version survives the "Jue test". -Tony