From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63AF4C4167B for ; Sun, 26 Nov 2023 12:25:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3976B6B0338; Sun, 26 Nov 2023 07:25:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3473F6B0360; Sun, 26 Nov 2023 07:25:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2364F6B0375; Sun, 26 Nov 2023 07:25:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 14C0B6B0338 for ; Sun, 26 Nov 2023 07:25:54 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D8920A0254 for ; Sun, 26 Nov 2023 12:25:53 +0000 (UTC) X-FDA: 81500026986.24.E039BE1 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) by imf05.hostedemail.com (Postfix) with ESMTP id D599F100019 for ; Sun, 26 Nov 2023 12:25:50 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of xueshuai@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=xueshuai@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701001552; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rGv1XIeTgXzrE82QL/lEVzlB5CR7aPW/Sf1F+1TgDNk=; b=01zlANaRSauQfMG1X9bpehXfruDV6YZ2wBiKcBfgUx0WpJIStuTcCgdiiADlkafeWB2yME Oqi+FKJGhkSvD1d2py4mwsoPcc8XEUQ6ecI3pM9+yn811oAzD/Z4rGRgLKry+NKAuW9DKx lC0OcIgELASx1+L+v82th04R5oQNOUo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701001552; a=rsa-sha256; cv=none; b=IePxAGS67UpEN8t1TSPOMB8c3ym3gry0QfUOjWlnKy9aSfKzaVqJPXNSt+rZ5XI6v4jVPW pxIj5KRQnrzLHmz3TeCAHwjv43BT+aTcZI4hqCpy4BrMrHeEmrEuSZ84R+GlI2roxl9sU/ BMWqbWeNnszNPnejy2OP/HwFwfl0vQA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of xueshuai@linux.alibaba.com designates 115.124.30.99 as permitted sender) smtp.mailfrom=xueshuai@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045192;MF=xueshuai@linux.alibaba.com;NM=1;PH=DS;RN=34;SR=0;TI=SMTPD_---0Vx6Gb4W_1701001542; Received: from 30.27.123.85(mailfrom:xueshuai@linux.alibaba.com fp:SMTPD_---0Vx6Gb4W_1701001542) by smtp.aliyun-inc.com; Sun, 26 Nov 2023 20:25:45 +0800 Message-ID: <1048123e-b608-4db1-8d5f-456dd113d06f@linux.alibaba.com> Date: Sun, 26 Nov 2023 20:25:38 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v9 0/2] ACPI: APEI: handle synchronous errors in task work with proper si_code Content-Language: en-US To: Borislav Petkov Cc: rafael@kernel.org, wangkefeng.wang@huawei.com, tanxiaofei@huawei.com, mawupeng1@huawei.com, tony.luck@intel.com, linmiaohe@huawei.com, naoya.horiguchi@nec.com, james.morse@arm.com, gregkh@linuxfoundation.org, will@kernel.org, jarkko@kernel.org, linux-acpi@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-edac@vger.kernel.org, acpica-devel@lists.linuxfoundation.org, stable@vger.kernel.org, x86@kernel.org, justin.he@arm.com, ardb@kernel.org, ying.huang@intel.com, ashish.kalra@amd.com, baolin.wang@linux.alibaba.com, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, lenb@kernel.org, hpa@zytor.com, robert.moore@intel.com, lvying6@huawei.com, xiexiuqi@huawei.com, zhuo.song@linux.alibaba.com References: <20221027042445.60108-1-xueshuai@linux.alibaba.com> <20231007072818.58951-1-xueshuai@linux.alibaba.com> <20231123150710.GEZV9qnkWMBWrggGc1@fat_crate.local> <9e92e600-86a4-4456-9de4-b597854b107c@linux.alibaba.com> <20231125121059.GAZWHkU27odMLns7TZ@fat_crate.local> From: Shuai Xue In-Reply-To: <20231125121059.GAZWHkU27odMLns7TZ@fat_crate.local> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: D599F100019 X-Rspam-User: X-Stat-Signature: fyxuf39qaqyxmhzfuc74atc6a5byahrk X-Rspamd-Server: rspam03 X-HE-Tag: 1701001550-847675 X-HE-Meta: U2FsdGVkX1+cpGct8fAHIVb0jm7rex61QZ5NysmRiSLvYNC5suAtyMb9gXsBAOaAkdHkIaBOQDZJkW+rqMrIug3K5pCwR18lUYTYTI9GWQBklYsiiNO1uwdV/eMgk79kzjHhTPZL8Z6Fm6YANUjs6ZONl5IiE82rTXw3ZwMX3oHo/VJtIw3g/ZO/mI+tRVD4YnCP96Nnje6LaPRrVQy23YrpeBqxjtoR5iehcCoXXyfRxuOjCLGSigsBYsqt7nAKwVcfXaoW+673K9FAeBpxUqNLRh/7K1TVsTDhCGZ9FFtUBf0DF+CWSq5eFWronz0TI9RTBfXqB05kmsl3fkirUCinrKfTlX2/qQCdlStC3eb+4Pq0TRb2o9J0qW1JM41PkNWd4KXAjZ2TYMHUWDnw1cWhxwD66c/04YkNEaFYAla65B7WlCLdOHAsJoUdOeQrom6fz3MOPlVRzsnAlsV/Kzy8gJeF0BGDwUWI0NHJ2PhwwIOnAN4DODetzK747vgOiooyIEGu/hinhiALfgArzjf52VFdKtEwtUawa883M5ITC+zyLpABu+GpvwpFq90RzsGSzvS2a6bZPRg6rou53rQqPCX+yecDovUSt8nClECZcNerXU1kP4mdMCGVtj2pTfMit9H/sqx0lxONKrTrVsd54Do7tybAbLc8JuVDTEfsQaVHtv2IOw5RSbGC89IFu6yGykgDraee08ld71nVe8cgjO8Nz5gHCTKIGPuUj1WVAtai3fV0XW5y0xh1F1Q9I5NTF6rEC3NNJiUEIvthQ/xdE9Nm7aEPNQzvE8SmBUMiXvRKp/OU8zI8mTdDWCr331vOJYw3ssmqn3eD9jr7h6POJNhOj7LrMlxeFh/BrVwXCQgil4NEM25XvsX9bm4KrvLAPLvc1U78xh40HpOrxqINpoxgoEi35I5B4QyAx1wcKSYyHYzWbE174SfQioOd4Ou7Xu28zFwGCfBEpRr WqgAkHZF m3uNP6Ch+x1vD3GkSnPz1tFSt63hyJDWjk6xFfuXQx9tSiPdzo2IHGdUt4qsi3/IojfdbOwBPsbR97zje+Bi1R5WatpbBLU4/2kRmA2PedDTnCnXm9tZSiBZ0mMsetftga1AxlORoiCH7xVITli56f3lGk+wRHa77k3PkF28w1cVNc/hEKBdowrbPjW+Pc0Zml/Xx8nttuHMBWN7x/YYXpTVPhg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2023/11/25 20:10, Borislav Petkov wrote: Hi, Borislav, Thank you for your reply, and sorry for the confusion I made. Please see my rely inline. Best Regards, Shuai > On Sat, Nov 25, 2023 at 02:44:52PM +0800, Shuai Xue wrote: >> - an AR error consumed by current process is deferred to handle in a >> dedicated kernel thread, but memory_failure() assumes that it runs in the >> current context > > On x86? ARM? > > Pease point to the exact code flow. An AR error consumed by current process is deferred to handle in a dedicated kernel thread on ARM platform. The AR error is handled in bellow flow: ----------------------------------------------------------------------------- [usr space task einj_mem_uc consumd data poison, CPU 3] STEP 0 ----------------------------------------------------------------------------- [ghes_sdei_critical_callback: current einj_mem_uc, CPU 3] STEP 1 ghes_sdei_critical_callback => __ghes_sdei_callback => ghes_in_nmi_queue_one_entry // peak and read estatus => irq_work_queue(&ghes_proc_irq_work) <=> ghes_proc_in_irq // irq_work [ghes_sdei_critical_callback: return] ----------------------------------------------------------------------------- [ghes_proc_in_irq: current einj_mem_uc, CPU 3] STEP 2 => ghes_do_proc => ghes_handle_memory_failure => ghes_do_memory_failure => memory_failure_queue // put work task on current CPU => if (kfifo_put(&mf_cpu->fifo, entry)) schedule_work_on(smp_processor_id(), &mf_cpu->work); => task_work_add(current, &estatus_node->task_work, TWA_RESUME); [ghes_proc_in_irq: return] ----------------------------------------------------------------------------- // kworker preempts einj_mem_uc on CPU 3 due to RESCHED flag STEP 3 [memory_failure_work_func: current kworker, CPU 3] => memory_failure_work_func(&mf_cpu->work) => while kfifo_get(&mf_cpu->fifo, &entry); // until get no work => memory_failure(entry.pfn, entry.flags); ----------------------------------------------------------------------------- [ghes_kick_task_work: current einj_mem_uc, other cpu] STEP 4 => memory_failure_queue_kick => cancel_work_sync - waiting memory_failure_work_func finish => memory_failure_work_func(&mf_cpu->work) => kfifo_get(&mf_cpu->fifo, &entry); // no work ----------------------------------------------------------------------------- [einj_mem_uc resume at the same PC, trigger a page fault STEP 5 STEP 0: A user space task, named einj_mem_uc consume a poison. The firmware notifies hardware error to kernel through is SDEI (ACPI_HEST_NOTIFY_SOFTWARE_DELEGATED). STEP 1: The swapper running on CPU 3 is interrupted. irq_work_queue() rasie a irq_work to handle hardware errors in IRQ context STEP2: In IRQ context, ghes_proc_in_irq() queues memory failure work on current CPU in workqueue and add task work to sync with the workqueue. STEP3: The kworker preempts the current running thread and get CPU 3. Then memory_failure() is processed in kworker. STEP4: ghes_kick_task_work() is called as task_work to ensure any queued workqueue has been done before returning to user-space. STEP5: Upon returning to user-space, the task einj_mem_uc resumes at the current instruction, because the poison page is unmapped by memory_failure() in step 3, so a page fault will be triggered. memory_failure() assumes that it runs in the current context on both x86 and ARM platform. for example: memory_failure() in mm/memory-failure.c: if (flags & MF_ACTION_REQUIRED) { folio = page_folio(p); res = kill_accessing_process(current, folio_pfn(folio), flags); } > >> - another page fault is not unnecessary, we can send sigbus to current >> process in the first Synchronous External Abort SEA on arm64 (analogy >> Machine Check Exception on x86) > > I have no clue what that means. What page fault? I mean page fault in step 5. We can simplify the above flow by queuing memory_failure() as a task work for AR errors in step 3 directly. > >> I just give an example that the user space process *really* relys on the >> si_code of signal to handle hardware errors > > No, don't give examples. > > Explain what the exact problem is you're seeing, in your use case, point > to the code and then state how you think it should be fixed and why. > > Right now your text is "all over the place" and I have no clue what you > even want. Ok, got it. Thank you. > >> The SIGBUS si_codes defined in include/uapi/asm-generic/siginfo.h says: >> >> /* hardware memory error consumed on a machine check: action required */ >> #define BUS_MCEERR_AR 4 >> /* hardware memory error detected in process but not consumed: action optional*/ >> #define BUS_MCEERR_AO 5 >> >> When a synchronous error is consumed by Guest, the kernel should send a >> signal with BUS_MCEERR_AR instead of BUS_MCEERR_AO. > > Can you drop this "synchronous" bla and concentrate on the error > *severity*? > > I think you want to say that there are some types of errors for which > error handling needs to happen immediately and for some reason that > doesn't happen. > > Which errors are those? Types? > > Why do you need them to be handled immediately? Well, the severity defined on x86 and ARM platform is quite different. I guess you mean taxonomy of producer error types. - X86: Software recoverable action required (SRAR) A UCR error that *requires* system software to take a recovery action on this processor *before scheduling another stream of execution on this processor*. (15.6.3 UCR Error Classification in Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3) - ARM: Recoverable state (UER) The PE determines that software *must* take action to locate and repair the error to successfully recover execution. This might be because the exception was taken before the error was architecturally consumed by the PE, at the point when the PE was not be able to make correct progress without either consuming the error or *otherwise making the state of the PE unrecoverable*. (2.3.2 PE error state classification in Arm RAS Supplement https://documentation-service.arm.com/static/63185614f72fad1903828eda) I think above two types of error need to be handled immediately. > >> Exactly. > > No, not exactly. Why is it ok to do that? What are the implications of > this? > > Is immediate killing the right decision? > > Is this ok for *every* possible kernel running out there - not only for > your use case? > > And so on and so on... > I don't have a clear answer here. I guess the poison data only effects the user space task which triggers exception. A panic is not necessary. On x86 platform, the current error handling of memory_failure() in kill_me_maybe() is just send a sigbus forcely. kill_me_maybe(): ret = memory_failure(pfn, flags); if (ret == -EHWPOISON || ret == -EOPNOTSUPP) return; pr_err("Memory error not recovered"); kill_me_now(cb); Do you have any comments or suggestion about this? I don't change x86 behavior. For arm64 platform, step 3 in above flow, memory_failure_work_func(), the call site of memory_failure(), does not handle the return code of memory_failure(). I just add the same behavior.