From: Xiaofei Tan <tanxiaofei@huawei.com>
To: Shuai Xue <xueshuai@linux.alibaba.com>, <mawupeng1@huawei.com>,
<tony.luck@intel.com>, <naoya.horiguchi@nec.com>
Cc: <linux-acpi@vger.kernel.org>, <linux-mm@kvack.org>,
<linux-kernel@vger.kernel.org>, <justin.he@arm.com>,
<akpm@linux-foundation.org>, <ardb@kernel.org>,
<ashish.kalra@amd.com>, <baolin.wang@linux.alibaba.com>,
<bp@alien8.de>, <cuibixuan@linux.alibaba.com>,
<dave.hansen@linux.intel.com>, <james.morse@arm.com>,
<jarkko@kernel.org>, <lenb@kernel.org>, <linmiaohe@huawei.com>,
<lvying6@huawei.com>, <rafael@kernel.org>, <xiexiuqi@huawei.com>,
<zhuo.song@linux.alibaba.com>
Subject: Re: [PATCH v4 2/2] ACPI: APEI: handle synchronous exceptions in task work
Date: Tue, 11 Apr 2023 17:02:54 +0800 [thread overview]
Message-ID: <10a10bc8-bf08-43b4-7f31-f017a9144534@huawei.com> (raw)
In-Reply-To: <34492b33-ce55-6aeb-4cf2-3cae7bd5faba@linux.alibaba.com>
在 2023/4/11 11:16, Shuai Xue 写道:
>
> On 2023/4/11 AM9:44, Xiaofei Tan wrote:
>> Hi Shuai,
>>
>> 在 2023/4/8 17:13, Shuai Xue 写道:
>>> Hardware errors could be signaled by synchronous interrupt, e.g. when an
>>> error is detected by a background scrubber, or signaled by synchronous
>>> exception, e.g. when an uncorrected error is consumed. Both synchronous and
>>> asynchronous error are queued and handled by a dedicated kthread in
>>> workqueue.
>>>
>>> commit 7f17b4a121d0 ("ACPI: APEI: Kick the memory_failure() queue for
>>> synchronous errors") keep track of whether memory_failure() work was
>>> queued, and make task_work pending to flush out the workqueue so that the
>>> work for synchronous error is processed before returning to user-space.
>>> The trick ensures that the corrupted page is unmapped and poisoned. And
>>> after returning to user-space, the task starts at current instruction which
>>> triggering a page fault in which kernel will send SIGBUS to current process
>>> due to VM_FAULT_HWPOISON.
>>>
>>> However, the memory failure recovery for hwpoison-aware mechanisms does not
>>> work as expected. For example, hwpoison-aware user-space processes like
>>> QEMU register their customized SIGBUS handler and enable early kill mode by
>>> seting PF_MCE_EARLY at initialization. Then the kernel will directy notify
>>> the process by sending a SIGBUS signal in memory failure with wrong
>>> si_code: the actual user-space process accessing the corrupt memory
>>> location, but its memory failure work is handled in a kthread context, so
>>> it will send SIGBUS with BUS_MCEERR_AO si_code to the actual user-space
>>> process instead of BUS_MCEERR_AR in kill_proc().
>>>
>>> To this end, separate synchronous and asynchronous error handling into
>>> different paths like X86 platform does:
>>>
>>> - valid synchronous errors: queue a task_work to synchronously send SIGBUS
>>> before ret_to_user.
>>> - valid asynchronous errors: queue a work into workqueue to asynchronously
>>> handle memory failure.
>>> - abnormal branches such as invalid PA, unexpected severity, no memory
>>> failure config support, invalid GUID section, OOM, etc.
>>>
>>> Then for valid synchronous errors, the current context in memory failure is
>>> exactly belongs to the task consuming poison data and it will send SIBBUS
>>> with proper si_code.
>>>
>>> Fixes: 7f17b4a121d0 ("ACPI: APEI: Kick the memory_failure() queue for synchronous errors")
>>> Signed-off-by: Shuai Xue <xueshuai@linux.alibaba.com>
>>> Tested-by: Ma Wupeng <mawupeng1@huawei.com>
>>> ---
>>> drivers/acpi/apei/ghes.c | 91 +++++++++++++++++++++++++++-------------
>>> include/acpi/ghes.h | 3 --
>>> mm/memory-failure.c | 13 ------
>>> 3 files changed, 61 insertions(+), 46 deletions(-)
>>>
>>> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
>>> index c479b85899f5..df5574264d1b 100644
>>> --- a/drivers/acpi/apei/ghes.c
>>> +++ b/drivers/acpi/apei/ghes.c
>>> @@ -452,28 +452,51 @@ static void ghes_clear_estatus(struct ghes *ghes,
>>> }
>>> /*
>>> - * Called as task_work before returning to user-space.
>>> - * Ensure any queued work has been done before we return to the context that
>>> - * triggered the notification.
>>> + * struct sync_task_work - for synchronous RAS event
>>> + *
>>> + * @twork: callback_head for task work
>>> + * @pfn: page frame number of corrupted page
>>> + * @flags: fine tune action taken
>>> + *
>>> + * Structure to pass task work to be handled before
>>> + * ret_to_user via task_work_add().
>>> */
>>> -static void ghes_kick_task_work(struct callback_head *head)
>>> +struct sync_task_work {
>>> + struct callback_head twork;
>>> + u64 pfn;
>>> + int flags;
>>> +};
>>> +
>>> +static void memory_failure_cb(struct callback_head *twork)
>>> {
>>> - struct acpi_hest_generic_status *estatus;
>>> - struct ghes_estatus_node *estatus_node;
>>> - u32 node_len;
>>> + int ret;
>>> + struct sync_task_work *twcb =
>>> + container_of(twork, struct sync_task_work, twork);
>>> - estatus_node = container_of(head, struct ghes_estatus_node, task_work);
>>> - if (IS_ENABLED(CONFIG_ACPI_APEI_MEMORY_FAILURE))
>>> - memory_failure_queue_kick(estatus_node->task_work_cpu);
>>> + ret = memory_failure(twcb->pfn, twcb->flags);
>>> + kfree(twcb);
>>> - estatus = GHES_ESTATUS_FROM_NODE(estatus_node);
>>> - node_len = GHES_ESTATUS_NODE_LEN(cper_estatus_len(estatus));
>>> - gen_pool_free(ghes_estatus_pool, (unsigned long)estatus_node, node_len);
>>> + if (!ret)
>>> + return;
>>> +
>>> + /*
>>> + * -EHWPOISON from memory_failure() means that it already sent SIGBUS
>>> + * to the current process with the proper error info,
>>> + * -EOPNOTSUPP means hwpoison_filter() filtered the error event,
>>> + *
>>> + * In both cases, no further processing is required.
>>> + */
>>> + if (ret == -EHWPOISON || ret == -EOPNOTSUPP)
>>> + return;
>>> +
>>> + pr_err("Memory error not recovered");
>>> + force_sig(SIGBUS);
>>> }
>>> static bool ghes_do_memory_failure(u64 physical_addr, int flags)
>>> {
>>> unsigned long pfn;
>>> + struct sync_task_work *twcb;
>>> if (!IS_ENABLED(CONFIG_ACPI_APEI_MEMORY_FAILURE))
>>> return false;
>>> @@ -486,6 +509,18 @@ static bool ghes_do_memory_failure(u64 physical_addr, int flags)
>>> return false;
>>> }
>>> + if (flags == MF_ACTION_REQUIRED && current->mm) {
>>> + twcb = kmalloc(sizeof(*twcb), GFP_ATOMIC);
>>> + if (!twcb)
>>> + return false;
>>> +
>>> + twcb->pfn = pfn;
>>> + twcb->flags = flags;
>>> + init_task_work(&twcb->twork, memory_failure_cb);
>>> + task_work_add(current, &twcb->twork, TWA_RESUME);
>>> + return true;
>>> + }
>>> +
>>> memory_failure_queue(pfn, flags);
>>> return true;
>>> }
>>> @@ -1000,9 +1035,8 @@ static void ghes_proc_in_irq(struct irq_work *irq_work)
>>> struct ghes_estatus_node *estatus_node;
>>> struct acpi_hest_generic *generic;
>>> struct acpi_hest_generic_status *estatus;
>>> - bool task_work_pending;
>>> + bool queued;
>>> u32 len, node_len;
>>> - int ret;
>>> llnode = llist_del_all(&ghes_estatus_llist);
>>> /*
>>> @@ -1017,25 +1051,23 @@ static void ghes_proc_in_irq(struct irq_work *irq_work)
>>> estatus = GHES_ESTATUS_FROM_NODE(estatus_node);
>>> len = cper_estatus_len(estatus);
>>> node_len = GHES_ESTATUS_NODE_LEN(len);
>>> - task_work_pending = ghes_do_proc(estatus_node->ghes, estatus);
>>> +
>>> + queued = ghes_do_proc(estatus_node->ghes, estatus);
>>> + /*
>>> + * No memory failure work is queued into work queue or task queue
>>> + * due to invalid PA, unexpected severity, OOM, etc, do a force
>>> + * kill.
>>> + */
>>> + if (!queued && current->mm)
>>> + force_sig(SIGBUS);
>> The SIGBUS needs to be sent to the current only for synchronous exceptions. The judgment of this if statement does not guarantee this.
>> Because the function ghes_proc_in_irq() is used for NMI, but NMI not only used for synchronous exception. One user SEA is synchronous
>> exception, and some other users, such as SDEI, may be not synchronous exception.
> Yes, you are right. I was going to handle abnormal cases for sync error
> and async error. But SIGBUS sent to the current task for an asynchronous
> error is totally wrong.
yes
> Is it safe to keep running when an asynchronous
> error is not handled?
I think so. Corrupt data should not be consumed silently. It should be guaranteed by Chip platorm.
If platform can't support this, it will still not be 100% safe even we panic the system here, once received
uncorrected memory error section.
>
> And should we add some warning message in abnormal cases?
> e.g pr_warn_ratelimited on invalid PA?
Do you mean here ? it is not needed, as ghes_print_estatus() has included this info.
if (!(mem_err->validation_bits & CPER_MEM_VALID_PA))
return false;
>> You could transfer the sync flag out from ghes_do_proc() and judge it here, or change meaning of the ghes_do_proc() return value
>> as if recovered.
> I think we could get sync flag by estatus_node, e.g:
>
> bool sync = is_hest_sync_notify(estatus_node->ghes);
It's ok for me.
>
> Then the condition in if statement should be:
>
> if (sync && !queued)
>
> I drop out current->mm from if statement. For sync errors, the current
> is guaranteed to be in user task, kernel task for sync error will panic
> in do_sea(), the caller of ghes_proc_in_irq(). For async errors, SIGBUS
> to current is meaningless.
OK. It is correct for ARM SEA, if want to support more sync notify type,
should consider in the future.
>
> Thank you.
>
> Best Regards,
> Shuai
>
>>
>>> +
>>> if (!ghes_estatus_cached(estatus)) {
>>> generic = estatus_node->generic;
>>> if (ghes_print_estatus(NULL, generic, estatus))
>>> ghes_estatus_cache_add(generic, estatus);
>>> }
>>> -
>>> - if (task_work_pending && current->mm) {
>>> - estatus_node->task_work.func = ghes_kick_task_work;
>>> - estatus_node->task_work_cpu = smp_processor_id();
>>> - ret = task_work_add(current, &estatus_node->task_work,
>>> - TWA_RESUME);
>>> - if (ret)
>>> - estatus_node->task_work.func = NULL;
>>> - }
>>> -
>>> - if (!estatus_node->task_work.func)
>>> - gen_pool_free(ghes_estatus_pool,
>>> - (unsigned long)estatus_node, node_len);
>>> + gen_pool_free(ghes_estatus_pool, (unsigned long)estatus_node,
>>> + node_len);
>>> llnode = next;
>>> }
>>> @@ -1096,7 +1128,6 @@ static int ghes_in_nmi_queue_one_entry(struct ghes *ghes,
>>> estatus_node->ghes = ghes;
>>> estatus_node->generic = ghes->generic;
>>> - estatus_node->task_work.func = NULL;
>>> estatus = GHES_ESTATUS_FROM_NODE(estatus_node);
>>> if (__ghes_read_estatus(estatus, buf_paddr, fixmap_idx, len)) {
>>> diff --git a/include/acpi/ghes.h b/include/acpi/ghes.h
>>> index 3c8bba9f1114..e5e0c308d27f 100644
>>> --- a/include/acpi/ghes.h
>>> +++ b/include/acpi/ghes.h
>>> @@ -35,9 +35,6 @@ struct ghes_estatus_node {
>>> struct llist_node llnode;
>>> struct acpi_hest_generic *generic;
>>> struct ghes *ghes;
>>> -
>>> - int task_work_cpu;
>>> - struct callback_head task_work;
>>> };
>>> struct ghes_estatus_cache {
>>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>>> index fae9baf3be16..6ea8c325acb3 100644
>>> --- a/mm/memory-failure.c
>>> +++ b/mm/memory-failure.c
>>> @@ -2355,19 +2355,6 @@ static void memory_failure_work_func(struct work_struct *work)
>>> }
>>> }
>>> -/*
>>> - * Process memory_failure work queued on the specified CPU.
>>> - * Used to avoid return-to-userspace racing with the memory_failure workqueue.
>>> - */
>>> -void memory_failure_queue_kick(int cpu)
>>> -{
>>> - struct memory_failure_cpu *mf_cpu;
>>> -
>>> - mf_cpu = &per_cpu(memory_failure_cpu, cpu);
>>> - cancel_work_sync(&mf_cpu->work);
>>> - memory_failure_work_func(&mf_cpu->work);
>>> -}
>>> -
>>> static int __init memory_failure_init(void)
>>> {
>>> struct memory_failure_cpu *mf_cpu;
> .
next prev parent reply other threads:[~2023-04-11 9:03 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20221027042445.60108-1-xueshuai@linux.alibaba.com>
2023-03-17 7:24 ` [PATCH v3 0/2] ACPI: APEI: handle synchronous exceptions with proper si_code Shuai Xue
2023-03-20 18:03 ` Rafael J. Wysocki
2023-03-30 6:11 ` Shuai Xue
2023-03-30 9:52 ` Rafael J. Wysocki
2023-03-21 7:17 ` mawupeng
2023-03-22 1:27 ` Shuai Xue
2023-03-17 7:24 ` [PATCH v3 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-03-17 7:24 ` [PATCH v3 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-04-06 12:39 ` Xiaofei Tan
2023-04-07 2:21 ` Shuai Xue
2023-04-08 9:13 ` [PATCH v4 0/2] ACPI: APEI: handle synchronous exceptions with proper si_code Shuai Xue
2023-04-08 9:13 ` [PATCH v4 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-04-08 9:13 ` [PATCH v4 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-04-11 1:44 ` Xiaofei Tan
2023-04-11 3:16 ` Shuai Xue
2023-04-11 9:02 ` Xiaofei Tan [this message]
2023-04-11 9:48 ` Shuai Xue
2023-04-11 10:48 ` [PATCH v5 0/2] ACPI: APEI: handle synchronous exceptions with proper si_code Shuai Xue
2023-04-11 10:48 ` [PATCH v5 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-04-11 14:17 ` Kefeng Wang
2023-04-12 2:54 ` Shuai Xue
2023-04-12 3:55 ` Xiaofei Tan
2023-04-13 1:42 ` Shuai Xue
2023-04-11 10:48 ` [PATCH v5 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-04-11 14:28 ` Kefeng Wang
2023-04-12 2:58 ` Shuai Xue
2023-04-12 4:05 ` Xiaofei Tan
2023-04-13 1:49 ` Shuai Xue
2023-04-12 11:27 ` [PATCH v6 0/2] ACPI: APEI: handle synchronous exceptions with proper si_code Shuai Xue
2023-04-12 11:28 ` [PATCH v6 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-04-12 11:28 ` [PATCH v6 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-04-17 1:14 ` [PATCH v7 0/2] ACPI: APEI: handle synchronous exceptions with proper si_code Shuai Xue
2023-04-24 6:24 ` Shuai Xue
2023-05-08 1:55 ` Shuai Xue
2023-04-17 1:14 ` [PATCH v7 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-04-17 1:14 ` [PATCH v7 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-09-19 2:21 ` [RESEND PATCH v8 0/2] ACPI: APEI: handle synchronous errors in task work with proper si_code Shuai Xue
2023-09-19 2:21 ` [RESEND PATCH v8 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-09-25 14:43 ` Jarkko Sakkinen
2023-09-26 6:23 ` Shuai Xue
2023-09-19 2:21 ` [RESEND PATCH v8 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-09-25 15:00 ` Jarkko Sakkinen
2023-09-26 6:38 ` Shuai Xue
2023-10-03 8:28 ` Naoya Horiguchi
2023-10-07 2:01 ` Shuai Xue
2023-10-07 7:28 ` [PATCH v9 0/2] ACPI: APEI: handle synchronous errors in task work with proper si_code Shuai Xue
2023-11-21 1:48 ` Shuai Xue
2023-11-23 15:07 ` Borislav Petkov
2023-11-25 6:44 ` Shuai Xue
2023-11-25 12:10 ` Borislav Petkov
2023-11-26 12:25 ` Shuai Xue
2023-11-29 18:54 ` Borislav Petkov
2023-11-30 2:58 ` Shuai Xue
2023-11-30 14:40 ` Borislav Petkov
2023-11-30 17:43 ` James Morse
2023-12-01 2:58 ` Shuai Xue
2023-11-30 17:39 ` James Morse
2023-12-01 3:37 ` Shuai Xue
2023-10-07 7:28 ` [PATCH v9 1/2] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-11-30 17:39 ` James Morse
2023-12-01 5:22 ` Shuai Xue
2023-10-07 7:28 ` [PATCH v9 2/2] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-11-30 17:39 ` James Morse
2023-12-01 7:03 ` Shuai Xue
2023-12-18 6:45 ` [PATCH v10 0/4] ACPI: APEI: handle synchronous errors in task work with proper si_code Shuai Xue
2023-12-18 6:45 ` [PATCH v10 1/4] ACPI: APEI: set memory failure flags as MF_ACTION_REQUIRED on synchronous events Shuai Xue
2023-12-18 6:53 ` Greg KH
2023-12-21 13:55 ` Rafael J. Wysocki
2023-12-22 1:07 ` Shuai Xue
2023-12-18 6:45 ` [PATCH v10 2/4] ACPI: APEI: send SIGBUS to current task if synchronous memory error not recovered Shuai Xue
2023-12-18 6:54 ` Greg KH
2023-12-18 6:45 ` [PATCH v10 3/4] mm: memory-failure: move memory_failure() return value documentation to function declaration Shuai Xue
2023-12-18 6:54 ` Greg KH
2023-12-18 6:45 ` [PATCH v10 4/4] ACPI: APEI: handle synchronous exceptions in task work Shuai Xue
2023-12-18 6:54 ` Greg KH
2024-02-04 8:01 ` [PATCH v11 0/3] ACPI: APEI: handle synchronous exceptions in task work to send correct SIGBUS si_code Shuai Xue
2024-02-19 1:46 ` Shuai Xue
2024-02-04 8:01 ` [PATCH v11 1/3] ACPI: APEI: send SIGBUS to current task if synchronous memory error not recovered Shuai Xue
2024-02-19 9:25 ` Borislav Petkov
2024-02-22 2:07 ` Shuai Xue
2024-02-23 5:26 ` Dan Williams
2024-02-23 12:08 ` Jonathan Cameron
2024-02-23 12:17 ` Jonathan Cameron
2024-02-24 6:08 ` Shuai Xue
2024-02-26 10:29 ` Borislav Petkov
2024-02-27 1:23 ` Shuai Xue
2024-02-24 19:42 ` Dan Williams
2024-02-24 19:40 ` Dan Williams
2024-02-04 8:01 ` [PATCH v11 2/3] mm: memory-failure: move return value documentation to function declaration Shuai Xue
2024-02-26 10:46 ` Borislav Petkov
2024-02-27 1:27 ` Shuai Xue
2024-02-04 8:01 ` [PATCH v11 3/3] ACPI: APEI: handle synchronous exceptions in task work to send correct SIGBUS si_code Shuai Xue
2024-02-29 7:05 ` Shuai Xue
2024-03-08 10:18 ` Borislav Petkov
2024-03-12 6:05 ` Shuai Xue
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=10a10bc8-bf08-43b4-7f31-f017a9144534@huawei.com \
--to=tanxiaofei@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=ardb@kernel.org \
--cc=ashish.kalra@amd.com \
--cc=baolin.wang@linux.alibaba.com \
--cc=bp@alien8.de \
--cc=cuibixuan@linux.alibaba.com \
--cc=dave.hansen@linux.intel.com \
--cc=james.morse@arm.com \
--cc=jarkko@kernel.org \
--cc=justin.he@arm.com \
--cc=lenb@kernel.org \
--cc=linmiaohe@huawei.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lvying6@huawei.com \
--cc=mawupeng1@huawei.com \
--cc=naoya.horiguchi@nec.com \
--cc=rafael@kernel.org \
--cc=tony.luck@intel.com \
--cc=xiexiuqi@huawei.com \
--cc=xueshuai@linux.alibaba.com \
--cc=zhuo.song@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).