From: <shiju.jose@huawei.com>
To: <mchehab@kernel.org>, <linux-edac@vger.kernel.org>
Cc: <rostedt@goodmis.org>, <mhiramat@kernel.org>,
<linux-kernel@vger.kernel.org>,
<linux-trace-kernel@vger.kernel.org>, <tanxiaofei@huawei.com>,
<jonathan.cameron@huawei.com>, <linuxarm@huawei.com>,
<shiju.jose@huawei.com>
Subject: [RFC PATCH 1/1] rasdaemon: Fix poll() on per_cpu trace_pipe_raw do not retun
Date: Thu, 2 Feb 2023 18:23:51 +0000 [thread overview]
Message-ID: <20230202182352.792-1-shiju.jose@huawei.com> (raw)
From: Shiju Jose <shiju.jose@huawei.com>
The error events are not received in the rasdaemon since kernel 6.1-rc6.
This issue is firstly detected and reported, when testing the CXL error
events in the rasdaemon.
Debugging showed, poll() on trace_pipe_raw in the ras-events.c do not return.
and this issue is seen after the commit
42fb0a1e84ff525ebe560e2baf9451ab69127e2b ("tracing/ring-buffer: Have
polling block on watermark").
This also verified using the test application for poll()
and select() on trace_pipe_raw.
There is also a bug reported on this issue,
https://lore.kernel.org/all/31eb3b12-3350-90a4-a0d9-d1494db7cf74@oracle.com/
This issue occurs for the per_cpu case, when calling the ring_buffer_poll_wait(),
in kernel/trace/ring_buffer.c, with the buffer_percent > 0 and then wait until the
percentage of pages are available. The default value set for the buffer_percent is 50
in the kernel/trace/trace.c.
As a fix, rasdaemon set buffer_percent = 0 through the
/sys/kernel/debug/tracing/instances/rasdaemon/buffer_percent, then the task will
wake up as soon as data is added to any of the specific cpu buffer and
poll() on per_cpu/cpuX/trace_pipe_raw does not block indefinitely.
Depends on kernel patch
tracing: Fix poll() and select() do not work on per_cpu trace_pipe and trace_pipe_raw
Signed-off-by: Shiju Jose <shiju.jose@huawei.com>
---
ras-events.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/ras-events.c b/ras-events.c
index 3691311..72225a9 100644
--- a/ras-events.c
+++ b/ras-events.c
@@ -383,6 +383,8 @@ static int read_ras_event_all_cpus(struct pthread_data *pdata,
int warnonce[n_cpus];
char pipe_raw[PATH_MAX];
int legacy_kernel = 0;
+ int fd;
+ char buf[2];
#if 0
int need_sleep = 0;
#endif
@@ -402,6 +404,33 @@ static int read_ras_event_all_cpus(struct pthread_data *pdata,
return -ENOMEM;
}
+ /* Fix for poll() do not return on per_cpu trace_pipe and
+ * trace_pipe_raw with the default buffer_percent in the kernel trace,
+ * introduced by the following change in the kernel
+ * https://lore.kernel.org/all/20221020231427.41be3f26@gandalf.local.home/T/#u.
+ * Set buffer_percent to 0 so that poll() returned immediately
+ * when the trace data is available in the ras per_cpu trace pipe_raw */
+ fd = open_trace(pdata[0].ras, "buffer_percent", O_RDWR);
+ if (fd < 0) {
+ log(TERM, LOG_ERR, "Can't open trace buffer_percent\n");
+ return -1;
+ }
+ size = write(fd, "0", 1);
+ if (size <= 0) {
+ log(TERM, LOG_ERR, "Can't set rasdaemon trace buffer_percent\n");
+ return -1;
+ }
+ size = read(fd, buf, 1);
+ if (size <= 0) {
+ log(TERM, LOG_ERR, "Can't read rasdaemon trace buffer_percent\n");
+ return -1;
+ }
+ if (atoi(buf) != 0) {
+ log(TERM, LOG_ERR, "Can't verify rasdaemon trace buffer_percent\n");
+ return -1;
+ }
+ close(fd);
+
for (i = 0; i < (n_cpus + 1); i++)
fds[i].fd = -1;
--
2.25.1
reply other threads:[~2023-02-02 18:24 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230202182352.792-1-shiju.jose@huawei.com \
--to=shiju.jose@huawei.com \
--cc=jonathan.cameron@huawei.com \
--cc=linux-edac@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=mchehab@kernel.org \
--cc=mhiramat@kernel.org \
--cc=rostedt@goodmis.org \
--cc=tanxiaofei@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).