linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Calvin Owens <calvinowens@fb.com>
To: Petr Mladek <pmladek@suse.com>,
	Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	Steven Rostedt <rostedt@goodmis.org>
Cc: "Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	"Jiri Slaby" <jslaby@suse.cz>,
	"Andrew Morton" <akpm@linux-foundation.org>,
	"Calvin Owens" <calvinowens@fb.com>,
	"Manuel Schölling" <manuel.schoelling@gmx.de>,
	"Hans de Goede" <hdegoede@redhat.com>,
	"Paul Burton" <paul.burton@imgtec.com>,
	linux-kernel@vger.kernel.org, kernel-team@fb.com
Subject: [RFC][PATCH 1/2] printk: Introduce per-console filtering of messages by loglevel
Date: Tue, 4 Apr 2017 16:02:48 -0700	[thread overview]
Message-ID: <bc1fe800930be76761b2bb4e6108fe5876a005c9.1491345440.git.calvinowens@fb.com> (raw)

Not all consoles are created equal: depending on the actual hardware,
the latency of a printk() call can vary dramatically. The worst examples
are serial consoles, where it can spin for tens of milliseconds banging
the UART to emit a message, which can cause application-level problems
when the kernel spews onto the console.

At Facebook we use netconsole to monitor our fleet, but we still have
serial consoles attached on each host for live debugging, and the latter
has caused problems. An obvious solution is to disable the kernel
console output to ttyS0, but this makes live debugging frustrating,
since crashes become silent and opaque to the ttyS0 user. Enabling it on
the fly when needed isn't feasible, since boxes you need to debug via
serial are likely to be borked in ways that make this impossible.

This puts us between a rock and a hard place: we'd love to set
kernel.printk to KERN_INFO and get all the logs. But while netconsole is
fast enough to permit that without perturbing userspace, ttyS0 is not,
and we're forced to limit console logging to KERN_WARNING and higher.

This patch lets us have our cake and eat it too: instead of being forced
to limit all consoles verbosity based on the speed of the slowest one,
we can limit each based on its own speed. A subsequent patch will
introduce a simple sysfs interface for changing this setting.

Signed-off-by: Calvin Owens <calvinowens@fb.com>
---
 include/linux/console.h |  1 +
 kernel/printk/printk.c  | 13 ++++++++++---
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/include/linux/console.h b/include/linux/console.h
index 5949d18..764a2c0 100644
--- a/include/linux/console.h
+++ b/include/linux/console.h
@@ -147,6 +147,7 @@ struct console {
 	int	cflag;
 	void	*data;
 	struct	 console *next;
+	int	maxlevel;
 };
 
 /*
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index 2984fb0..5393928 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1562,7 +1562,7 @@ SYSCALL_DEFINE3(syslog, int, type, char __user *, buf, int, len)
  * The console_lock must be held.
  */
 static void call_console_drivers(const char *ext_text, size_t ext_len,
-				 const char *text, size_t len)
+				 const char *text, size_t len, int level)
 {
 	struct console *con;
 
@@ -1581,6 +1581,8 @@ static void call_console_drivers(const char *ext_text, size_t ext_len,
 		if (!cpu_online(smp_processor_id()) &&
 		    !(con->flags & CON_ANYTIME))
 			continue;
+		if (level > con->maxlevel)
+			continue;
 		if (con->flags & CON_EXTENDED)
 			con->write(con, ext_text, ext_len);
 		else
@@ -1869,7 +1871,7 @@ static ssize_t msg_print_ext_body(char *buf, size_t size,
 				  char *dict, size_t dict_len,
 				  char *text, size_t text_len) { return 0; }
 static void call_console_drivers(const char *ext_text, size_t ext_len,
-				 const char *text, size_t len) {}
+				 const char *text, size_t len, int level) {}
 static size_t msg_print_text(const struct printk_log *msg,
 			     bool syslog, char *buf, size_t size) { return 0; }
 static bool suppress_message_printing(int level) { return false; }
@@ -2238,7 +2240,7 @@ void console_unlock(void)
 		raw_spin_unlock(&logbuf_lock);
 
 		stop_critical_timings();	/* don't trace print latency */
-		call_console_drivers(ext_text, ext_len, text, len);
+		call_console_drivers(ext_text, ext_len, text, len, msg->level);
 		start_critical_timings();
 		printk_safe_exit_irqrestore(flags);
 
@@ -2504,6 +2506,11 @@ void register_console(struct console *newcon)
 		newcon->flags &= ~CON_PRINTBUFFER;
 
 	/*
+	 * By default, the per-console loglevel filter permits all messages.
+	 */
+	newcon->maxlevel = LOGLEVEL_DEBUG;
+
+	/*
 	 *	Put this console in the list - keep the
 	 *	preferred driver at the head of the list.
 	 */
-- 
2.9.3

             reply	other threads:[~2017-04-04 23:03 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-04 23:02 Calvin Owens [this message]
2017-04-04 23:03 ` [RFC][PATCH 2/2] printk: Add /sys/consoles/${con}/ and maxlevel attribute Calvin Owens
2017-04-05  3:30   ` Steven Rostedt
2017-04-05 23:55     ` Calvin Owens
2017-04-05  1:51 ` [RFC][PATCH 1/2] printk: Introduce per-console filtering of messages by loglevel Joe Perches
2017-04-05  2:08 ` Sergey Senozhatsky
2017-04-05  2:16   ` Sergey Senozhatsky
2017-04-05  3:27     ` Steven Rostedt
2017-04-05 23:53       ` Calvin Owens
2017-04-05 15:22     ` Petr Mladek
2017-04-06  0:38       ` Calvin Owens
2017-04-06 14:02         ` Petr Mladek
2017-04-06 22:52           ` Calvin Owens

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bc1fe800930be76761b2bb4e6108fe5876a005c9.1491345440.git.calvinowens@fb.com \
    --to=calvinowens@fb.com \
    --cc=akpm@linux-foundation.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hdegoede@redhat.com \
    --cc=jslaby@suse.cz \
    --cc=kernel-team@fb.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=manuel.schoelling@gmx.de \
    --cc=paul.burton@imgtec.com \
    --cc=pmladek@suse.com \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).