From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Ingo Molnar <mingo@kernel.org>, Thomas Gleixner <tglx@linutronix.de>
Cc: Jiri Olsa <jolsa@kernel.org>, Namhyung Kim <namhyung@kernel.org>,
Clark Williams <williams@redhat.com>,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Jiri Olsa <jolsa@redhat.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Song Liu <songliubraving@fb.com>, Wang Nan <wangnan0@huawei.com>
Subject: [PATCH 30/91] perf record: Introduce --switch-output-event
Date: Wed, 6 May 2020 12:21:33 -0300 [thread overview]
Message-ID: <20200506152234.21977-31-acme@kernel.org> (raw)
In-Reply-To: <20200506152234.21977-1-acme@kernel.org>
From: Arnaldo Carvalho de Melo <acme@redhat.com>
Now we can use it with --overwrite to have a flight recorder mode that
gets snapshot requests from arbitrary events that are processed in the
side band thread together with the PERF_RECORD_BPF_EVENT processing.
Example:
To collect scheduler events until a recvmmsg syscall happens, system
wide:
[root@five a]# rm -f perf.data.2020042717*
[root@five a]# perf record --overwrite -e sched:*switch,syscalls:*recvmmsg --switch-output-event syscalls:sys_enter_recvmmsg
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2020042717585458 ]
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2020042717590235 ]
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2020042717590398 ]
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Dump perf.data.2020042717590511 ]
[ perf record: Captured and wrote 7.244 MB perf.data.<timestamp> ]
So in the above case we had 3 snapshots, the fourth was forced by
control+C:
[root@five a]# ls -la
total 20440
drwxr-xr-x. 2 root root 4096 Apr 27 17:59 .
dr-xr-x---. 12 root root 4096 Apr 27 17:46 ..
-rw-------. 1 root root 3936125 Apr 27 17:58 perf.data.2020042717585458
-rw-------. 1 root root 5074869 Apr 27 17:59 perf.data.2020042717590235
-rw-------. 1 root root 4291037 Apr 27 17:59 perf.data.2020042717590398
-rw-------. 1 root root 7617037 Apr 27 17:59 perf.data.2020042717590511
[root@five a]#
One can make this more precise by adding the switch output event to the
main -e events list, as since this is done asynchronously, a few events
after the signal event will appear in the snapshots, as can be seen
with:
[root@five a]# rm -f perf.data.20200427175*
[root@five a]# perf record --overwrite -e sched:*switch,syscalls:*recvmmsg --switch-output-event syscalls:sys_enter_recvmmsg
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2020042718024203 ]
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2020042718024301 ]
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2020042718024484 ]
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Dump perf.data.2020042718024562 ]
[ perf record: Captured and wrote 7.337 MB perf.data.<timestamp> ]
[root@five a]# perf script -i perf.data.2020042718024203 | tail -15
PacerThread 148586 [005] 122.830729: sched:sched_switch: prev_comm=PacerThread prev_pid=148586...
swapper 0 [000] 122.833588: sched:sched_switch: prev_comm=swapper/0 prev_pid=...
NetworkManager 1251 [000] 122.833619: syscalls:sys_enter_recvmmsg: fd: 0x0000001c, mmsg: 0x7ffe83054a1...
swapper 0 [002] 122.833624: sched:sched_switch: prev_comm=swapper/2 prev_pid=...
swapper 0 [003] 122.833624: sched:sched_switch: prev_comm=swapper/3 prev_pid=...
NetworkManager 1251 [000] 122.833626: syscalls:sys_exit_recvmmsg: 0x1
kworker/3:3-eve 158946 [003] 122.833628: sched:sched_switch: prev_comm=kworker/3:3 prev_pid=15894...
swapper 0 [004] 122.833641: sched:sched_switch: prev_comm=swapper/4 prev_pid=...
NetworkManager 1251 [000] 122.833642: sched:sched_switch: prev_comm=NetworkManage...
perf 228273 [002] 122.833645: sched:sched_switch: prev_comm=perf prev_pid=22827...
swapper 0 [011] 122.833646: sched:sched_switch: prev_comm=swapper/1...
swapper 0 [002] 122.833648: sched:sched_switch: prev_comm=swapper/...
kworker/0:2-eve 207387 [000] 122.833648: sched:sched_switch: prev_comm=kworker/0:2 prev_pid=20738...
kworker/2:3-eve 232038 [002] 122.833652: sched:sched_switch: prev_comm=kworker/2:3 prev_pid=23203...
perf 235825 [003] 122.833653: sched:sched_switch: prev_comm=perf prev_pid=23582...
[root@five a]#
Acked-by: Jiri Olsa <jolsa@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Song Liu <songliubraving@fb.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lore.kernel.org/lkml/20200429131106.27974-8-acme@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/Documentation/perf-record.txt | 13 ++++++++
tools/perf/builtin-record.c | 41 +++++++++++++++++++++---
2 files changed, 50 insertions(+), 4 deletions(-)
diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index 6e8b4649307c..561ef55743e2 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -556,6 +556,19 @@ overhead. You can still switch them on with:
--switch-output --no-no-buildid --no-no-buildid-cache
+--switch-output-event::
+Events that will cause the switch of the perf.data file, auto-selecting
+--switch-output=signal, the results are similar as internally the side band
+thread will also send a SIGUSR2 to the main one.
+
+Uses the same syntax as --event, it will just not be recorded, serving only to
+switch the perf.data file as soon as the --switch-output event is processed by
+a separate sideband thread.
+
+This sideband thread is also used to other purposes, like processing the
+PERF_RECORD_BPF_EVENT records as they happen, asking the kernel for extra BPF
+information, etc.
+
--switch-max-files=N::
When rotating perf.data with --switch-output, only keep N files.
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 5b6a1d23efe5..bb5b4d2fc32b 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -88,7 +88,9 @@ struct record {
struct evlist *evlist;
struct perf_session *session;
struct evlist *sb_evlist;
+ pthread_t thread_id;
int realtime_prio;
+ bool switch_output_event_set;
bool no_buildid;
bool no_buildid_set;
bool no_buildid_cache;
@@ -1437,6 +1439,13 @@ static int record__synthesize(struct record *rec, bool tail)
return err;
}
+static int record__process_signal_event(union perf_event *event __maybe_unused, void *data)
+{
+ struct record *rec = data;
+ pthread_kill(rec->thread_id, SIGUSR2);
+ return 0;
+}
+
static int __cmd_record(struct record *rec, int argc, const char **argv)
{
int err;
@@ -1581,12 +1590,24 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
goto out_child;
}
- if (!opts->no_bpf_event) {
- rec->sb_evlist = evlist__new();
+ if (rec->sb_evlist != NULL) {
+ /*
+ * We get here if --switch-output-event populated the
+ * sb_evlist, so associate a callback that will send a SIGUSR2
+ * to the main thread.
+ */
+ evlist__set_cb(rec->sb_evlist, record__process_signal_event, rec);
+ rec->thread_id = pthread_self();
+ }
+ if (!opts->no_bpf_event) {
if (rec->sb_evlist == NULL) {
- pr_err("Couldn't create side band evlist.\n.");
- goto out_child;
+ rec->sb_evlist = evlist__new();
+
+ if (rec->sb_evlist == NULL) {
+ pr_err("Couldn't create side band evlist.\n.");
+ goto out_child;
+ }
}
if (evlist__add_bpf_sb_event(rec->sb_evlist, &session->header.env)) {
@@ -2180,10 +2201,19 @@ static int switch_output_setup(struct record *rec)
};
unsigned long val;
+ /*
+ * If we're using --switch-output-events, then we imply its
+ * --switch-output=signal, as we'll send a SIGUSR2 from the side band
+ * thread to its parent.
+ */
+ if (rec->switch_output_event_set)
+ goto do_signal;
+
if (!s->set)
return 0;
if (!strcmp(s->str, "signal")) {
+do_signal:
s->signal = true;
pr_debug("switch-output with SIGUSR2 signal\n");
goto enabled;
@@ -2441,6 +2471,9 @@ static struct option __record_options[] = {
&record.switch_output.set, "signal or size[BKMG] or time[smhd]",
"Switch output when receiving SIGUSR2 (signal) or cross a size or time threshold",
"signal"),
+ OPT_CALLBACK_SET(0, "switch-output-event", &record.sb_evlist, &record.switch_output_event_set, "switch output event",
+ "switch output event selector. use 'perf list' to list available events",
+ parse_events_option_new_evlist),
OPT_INTEGER(0, "switch-max-files", &record.switch_output.num_files,
"Limit number of switch output generated files"),
OPT_BOOLEAN(0, "dry-run", &dry_run,
--
2.21.1
next prev parent reply other threads:[~2020-05-06 15:24 UTC|newest]
Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-06 15:21 [GIT PULL] perf/core improvements and fixes Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 01/91] perf cgroup: Avoid needless closing of unopened fd Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 02/91] perf bench: Fix div-by-zero if runtime is zero Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 03/91] perf evlist: Remove duplicate headers Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 04/91] perf script: Avoid NULL dereference on symbol Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 05/91] perf stat: Zero all the 'ena' and 'run' array slot stats for interval mode Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 06/91] perf stat: Improve runtime stat " Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 07/91] perf test session topology: Fix data path Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 08/91] perf record: Add num-synthesize-threads option Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 09/91] perf bench: Add a multi-threaded synthesize benchmark Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 10/91] tools api: Add a lightweight buffered reading api Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 11/91] perf synthetic events: Remove use of sscanf from /proc reading Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 12/91] perf script: Remove extraneous newline in perf_sample__fprintf_regs() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 13/91] libtraceevent: Remove unneeded semicolon Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 14/91] perf c2c: " Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 15/91] perf tools: Remove unneeded semicolons Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 16/91] perf report: Fix warning assignment of 0/1 to bool variable Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 17/91] perf pmu: Fix function name in comment, its get_cpuid_str(), not get_cpustr() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 18/91] perf metricgroups: Enhance JSON/metric infrastructure to handle "?" Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 19/91] perf tests expr: Added test for runtime param in metric expression Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 20/91] perf tools: Enable Hz/hz prinitg for --metric-only option Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 21/91] perf vendor events power9: Add hv_24x7 socket/chip level metric events Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 22/91] perf tools: Move routines that probe for perf API features to separate file Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 23/91] perf record: Move sb_evlist to 'struct record' Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 24/91] perf top: Move sb_evlist to 'struct perf_top' Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 25/91] perf bpf: Decouple creating the evlist from adding the SB event Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 26/91] perf parse-events: Add parse_events_option() variant that creates evlist Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 27/91] perf evlist: Move the sideband thread routines to separate object Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 28/91] perf evlist: Allow reusing the side band thread for more purposes Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 29/91] libsubcmd: Introduce OPT_CALLBACK_SET() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` Arnaldo Carvalho de Melo [this message]
2020-05-06 15:21 ` [PATCH 31/91] perf record: Move side band evlist setup to separate routine Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 32/91] libperf: Add NULL pointer check for cpu_map iteration and NULL assignment for all_cpus Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 33/91] perf parse-events: Fix memory leaks found on parse_events Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 34/91] " Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 35/91] perf parse-events: Fix another memory leaks found on parse_events() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 36/91] libperf evlist: Fix a refcount leak Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 37/91] perf tools: Fix reading new topology attribute "core_cpus" Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 38/91] perf tools: Simplify checking if SMT is active Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 39/91] perf thread-stack: Add branch stack support Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 40/91] perf intel-pt: Consolidate thread-stack use condition Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 41/91] perf intel-pt: Change branch stack support to use thread-stacks Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 42/91] perf auxtrace: Add option to synthesize branch stack for regular events Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 43/91] perf evsel: Add support for synthesized branch stack sample type Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 44/91] perf thread-stack: Add thread_stack__br_sample_late() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 45/91] perf intel-pt: Add support for synthesizing branch stacks for regular events Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 46/91] perf intel-pt: Update documentation about itrace G and L options Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 47/91] perf intel-pt: Update documentation about using /proc/kcore Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 48/91] perf evsel: Rename 'struct perf_evsel__sb_cb_t' to 'struct evsel__sb_cb_t' Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 49/91] perf evsel: Rename perf_evsel__nr_cpus() to evsel__nr_cpus() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 50/91] perf evsel: Rename perf_evsel__compute_deltas() to evsel__compute_deltas() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 51/91] perf evsel: Rename perf_evsel__find_pmu() to evsel__find_pmu() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 52/91] perf evsel: Rename perf_evsel__is_aux_event() to evsel__is_aux_event() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 53/91] perf evsel: Rename perf_evsel__exit() to evsel__exit() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 54/91] perf evsel: Rename perf_evsel__config*() to evsel__config*() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 55/91] perf evsel: Rename perf_evsel__calc_id_pos() to evsel__calc_id_pos() Arnaldo Carvalho de Melo
2020-05-06 15:21 ` [PATCH 56/91] perf evsel: Rename __perf_evsel__sample_size() to __evsel__sample_size() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 57/91] perf evsel: Rename *perf_evsel__*name() to *evsel__*name() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 58/91] perf evsel: Rename perf_evsel__group_desc() to evsel__group_desc() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 59/91] perf evsel: Rename *perf_evsel__*set_sample_*() to *evsel__*set_sample_*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 60/91] perf evsel: Rename perf_evsel__*filter*() to evsel__*filter*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 61/91] perf evsel: Rename perf_evsel__open_per_*() to evsel__open_per_*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 62/91] perf evsel: Rename perf_evsel__{str,int}val() and other tracepoint field metehods to to evsel__*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 63/91] perf mem2node: Avoid double free related to realloc Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 64/91] perf doc: Pass ASCIIDOC_EXTRA as an argument Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 65/91] tools feature: Add support for detecting libpfm4 Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 66/91] perf pmu: Add perf_pmu__find_by_type helper Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 67/91] perf evsel: Rename perf_evsel__is_*() to evsel__is*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 68/91] perf evsel: Ditch perf_evsel__cmp(), not used for quite a while Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 69/91] perf evsel: Rename *perf_evsel__read*() to *evsel__read() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 70/91] perf evsel: Rename perf_evsel__parse_sample*() to evsel__parse_sample*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 71/91] perf evsel: Rename perf_evsel__{prev,next}() to evsel__{prev,next}() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 72/91] perf evsel: Rename perf_evsel__has*() to evsel__has*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 73/91] perf evsel: Rename perf_evsel__fallback() to evsel__fallback() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 74/91] perf evsel: Rename perf_evsel__group_idx() to evsel__group_idx() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 75/91] perf evsel: Rename perf_evsel__env() to evsel__env() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 76/91] perf evsel: Rename perf_evsel__store_ids() to evsel__store_id() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 77/91] perf stat: Rename perf_evsel__*() operating on 'struct evsel *' to evsel__*() Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 78/91] perf kmem: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 79/91] perf lock: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 80/91] perf sched: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 81/91] perf script: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 82/91] perf trace: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 83/91] perf annotate: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 84/91] perf inject: " Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 85/91] perf symbol: Fix kernel symbol address display Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 86/91] perf: cs-etm: Update to build with latest opencsd version Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 87/91] perf bench: Add kallsyms parsing Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 88/91] libsymbols kallsyms: Parse using io api Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 89/91] libsymbols kallsyms: Move hex2u64 out of header Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 90/91] perf cs-etm: Move definition of 'traceid_list' global variable from header file Arnaldo Carvalho de Melo
2020-05-06 15:22 ` [PATCH 91/91] perf flamegraph: Use /bin/bash for report and record scripts Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200506152234.21977-31-acme@kernel.org \
--to=acme@kernel.org \
--cc=acme@redhat.com \
--cc=adrian.hunter@intel.com \
--cc=jolsa@kernel.org \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=namhyung@kernel.org \
--cc=songliubraving@fb.com \
--cc=tglx@linutronix.de \
--cc=wangnan0@huawei.com \
--cc=williams@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).