From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Ian Rogers <irogers@google.com>,
"Peter Zijlstra (Intel)" <peterz@infradead.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Borislav Petkov <bp@alien8.de>, Jiri Olsa <jolsa@redhat.com>,
Kan Liang <kan.liang@linux.intel.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Namhyung Kim <namhyung@kernel.org>,
Stephane Eranian <eranian@google.com>,
Thomas Gleixner <tglx@linutronix.de>,
Vince Weaver <vincent.weaver@maine.edu>,
Ingo Molnar <mingo@kernel.org>,
Wen Yang <wenyang@linux.alibaba.com>
Subject: [PATCH 4.19 50/58] perf/cgroups: Dont rotate events for cgroups unnecessarily
Date: Tue, 8 Jun 2021 20:27:31 +0200 [thread overview]
Message-ID: <20210608175933.924635262@linuxfoundation.org> (raw)
In-Reply-To: <20210608175932.263480586@linuxfoundation.org>
From: Ian Rogers <irogers@google.com>
commit fd7d55172d1e2e501e6da0a5c1de25f06612dc2e upstream.
Currently perf_rotate_context assumes that if the context's nr_events !=
nr_active a rotation is necessary for perf event multiplexing. With
cgroups, nr_events is the total count of events for all cgroups and
nr_active will not include events in a cgroup other than the current
task's. This makes rotation appear necessary for cgroups when it is not.
Add a perf_event_context flag that is set when rotation is necessary.
Clear the flag during sched_out and set it when a flexible sched_in
fails due to resources.
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: https://lkml.kernel.org/r/20190601082722.44543-1-irogers@google.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Wen Yang <wenyang@linux.alibaba.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
include/linux/perf_event.h | 5 +++++
kernel/events/core.c | 42 ++++++++++++++++++++++--------------------
2 files changed, 27 insertions(+), 20 deletions(-)
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -747,6 +747,11 @@ struct perf_event_context {
int nr_stat;
int nr_freq;
int rotate_disable;
+ /*
+ * Set when nr_events != nr_active, except tolerant to events not
+ * necessary to be active due to scheduling constraints, such as cgroups.
+ */
+ int rotate_necessary;
atomic_t refcount;
struct task_struct *task;
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2952,6 +2952,12 @@ static void ctx_sched_out(struct perf_ev
if (!ctx->nr_active || !(is_active & EVENT_ALL))
return;
+ /*
+ * If we had been multiplexing, no rotations are necessary, now no events
+ * are active.
+ */
+ ctx->rotate_necessary = 0;
+
perf_pmu_disable(ctx->pmu);
if (is_active & EVENT_PINNED) {
list_for_each_entry_safe(event, tmp, &ctx->pinned_active, active_list)
@@ -3319,10 +3325,13 @@ static int flexible_sched_in(struct perf
return 0;
if (group_can_go_on(event, sid->cpuctx, sid->can_add_hw)) {
- if (!group_sched_in(event, sid->cpuctx, sid->ctx))
- list_add_tail(&event->active_list, &sid->ctx->flexible_active);
- else
+ int ret = group_sched_in(event, sid->cpuctx, sid->ctx);
+ if (ret) {
sid->can_add_hw = 0;
+ sid->ctx->rotate_necessary = 1;
+ return 0;
+ }
+ list_add_tail(&event->active_list, &sid->ctx->flexible_active);
}
return 0;
@@ -3690,24 +3699,17 @@ ctx_first_active(struct perf_event_conte
static bool perf_rotate_context(struct perf_cpu_context *cpuctx)
{
struct perf_event *cpu_event = NULL, *task_event = NULL;
- bool cpu_rotate = false, task_rotate = false;
- struct perf_event_context *ctx = NULL;
+ struct perf_event_context *task_ctx = NULL;
+ int cpu_rotate, task_rotate;
/*
* Since we run this from IRQ context, nobody can install new
* events, thus the event count values are stable.
*/
- if (cpuctx->ctx.nr_events) {
- if (cpuctx->ctx.nr_events != cpuctx->ctx.nr_active)
- cpu_rotate = true;
- }
-
- ctx = cpuctx->task_ctx;
- if (ctx && ctx->nr_events) {
- if (ctx->nr_events != ctx->nr_active)
- task_rotate = true;
- }
+ cpu_rotate = cpuctx->ctx.rotate_necessary;
+ task_ctx = cpuctx->task_ctx;
+ task_rotate = task_ctx ? task_ctx->rotate_necessary : 0;
if (!(cpu_rotate || task_rotate))
return false;
@@ -3716,7 +3718,7 @@ static bool perf_rotate_context(struct p
perf_pmu_disable(cpuctx->ctx.pmu);
if (task_rotate)
- task_event = ctx_first_active(ctx);
+ task_event = ctx_first_active(task_ctx);
if (cpu_rotate)
cpu_event = ctx_first_active(&cpuctx->ctx);
@@ -3724,17 +3726,17 @@ static bool perf_rotate_context(struct p
* As per the order given at ctx_resched() first 'pop' task flexible
* and then, if needed CPU flexible.
*/
- if (task_event || (ctx && cpu_event))
- ctx_sched_out(ctx, cpuctx, EVENT_FLEXIBLE);
+ if (task_event || (task_ctx && cpu_event))
+ ctx_sched_out(task_ctx, cpuctx, EVENT_FLEXIBLE);
if (cpu_event)
cpu_ctx_sched_out(cpuctx, EVENT_FLEXIBLE);
if (task_event)
- rotate_ctx(ctx, task_event);
+ rotate_ctx(task_ctx, task_event);
if (cpu_event)
rotate_ctx(&cpuctx->ctx, cpu_event);
- perf_event_sched_in(cpuctx, ctx, current);
+ perf_event_sched_in(cpuctx, task_ctx, current);
perf_pmu_enable(cpuctx->ctx.pmu);
perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
next prev parent reply other threads:[~2021-06-08 18:41 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-08 18:26 [PATCH 4.19 00/58] 4.19.194-rc1 review Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 01/58] net: usb: cdc_ncm: dont spew notifications Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 02/58] ALSA: usb: update old-style static const declaration Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 03/58] nl80211: validate key indexes for cfg80211_registered_device Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 04/58] efi: Allow EFI_MEMORY_XP and EFI_MEMORY_RO both to be cleared Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 05/58] efi: cper: fix snprintf() use in cper_dimm_err_location() Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 06/58] vfio/pci: Fix error return code in vfio_ecap_init() Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 07/58] vfio/pci: zap_vma_ptes() needs MMU Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 08/58] samples: vfio-mdev: fix error handing in mdpy_fb_probe() Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 09/58] vfio/platform: fix module_put call in error flow Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 10/58] ipvs: ignore IP_VS_SVC_F_HASHED flag when adding service Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 11/58] HID: pidff: fix error return code in hid_pidff_init() Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 12/58] HID: i2c-hid: fix format string mismatch Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 13/58] netfilter: nfnetlink_cthelper: hit EBUSY on updates if size mismatches Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 14/58] ieee802154: fix error return code in ieee802154_add_iface() Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 15/58] ieee802154: fix error return code in ieee802154_llsec_getparams() Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 16/58] ixgbevf: add correct exception tracing for XDP Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 17/58] tipc: add extack messages for bearer/media failure Greg Kroah-Hartman
2021-06-08 18:26 ` [PATCH 4.19 18/58] tipc: fix unique bearer names sanity check Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 19/58] Bluetooth: fix the erroneous flush_work() order Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 20/58] Bluetooth: use correct lock to prevent UAF of hdev object Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 21/58] net: caif: added cfserl_release function Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 22/58] net: caif: add proper error handling Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 23/58] net: caif: fix memory leak in caif_device_notify Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 24/58] net: caif: fix memory leak in cfusbl_device_notify Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 25/58] HID: multitouch: require Finger field to mark Win8 reports as MT Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 26/58] ALSA: timer: Fix master timer notification Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 27/58] ALSA: hda: Fix for mute key LED for HP Pavilion 15-CK0xx Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 28/58] ARM: dts: imx6q-dhcom: Add PU,VDD1P1,VDD2P5 regulators Greg Kroah-Hartman
2021-06-08 19:09 ` Sudip Mukherjee
2021-06-09 6:25 ` Greg Kroah-Hartman
2021-06-08 19:13 ` Naresh Kamboju
2021-06-09 6:25 ` Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 29/58] ext4: fix bug on in ext4_es_cache_extent as ext4_split_extent_at failed Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 30/58] usb: dwc2: Fix build in periphal-only mode Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 31/58] pid: take a reference when initializing `cad_pid` Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 32/58] ocfs2: fix data corruption by fallocate Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 33/58] nfc: fix NULL ptr dereference in llcp_sock_getname() after failed connect Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 34/58] x86/apic: Mark _all_ legacy interrupts when IO/APIC is missing Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 35/58] btrfs: mark ordered extent and inode with error if we fail to finish Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 36/58] btrfs: fix error handling in btrfs_del_csums Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 37/58] btrfs: return errors from btrfs_del_csums in cleanup_ref_head Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 38/58] btrfs: fixup error handling in fixup_inode_link_counts Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 39/58] mm, hugetlb: fix simple resv_huge_pages underflow on UFFDIO_COPY Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 40/58] bpf: fix test suite to enable all unpriv program types Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 41/58] bpf: test make sure to run unpriv test cases in test_verifier Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 42/58] selftests/bpf: Generalize dummy program types Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 43/58] bpf: Add BPF_F_ANY_ALIGNMENT Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 44/58] bpf: Adjust F_NEEDS_EFFICIENT_UNALIGNED_ACCESS handling in test_verifier.c Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 45/58] bpf: Make more use of any alignment " Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 46/58] bpf: Apply F_NEEDS_EFFICIENT_UNALIGNED_ACCESS to more ACCEPT test cases Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 47/58] selftests/bpf: add "any alignment" annotation for some tests Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 48/58] selftests/bpf: Avoid running unprivileged tests with alignment requirements Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 49/58] bnxt_en: Remove the setting of dev_port Greg Kroah-Hartman
2021-06-08 18:27 ` Greg Kroah-Hartman [this message]
2021-06-08 18:27 ` [PATCH 4.19 51/58] perf/core: Fix corner case in perf_rotate_context() Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 52/58] btrfs: fix unmountable seed device after fstrim Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 53/58] KVM: SVM: Truncate GPR value for DR and CR accesses in !64-bit mode Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 54/58] KVM: arm64: Fix debug register indexing Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 55/58] ACPI: probe ECDT before loading AML tables regardless of module-level code flag Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 56/58] ACPI: EC: Look for ECDT EC after calling acpi_load_tables() Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 57/58] sched/fair: Optimize select_idle_cpu Greg Kroah-Hartman
2021-06-08 18:27 ` [PATCH 4.19 58/58] xen-pciback: redo VF placement in the virtual topology Greg Kroah-Hartman
2021-06-08 18:45 ` [PATCH 4.19 00/58] 4.19.194-rc1 review Naresh Kamboju
2021-06-08 22:41 ` Pavel Machek
2021-06-09 6:25 ` Greg Kroah-Hartman
2021-06-10 11:40 ` Marek Vasut
2021-06-11 7:28 ` Greg Kroah-Hartman
2021-06-09 2:55 ` Shuah Khan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210608175933.924635262@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=acme@kernel.org \
--cc=acme@redhat.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=bp@alien8.de \
--cc=eranian@google.com \
--cc=irogers@google.com \
--cc=jolsa@redhat.com \
--cc=kan.liang@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=vincent.weaver@maine.edu \
--cc=wenyang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).