From: Andrew Murray <andrew.murray@arm.com>
To: Christoffer Dall <christoffer.dall@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>,
linux-alpha@vger.kernel.org,
Peter Zijlstra <peterz@infradead.org>,
Sascha Hauer <s.hauer@pengutronix.de>,
x86@kernel.org, Will Deacon <will.deacon@arm.com>,
linuxppc-dev@lists.ozlabs.org,
Arnaldo Carvalho de Melo <acme@kernel.org>,
linux-kernel@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
Borislav Petkov <bp@alien8.de>,
"paulus@samba.org" <paulus@samba.org>,
Thomas Gleixner <tglx@linutronix.de>,
Shawn Guo <shawnguo@kernel.org>, Joerg Roedel <joro@8bytes.org>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 10/10] perf/doc: update design.txt for exclude_{host|guest} flags
Date: Wed, 12 Dec 2018 17:08:19 +0000 [thread overview]
Message-ID: <20181212170819.GD48249@e119886-lin.cambridge.arm.com> (raw)
In-Reply-To: <20181212080742.GA30263@e113682-lin.lund.arm.com>
On Wed, Dec 12, 2018 at 09:07:42AM +0100, Christoffer Dall wrote:
> On Tue, Dec 11, 2018 at 01:59:03PM +0000, Andrew Murray wrote:
> > On Tue, Dec 11, 2018 at 10:06:53PM +1100, Michael Ellerman wrote:
> > > [ Reviving old thread. ]
> > >
> > > Andrew Murray <andrew.murray@arm.com> writes:
> > > > On Tue, Nov 20, 2018 at 10:31:36PM +1100, Michael Ellerman wrote:
> > > >> Andrew Murray <andrew.murray@arm.com> writes:
> > > >>
> > > >> > Update design.txt to reflect the presence of the exclude_host
> > > >> > and exclude_guest perf flags.
> > > >> >
> > > >> > Signed-off-by: Andrew Murray <andrew.murray@arm.com>
> > > >> > ---
> > > >> > tools/perf/design.txt | 4 ++++
> > > >> > 1 file changed, 4 insertions(+)
> > > >> >
> > > >> > diff --git a/tools/perf/design.txt b/tools/perf/design.txt
> > > >> > index a28dca2..7de7d83 100644
> > > >> > --- a/tools/perf/design.txt
> > > >> > +++ b/tools/perf/design.txt
> > > >> > @@ -222,6 +222,10 @@ The 'exclude_user', 'exclude_kernel' and 'exclude_hv' bits provide a
> > > >> > way to request that counting of events be restricted to times when the
> > > >> > CPU is in user, kernel and/or hypervisor mode.
> > > >> >
> > > >> > +Furthermore the 'exclude_host' and 'exclude_guest' bits provide a way
> > > >> > +to request counting of events restricted to guest and host contexts when
> > > >> > +using virtualisation.
> > > >>
> > > >> How does exclude_host differ from exclude_hv ?
> > > >
> > > > I believe exclude_host / exclude_guest are intented to distinguish
> > > > between host and guest in the hosted hypervisor context (KVM).
> > >
> > > OK yeah, from the perf-list man page:
> > >
> > > u - user-space counting
> > > k - kernel counting
> > > h - hypervisor counting
> > > I - non idle counting
> > > G - guest counting (in KVM guests)
> > > H - host counting (not in KVM guests)
> > >
> > > > Whereas exclude_hv allows to distinguish between guest and
> > > > hypervisor in the bare-metal type hypervisors.
> > >
> > > Except that's exactly not how we use them on powerpc :)
> > >
> > > We use exclude_hv to exclude "the hypervisor", regardless of whether
> > > it's KVM or PowerVM (which is a bare-metal hypervisor).
> > >
> > > We don't use exclude_host / exclude_guest at all, which I guess is a
> > > bug, except I didn't know they existed until this thread.
> > >
> > > eg, in a KVM guest:
> > >
> > > $ perf record -e cycles:G /bin/bash -c "for i in {0..100000}; do :;done"
> > > $ perf report -D | grep -Fc "dso: [hypervisor]"
> > > 16
> > >
> > >
> > > > In the case of arm64 - if VHE extensions are present then the host
> > > > kernel will run at a higher privilege to the guest kernel, in which
> > > > case there is no distinction between hypervisor and host so we ignore
> > > > exclude_hv. But where VHE extensions are not present then the host
> > > > kernel runs at the same privilege level as the guest and we use a
> > > > higher privilege level to switch between them - in this case we can
> > > > use exclude_hv to discount that hypervisor role of switching between
> > > > guests.
> > >
> > > I couldn't find any arm64 perf code using exclude_host/guest at all?
> >
> > Correct - but this is in flight as I am currently adding support for this
> > see [1].
> >
> > >
> > > And I don't see any x86 code using exclude_hv.
> >
> > I can't find any either.
> >
> > >
> > > But maybe that's OK, I just worry this is confusing for users.
> >
> > There is some extra context regarding this where exclude_guest/exclude_host
> > was added, see [2] and where exclude_hv was added, see [3]
> >
> > Generally it seems that exclude_guest/exclude_host relies upon switching
> > counters off/on on guest/host switch code (which works well in the nested
> > virt case). Whereas exclude_hv tends to rely solely on hardware capability
> > based on privilege level (which works well in the bare metal case where
> > the guest doesn't run at same privilege as the host).
> >
> > I think from the user perspective exclude_hv allows you to see your overhead
> > if you are a guest (i.e. work done by bare metal hypervisor associated with
> > you as the guest). Whereas exclude_guest/exclude_host doesn't allow you to
> > see events above you (i.e. the kernel hypervisor) if you are the guest...
> >
> > At least that's how I read this, I've copied in others that may provide
> > more authoritative feedback.
> >
> > [1] https://lists.cs.columbia.edu/pipermail/kvmarm/2018-December/033698.html
> > [2] https://www.spinics.net/lists/kvm/msg53996.html
> > [3] https://lore.kernel.org/patchwork/patch/143918/
> >
>
> I'll try to answer this in a different way, based on previous
> discussions with Joerg et al. who introduced these flags. Assume no
> support for nested virtualization as a first approximation:
>
> If you are running as a guest:
> - exclude_hv: stop counting events when the hypervisor runs
> - exclude_host: has no effect
> - exclude_guest: has no effect
>
> If you are running as a host/hypervisor:
> - exclude_hv: has no effect
> - exclude_host: only count events when the guest is running
> - exclude_guest: only count events when the host is running
>
> With nested virtualization, you get the natural union of the above.
>
> **This has nothing to do with the design of the hypervisor such as the
> ARM non-VHE KVM which splits its execution across EL1 and EL2 -- those
> are both considered host from the point of view of Linux as a hypervisor
> using KVM, and both considered hypervisor from the point of view of a
> guest.**
For clarity, this is what arm64 currently does (assuming no nesting and
without the current version of this patchset):
If you are running as a guest (VHE or !VHE host):
- exclude_hv: has no effect for a KVM guest (filters hypervisor on !VHE
bare metal hypervisor guest)
- exclude_host: has no effect
- exclude_guest: has no effect
If you are running as a host/hypervisor:
- exclude_hv: has no effect for VHE (filters EL2 on !VHE)
- exclude_host: only count events when the guest is running
- exclude_guest: only count events when the host is running
Is this as expected?
Thanks,
Andrew Murray
>
>
> Thanks,
>
> Christoffer
next prev parent reply other threads:[~2018-12-12 17:11 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-16 10:24 [PATCH 00/10] perf/core: Generalise event exclusion checking Andrew Murray
2018-11-16 10:24 ` [PATCH 01/10] perf/core: Add macro to test for event exclusion flags Andrew Murray
2018-11-19 12:58 ` Peter Zijlstra
2018-11-19 16:01 ` Mark Rutland
2018-11-20 11:28 ` Michael Ellerman
2018-11-23 14:32 ` Andrew Murray
2018-11-16 10:24 ` [PATCH 02/10] arm: perf/core: generalise event exclusion checking with perf macro Andrew Murray
2018-11-16 10:24 ` [PATCH 03/10] arm: perf: add additional validation to set_event_filter Andrew Murray
2018-11-16 10:24 ` [PATCH 04/10] powerpc: perf/core: generalise event exclusion checking with perf macro Andrew Murray
2018-11-20 11:21 ` Michael Ellerman
2018-11-16 10:24 ` [PATCH 05/10] powerpc/pmu/fsl: add additional validation to event_init Andrew Murray
2018-11-16 10:24 ` [PATCH 06/10] alpha: perf/core: generalise event exclusion checking with perf macro Andrew Murray
2018-11-16 10:24 ` [PATCH 07/10] x86: " Andrew Murray
2018-11-16 10:24 ` [PATCH 08/10] perf/core: Remove unused perf_flags Andrew Murray
2018-11-16 10:24 ` [PATCH 09/10] drivers/perf: perf/core: generalise event exclusion checking with perf macro Andrew Murray
2018-11-19 16:03 ` Mark Rutland
2018-11-23 14:31 ` Andrew Murray
2018-11-16 10:24 ` [PATCH 10/10] perf/doc: update design.txt for exclude_{host|guest} flags Andrew Murray
2018-11-20 11:31 ` Michael Ellerman
2018-11-20 13:32 ` Andrew Murray
2018-12-11 11:06 ` Michael Ellerman
2018-12-11 13:59 ` Andrew Murray
2018-12-12 4:48 ` Michael Ellerman
2018-12-12 8:07 ` Christoffer Dall
2018-12-12 17:08 ` Andrew Murray [this message]
2018-11-19 13:08 ` [PATCH 00/10] perf/core: Generalise event exclusion checking Peter Zijlstra
2018-11-22 12:21 ` Andrew Murray
2018-11-22 12:26 ` Peter Zijlstra
2018-11-22 12:59 ` Andrew Murray
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181212170819.GD48249@e119886-lin.cambridge.arm.com \
--to=andrew.murray@arm.com \
--cc=acme@kernel.org \
--cc=bp@alien8.de \
--cc=christoffer.dall@arm.com \
--cc=joro@8bytes.org \
--cc=linux-alpha@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=paulus@samba.org \
--cc=peterz@infradead.org \
--cc=s.hauer@pengutronix.de \
--cc=shawnguo@kernel.org \
--cc=tglx@linutronix.de \
--cc=will.deacon@arm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).