* [PATCH 1/2] tracing: show size of requested buffer
@ 2021-08-31 4:37 Robin H. Johnson
2021-08-31 4:37 ` [PATCH 2/2] tracing: increase PERF_MAX_TRACE_SIZE to handle Sentinel1 and docker together Robin H. Johnson
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Robin H. Johnson @ 2021-08-31 4:37 UTC (permalink / raw)
To: linux-kernel; +Cc: rostedt, mingo, rjohnson, Robin H. Johnson
If the perf buffer isn't large enough, provide a hint about how large it
needs to be for whatever is running.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
kernel/trace/trace_event_perf.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
index 03be4435d103..26eed4b89100 100644
--- a/kernel/trace/trace_event_perf.c
+++ b/kernel/trace/trace_event_perf.c
@@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
- "perf buffer not large enough"))
+ "perf buffer not large enough, wanted %d, have %d",
+ size, PERF_MAX_TRACE_SIZE))
return NULL;
*rctxp = rctx = perf_swevent_get_recursion_context();
--
2.33.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 2/2] tracing: increase PERF_MAX_TRACE_SIZE to handle Sentinel1 and docker together
2021-08-31 4:37 [PATCH 1/2] tracing: show size of requested buffer Robin H. Johnson
@ 2021-08-31 4:37 ` Robin H. Johnson
2021-09-08 1:24 ` [PATCH 1/2] tracing: show size of requested buffer Steven Rostedt
2021-10-06 22:26 ` Steven Rostedt
2 siblings, 0 replies; 14+ messages in thread
From: Robin H. Johnson @ 2021-08-31 4:37 UTC (permalink / raw)
To: linux-kernel; +Cc: rostedt, mingo, rjohnson, Robin H. Johnson
Running endpoint security solutions like Sentinel1 that use perf-based
tracing heavily lead to this repeated dump complaining about dockerd.
The default value of 2048 is nowhere near not large enough.
Using the prior patch "tracing: show size of requested buffer", we get
"perf buffer not large enough, wanted 6644, have 6144", after repeated
up-sizing (I did 2/4/6/8K). With 8K, the problem doesn't occur at all,
so below is the trace for 6K.
I'm wondering if this value should be selectable at boot time, but this
is a good starting point.
```
------------[ cut here ]------------
perf buffer not large enough, wanted 6644, have 6144
WARNING: CPU: 1 PID: 4997 at kernel/trace/trace_event_perf.c:402 perf_trace_buf_alloc+0x8c/0xa0
Modules linked in: veth rfcomm dm_integrity async_xor async_tx hid_logitech_hidpp hid_logitech_dj hid_lg_g15 nf_conntrack_netlink nfnetlink br_netfilter bridge stp llc overlay sctp ip6_udp_tunnel udp_tunnel cachefiles fscache netfs bnep ip6table_raw ip6table_security ip6table_mangle ip6table_nat ip6table_filter ip6_tables iptable_raw iptable_security xt_CHECKSUM iptable_mangle xt_nat xt_MASQUERADE xt_addrtype iptable_nat nf_nat ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_filter ip_tables bpfilter xfs vfat fat typec_displayport btusb btrtl btbcm btintel bluetooth snd_usb_audio uvcvideo joydev snd_usbmidi_lib snd_rawmidi ecdh_generic ecc snd_seq_device hid_multitouch rmi_smbus rmi_core videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc iTCO_wdt intel_pmc_bxt iTCO_vendor_support mei_hdcp mei_wdt intel_pmc_core_pltdrv intel_pmc_core wmi_bmof intel_wmi_thunderbolt tcp_bbr binfmt_misc intel_tcc_cooling
x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm dm_crypt trusted irqbypass iwlmvm asn1_encoder snd_hda_codec_hdmi crct10dif_pclmul ghash_clmulni_intel rapl intel_cstate snd_ctl_led mac80211 intel_uncore snd_hda_codec_realtek tun psmouse snd_hda_codec_generic libarc4 pkcs8_key_parser pcspkr serio_raw snd_hda_intel snd_intel_dspcfg iwlwifi snd_hda_codec i2c_i801 coretemp snd_hwdep i2c_smbus snd_hda_core snd_pcm snd_timer thunderbolt cfg80211 mei_me mei intel_pch_thermal intel_xhci_usb_role_switch processor_thermal_device processor_thermal_rfim processor_thermal_mbox processor_thermal_rapl intel_soc_dts_iosf ucsi_acpi typec_ucsi typec thinkpad_acpi wmi platform_profile ledtrig_audio snd soundcore rfkill int3403_thermal int340x_thermal_zone i2c_hid_acpi i2c_hid int3400_thermal acpi_thermal_rel acpi_pad uas usb_storage e1000e crc32_pclmul nvme crc32c_intel nvme_core ptp hwmon
CPU: 1 PID: 4997 Comm: sh Tainted: G T 5.13.13-x86_64-00039-gb3959163488e #63
Hardware name: LENOVO 20KH002JUS/20KH002JUS, BIOS N23ET66W (1.41 ) 09/02/2019
RIP: 0010:perf_trace_buf_alloc+0x8c/0xa0
Code: 80 3d 43 97 d0 01 00 74 07 31 c0 5b 5d 41 5c c3 ba 00 18 00 00 89 ee 48 c7 c7 00 82 7d 91 c6 05 25 97 d0 01 01 e8 22 ee bc 00 <0f> 0b 31 c0 eb db 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 55 89
RSP: 0018:ffffb922026b7d58 EFLAGS: 00010282
RAX: 0000000000000000 RBX: ffff9da5ee012000 RCX: 0000000000000027
RDX: ffff9da881657828 RSI: 0000000000000001 RDI: ffff9da881657820
RBP: 00000000000019f4 R08: 0000000000000000 R09: ffffb922026b7b80
R10: ffffb922026b7b78 R11: ffffffff91dda688 R12: 000000000000000f
R13: ffff9da5ee012108 R14: ffff9da8816570a0 R15: ffffb922026b7e30
FS: 00007f420db1a080(0000) GS:ffff9da881640000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000060 CR3: 00000002504a8006 CR4: 00000000003706e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
kprobe_perf_func+0x11e/0x270
? do_execveat_common.isra.0+0x1/0x1c0
? do_execveat_common.isra.0+0x5/0x1c0
kprobe_ftrace_handler+0x10e/0x1d0
0xffffffffc03aa0c8
? do_execveat_common.isra.0+0x1/0x1c0
do_execveat_common.isra.0+0x5/0x1c0
__x64_sys_execve+0x33/0x40
do_syscall_64+0x6b/0xc0
? do_syscall_64+0x11/0xc0
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f420dc1db37
Code: ff ff 76 e7 f7 d8 64 41 89 00 eb df 0f 1f 80 00 00 00 00 f7 d8 64 41 89 00 eb dc 0f 1f 84 00 00 00 00 00 b8 3b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 01 43 0f 00 f7 d8 64 89 01 48
RSP: 002b:00007ffd4e8b4e38 EFLAGS: 00000246 ORIG_RAX: 000000000000003b
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f420dc1db37
RDX: 0000564338d1e740 RSI: 0000564338d32d50 RDI: 0000564338d28f00
RBP: 0000564338d28f00 R08: 0000564338d32d50 R09: 0000000000000020
R10: 00000000000001b6 R11: 0000000000000246 R12: 0000564338d28f00
R13: 0000564338d32d50 R14: 0000564338d1e740 R15: 0000564338d28c60
---[ end trace 83ab3e8e16275e49 ]---
```
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
include/linux/trace_events.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index ad413b382a3c..0f5e177ec5cb 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -622,7 +622,7 @@ struct trace_event_file {
} \
early_initcall(trace_init_perf_perm_##name);
-#define PERF_MAX_TRACE_SIZE 2048
+#define PERF_MAX_TRACE_SIZE 8192
#define MAX_FILTER_STR_VAL 256 /* Should handle KSYM_SYMBOL_LEN */
--
2.33.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-08-31 4:37 [PATCH 1/2] tracing: show size of requested buffer Robin H. Johnson
2021-08-31 4:37 ` [PATCH 2/2] tracing: increase PERF_MAX_TRACE_SIZE to handle Sentinel1 and docker together Robin H. Johnson
@ 2021-09-08 1:24 ` Steven Rostedt
2021-10-07 7:11 ` Peter Zijlstra
2021-10-06 22:26 ` Steven Rostedt
2 siblings, 1 reply; 14+ messages in thread
From: Steven Rostedt @ 2021-09-08 1:24 UTC (permalink / raw)
To: Robin H. Johnson
Cc: linux-kernel, mingo, rjohnson, Peter Zijlstra, Arnaldo Carvalho de Melo
I'll need Acks for these patches from the Perf maintainers.
-- Steve
On Mon, 30 Aug 2021 21:37:22 -0700
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> If the perf buffer isn't large enough, provide a hint about how large it
> needs to be for whatever is running.
>
> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
> ---
> kernel/trace/trace_event_perf.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
> index 03be4435d103..26eed4b89100 100644
> --- a/kernel/trace/trace_event_perf.c
> +++ b/kernel/trace/trace_event_perf.c
> @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
>
> if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> - "perf buffer not large enough"))
> + "perf buffer not large enough, wanted %d, have %d",
> + size, PERF_MAX_TRACE_SIZE))
> return NULL;
>
> *rctxp = rctx = perf_swevent_get_recursion_context();
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-08-31 4:37 [PATCH 1/2] tracing: show size of requested buffer Robin H. Johnson
2021-08-31 4:37 ` [PATCH 2/2] tracing: increase PERF_MAX_TRACE_SIZE to handle Sentinel1 and docker together Robin H. Johnson
2021-09-08 1:24 ` [PATCH 1/2] tracing: show size of requested buffer Steven Rostedt
@ 2021-10-06 22:26 ` Steven Rostedt
2021-10-06 22:48 ` Robin H. Johnson
2 siblings, 1 reply; 14+ messages in thread
From: Steven Rostedt @ 2021-10-06 22:26 UTC (permalink / raw)
To: Robin H. Johnson
Cc: linux-kernel, mingo, rjohnson, Peter Zijlstra, Jiri Olsa,
Frederic Weisbecker
On Mon, 30 Aug 2021 21:37:22 -0700
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
Sorry for the late reply, I was on holiday when this was sent, and I'm just
getting to looking at this email now (as my OoO email should have suggested ;-)
Anyway, this needs to be reviewed by the Perf maintainers (Cc'd)
(Lore link for patch 2:
https://lore.kernel.org/all/20210831043723.13481-2-robbat2@gentoo.org/ )
-- Steve
> If the perf buffer isn't large enough, provide a hint about how large it
> needs to be for whatever is running.
>
> Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
> ---
> kernel/trace/trace_event_perf.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
> index 03be4435d103..26eed4b89100 100644
> --- a/kernel/trace/trace_event_perf.c
> +++ b/kernel/trace/trace_event_perf.c
> @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
>
> if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> - "perf buffer not large enough"))
> + "perf buffer not large enough, wanted %d, have %d",
> + size, PERF_MAX_TRACE_SIZE))
> return NULL;
>
> *rctxp = rctx = perf_swevent_get_recursion_context();
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-06 22:26 ` Steven Rostedt
@ 2021-10-06 22:48 ` Robin H. Johnson
2021-10-07 0:55 ` Steven Rostedt
0 siblings, 1 reply; 14+ messages in thread
From: Robin H. Johnson @ 2021-10-06 22:48 UTC (permalink / raw)
To: Steven Rostedt
Cc: Robin H. Johnson, linux-kernel, mingo, rjohnson, Peter Zijlstra,
Jiri Olsa, Frederic Weisbecker
[-- Attachment #1: Type: text/plain, Size: 937 bytes --]
On Wed, Oct 06, 2021 at 06:26:52PM -0400, Steven Rostedt wrote:
> On Mon, 30 Aug 2021 21:37:22 -0700
> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>
> Sorry for the late reply, I was on holiday when this was sent, and I'm just
> getting to looking at this email now (as my OoO email should have suggested ;-)
>
> Anyway, this needs to be reviewed by the Perf maintainers (Cc'd)
>
> (Lore link for patch 2:
> https://lore.kernel.org/all/20210831043723.13481-2-robbat2@gentoo.org/ )
You already CC'd them on Sept 7th, no response yet.
Does MAINTAINERS need an update for kernel/trace/trace_event_perf.c?
It points to Ingo & yourself for that directory, and not to the Perf
maintainers.
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-06 22:48 ` Robin H. Johnson
@ 2021-10-07 0:55 ` Steven Rostedt
0 siblings, 0 replies; 14+ messages in thread
From: Steven Rostedt @ 2021-10-07 0:55 UTC (permalink / raw)
To: Robin H. Johnson
Cc: linux-kernel, mingo, rjohnson, Peter Zijlstra, Jiri Olsa,
Frederic Weisbecker
On Wed, 6 Oct 2021 22:48:35 +0000
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> On Wed, Oct 06, 2021 at 06:26:52PM -0400, Steven Rostedt wrote:
> > On Mon, 30 Aug 2021 21:37:22 -0700
> > "Robin H. Johnson" <robbat2@gentoo.org> wrote:
> >
> > Sorry for the late reply, I was on holiday when this was sent, and I'm just
> > getting to looking at this email now (as my OoO email should have suggested ;-)
> >
> > Anyway, this needs to be reviewed by the Perf maintainers (Cc'd)
> >
> > (Lore link for patch 2:
> > https://lore.kernel.org/all/20210831043723.13481-2-robbat2@gentoo.org/ )
>
> You already CC'd them on Sept 7th, no response yet.
Oh good, that means I'm not the one that dropped the ball on this ;-)
>
> Does MAINTAINERS need an update for kernel/trace/trace_event_perf.c?
> It points to Ingo & yourself for that directory, and not to the Perf
> maintainers.
Well, it is maintained by both of us. I usually just ask them to review
before taking it, but if there's no response from them by the end of
the week, I'll add it to my "for-next" queue.
-- Steve
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-09-08 1:24 ` [PATCH 1/2] tracing: show size of requested buffer Steven Rostedt
@ 2021-10-07 7:11 ` Peter Zijlstra
2021-10-07 13:23 ` Steven Rostedt
0 siblings, 1 reply; 14+ messages in thread
From: Peter Zijlstra @ 2021-10-07 7:11 UTC (permalink / raw)
To: Steven Rostedt
Cc: Robin H. Johnson, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
On Tue, Sep 07, 2021 at 09:24:26PM -0400, Steven Rostedt wrote:
>
> I'll need Acks for these patches from the Perf maintainers.
>
> -- Steve
>
>
> On Mon, 30 Aug 2021 21:37:22 -0700
> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>
> > If the perf buffer isn't large enough, provide a hint about how large it
> > needs to be for whatever is running.
Is that an actual reason?
> > Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
> > ---
> > kernel/trace/trace_event_perf.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
> > index 03be4435d103..26eed4b89100 100644
> > --- a/kernel/trace/trace_event_perf.c
> > +++ b/kernel/trace/trace_event_perf.c
> > @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> > BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
> >
> > if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> > - "perf buffer not large enough"))
> > + "perf buffer not large enough, wanted %d, have %d",
> > + size, PERF_MAX_TRACE_SIZE))
Priting a constant seems daft.. why is any of this important in any way?
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-07 7:11 ` Peter Zijlstra
@ 2021-10-07 13:23 ` Steven Rostedt
2021-10-07 17:26 ` Robin H. Johnson
2021-10-26 20:43 ` Steven Rostedt
0 siblings, 2 replies; 14+ messages in thread
From: Steven Rostedt @ 2021-10-07 13:23 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Robin H. Johnson, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
On Thu, 7 Oct 2021 09:11:51 +0200
Peter Zijlstra <peterz@infradead.org> wrote:
> > > +++ b/kernel/trace/trace_event_perf.c
> > > @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> > > BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
> > >
> > > if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> > > - "perf buffer not large enough"))
> > > + "perf buffer not large enough, wanted %d, have %d",
> > > + size, PERF_MAX_TRACE_SIZE))
>
> Priting a constant seems daft.. why is any of this important in any way?
I see your point, but it can be useful if you changed it, and want to know
if you are running the kernel with the change or not.
I've done daft things were I changed a const and was running a kernel
without the change and couldn't understand why it wasn't working ;-)
-- Steve
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-07 13:23 ` Steven Rostedt
@ 2021-10-07 17:26 ` Robin H. Johnson
2021-10-08 22:13 ` Steven Rostedt
2021-10-26 20:43 ` Steven Rostedt
1 sibling, 1 reply; 14+ messages in thread
From: Robin H. Johnson @ 2021-10-07 17:26 UTC (permalink / raw)
To: Steven Rostedt
Cc: Peter Zijlstra, Robin H. Johnson, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]
On Thu, Oct 07, 2021 at 09:23:58AM -0400, Steven Rostedt wrote:
> On Thu, 7 Oct 2021 09:11:51 +0200
> Peter Zijlstra <peterz@infradead.org> wrote:
>
> > > > +++ b/kernel/trace/trace_event_perf.c
> > > > @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> > > > BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
> > > >
> > > > if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> > > > - "perf buffer not large enough"))
> > > > + "perf buffer not large enough, wanted %d, have %d",
> > > > + size, PERF_MAX_TRACE_SIZE))
> >
> > Priting a constant seems daft.. why is any of this important in any way?
>
> I see your point, but it can be useful if you changed it, and want to know
> if you are running the kernel with the change or not.
>
> I've done daft things were I changed a const and was running a kernel
> without the change and couldn't understand why it wasn't working ;-)
Yes, my initial internal versions of this series only bumped the
constant, and then I was running a few different kernels, and being able
to compare which value was in use became a problem.
I was trying to think further what would make sense for the constant.
- What are the negative impacts of a too-large value?
- Is there demand for more reconfigurability?
- Should PERF_MAX_TRACE_SIZE be a knob in Kconfig?
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-07 17:26 ` Robin H. Johnson
@ 2021-10-08 22:13 ` Steven Rostedt
2021-10-09 0:21 ` Robin H. Johnson
0 siblings, 1 reply; 14+ messages in thread
From: Steven Rostedt @ 2021-10-08 22:13 UTC (permalink / raw)
To: Robin H. Johnson
Cc: Peter Zijlstra, linux-kernel, mingo, rjohnson, Arnaldo Carvalho de Melo
On Thu, 7 Oct 2021 17:26:04 +0000
"Robin H. Johnson" <robbat2@gentoo.org> wrote:
> I was trying to think further what would make sense for the constant.
> - What are the negative impacts of a too-large value?
> - Is there demand for more reconfigurability?
> - Should PERF_MAX_TRACE_SIZE be a knob in Kconfig?
One thing you haven't discussed was, have you hit this warning, and if so,
what were you doing?
-- Steve
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-08 22:13 ` Steven Rostedt
@ 2021-10-09 0:21 ` Robin H. Johnson
0 siblings, 0 replies; 14+ messages in thread
From: Robin H. Johnson @ 2021-10-09 0:21 UTC (permalink / raw)
To: Steven Rostedt
Cc: Robin H. Johnson, Peter Zijlstra, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
[-- Attachment #1: Type: text/plain, Size: 1677 bytes --]
On Fri, Oct 08, 2021 at 06:13:48PM -0400, Steven Rostedt wrote:
> On Thu, 7 Oct 2021 17:26:04 +0000
> "Robin H. Johnson" <robbat2@gentoo.org> wrote:
>
> > I was trying to think further what would make sense for the constant.
> > - What are the negative impacts of a too-large value?
> > - Is there demand for more reconfigurability?
> > - Should PERF_MAX_TRACE_SIZE be a knob in Kconfig?
>
> One thing you haven't discussed was, have you hit this warning, and if so,
> what were you doing?
Ah, I covered that in patch 2/2 in the original series, which discussed
why I was raising the limit, and how I came to the 8K value for
PERF_MAX_TRACE_SIZE.
https://lore.kernel.org/all/20210831043723.13481-2-robbat2@gentoo.org/
To summarize here:
$work requires that all employees run endpoint security software from
SentinalOne [1]. I can see that it uses trace/perf stuff to dig deeply
into what's taking place on the system.
Something in my personal setup/configuration, leads to SentinelOne
getting large perf backlogs during heavy workloads, primarily when I'm
doing deep things with containers (esp with tun devices inside
containers). I haven't been able to narrow it down much more than that,
and I don't have the source to SentinelOne at all.
It does feel like SentinelOne either has a leak of some sort, or gets a
backlog of work that takes a noticible amount of time to clean up.
[1] https://www.sentinelone.com/
--
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1113 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-07 13:23 ` Steven Rostedt
2021-10-07 17:26 ` Robin H. Johnson
@ 2021-10-26 20:43 ` Steven Rostedt
2021-10-26 21:23 ` Peter Zijlstra
1 sibling, 1 reply; 14+ messages in thread
From: Steven Rostedt @ 2021-10-26 20:43 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Robin H. Johnson, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
On Thu, 7 Oct 2021 09:23:58 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
> On Thu, 7 Oct 2021 09:11:51 +0200
> Peter Zijlstra <peterz@infradead.org> wrote:
>
> > > > +++ b/kernel/trace/trace_event_perf.c
> > > > @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> > > > BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
> > > >
> > > > if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> > > > - "perf buffer not large enough"))
> > > > + "perf buffer not large enough, wanted %d, have %d",
> > > > + size, PERF_MAX_TRACE_SIZE))
> >
> > Priting a constant seems daft.. why is any of this important in any way?
>
> I see your point, but it can be useful if you changed it, and want to know
> if you are running the kernel with the change or not.
>
> I've done daft things were I changed a const and was running a kernel
> without the change and couldn't understand why it wasn't working ;-)
Peter,
Do you have any real issue if I just take this patch set through my tree?
-- Steve
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-26 20:43 ` Steven Rostedt
@ 2021-10-26 21:23 ` Peter Zijlstra
2021-10-26 21:26 ` Steven Rostedt
0 siblings, 1 reply; 14+ messages in thread
From: Peter Zijlstra @ 2021-10-26 21:23 UTC (permalink / raw)
To: Steven Rostedt
Cc: Robin H. Johnson, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
On Tue, Oct 26, 2021 at 04:43:43PM -0400, Steven Rostedt wrote:
> On Thu, 7 Oct 2021 09:23:58 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
>
> > On Thu, 7 Oct 2021 09:11:51 +0200
> > Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > > > > +++ b/kernel/trace/trace_event_perf.c
> > > > > @@ -400,7 +400,8 @@ void *perf_trace_buf_alloc(int size, struct pt_regs **regs, int *rctxp)
> > > > > BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
> > > > >
> > > > > if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE,
> > > > > - "perf buffer not large enough"))
> > > > > + "perf buffer not large enough, wanted %d, have %d",
> > > > > + size, PERF_MAX_TRACE_SIZE))
> > >
> > > Priting a constant seems daft.. why is any of this important in any way?
> >
> > I see your point, but it can be useful if you changed it, and want to know
> > if you are running the kernel with the change or not.
> >
> > I've done daft things were I changed a const and was running a kernel
> > without the change and couldn't understand why it wasn't working ;-)
>
> Peter,
>
> Do you have any real issue if I just take this patch set through my tree?
No real objections; just weary, huge events like that are fairly sucky
for performance.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/2] tracing: show size of requested buffer
2021-10-26 21:23 ` Peter Zijlstra
@ 2021-10-26 21:26 ` Steven Rostedt
0 siblings, 0 replies; 14+ messages in thread
From: Steven Rostedt @ 2021-10-26 21:26 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Robin H. Johnson, linux-kernel, mingo, rjohnson,
Arnaldo Carvalho de Melo
On Tue, 26 Oct 2021 23:23:32 +0200
Peter Zijlstra <peterz@infradead.org> wrote:
> > Do you have any real issue if I just take this patch set through my tree?
>
> No real objections; just weary, huge events like that are fairly sucky
> for performance.
OK. Then I'll just take this in, and let people have sucky performance if
they want huge events ;-)
-- Steve
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-10-26 21:26 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-31 4:37 [PATCH 1/2] tracing: show size of requested buffer Robin H. Johnson
2021-08-31 4:37 ` [PATCH 2/2] tracing: increase PERF_MAX_TRACE_SIZE to handle Sentinel1 and docker together Robin H. Johnson
2021-09-08 1:24 ` [PATCH 1/2] tracing: show size of requested buffer Steven Rostedt
2021-10-07 7:11 ` Peter Zijlstra
2021-10-07 13:23 ` Steven Rostedt
2021-10-07 17:26 ` Robin H. Johnson
2021-10-08 22:13 ` Steven Rostedt
2021-10-09 0:21 ` Robin H. Johnson
2021-10-26 20:43 ` Steven Rostedt
2021-10-26 21:23 ` Peter Zijlstra
2021-10-26 21:26 ` Steven Rostedt
2021-10-06 22:26 ` Steven Rostedt
2021-10-06 22:48 ` Robin H. Johnson
2021-10-07 0:55 ` Steven Rostedt
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.