* [PATCH] Fix `make -jN trace-cmd gui` @ 2020-11-17 21:16 Dario Faggioli 2020-11-20 3:09 ` Steven Rostedt 0 siblings, 1 reply; 8+ messages in thread From: Dario Faggioli @ 2020-11-17 21:16 UTC (permalink / raw) To: rostedt; +Cc: linux-trace-devel Doing `make -j8 trace-cmd gui` fails like this: CMake Error at build/FindTraceCmd.cmake:110 (MESSAGE): Could not find libtraceevent! Call Stack (most recent call first): CMakeLists.txt:24 (include) Or like this: CMake Error at build/FindTraceCmd.cmake:64 (MESSAGE): Could not find trace-cmd! Call Stack (most recent call first): CMakeLists.txt:20 (include) That's because we need `trace-cmd` to have finished building before starting to build `gui`. Signed-off-by: Dario Faggioli <dfaggioli@suse.com> --- Makefile | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index b034042..c8d1e02 100644 --- a/Makefile +++ b/Makefile @@ -301,7 +301,9 @@ BUILD_TYPE ?= RelWithDebInfo $(kshark-dir)/build/Makefile: $(kshark-dir)/CMakeLists.txt $(Q) cd $(kshark-dir)/build && $(CMAKE_COMMAND) -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -D_INSTALL_PREFIX=$(prefix) -D_LIBDIR=$(libdir) .. -gui: force $(CMD_TARGETS) $(kshark-dir)/build/Makefile +gui: force + $(MAKE) $(CMD_TARGETS) + $(MAKE) $(kshark-dir)/build/Makefile $(Q)$(MAKE) $(S) -C $(kshark-dir)/build @echo "gui build complete" @echo " kernelshark located at $(kshark-dir)/bin" ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] Fix `make -jN trace-cmd gui` 2020-11-17 21:16 [PATCH] Fix `make -jN trace-cmd gui` Dario Faggioli @ 2020-11-20 3:09 ` Steven Rostedt 2020-11-20 12:06 ` Dario Faggioli 0 siblings, 1 reply; 8+ messages in thread From: Steven Rostedt @ 2020-11-20 3:09 UTC (permalink / raw) To: Dario Faggioli; +Cc: linux-trace-devel On Tue, 17 Nov 2020 21:16:45 +0000 Dario Faggioli <dfaggioli@suse.com> wrote: > Doing `make -j8 trace-cmd gui` fails like this: > > CMake Error at build/FindTraceCmd.cmake:110 (MESSAGE): > > Could not find libtraceevent! > > Call Stack (most recent call first): > CMakeLists.txt:24 (include) > > Or like this: > > CMake Error at build/FindTraceCmd.cmake:64 (MESSAGE): > > Could not find trace-cmd! > > Call Stack (most recent call first): > CMakeLists.txt:20 (include) > > That's because we need `trace-cmd` to have finished > building before starting to build `gui`. > > Signed-off-by: Dario Faggioli <dfaggioli@suse.com> Thanks Dario, I applied it. But note, we will be releasing kernelshark soon in its own repository (a new and improved version!) Stay tuned! -- Steve ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Fix `make -jN trace-cmd gui` 2020-11-20 3:09 ` Steven Rostedt @ 2020-11-20 12:06 ` Dario Faggioli 2020-11-20 12:32 ` Yordan Karadzhov (VMware) 0 siblings, 1 reply; 8+ messages in thread From: Dario Faggioli @ 2020-11-20 12:06 UTC (permalink / raw) To: Steven Rostedt; +Cc: linux-trace-devel [-- Attachment #1: Type: text/plain, Size: 1263 bytes --] On Thu, 2020-11-19 at 22:09 -0500, Steven Rostedt wrote: > On Tue, 17 Nov 2020 21:16:45 +0000 > Dario Faggioli <dfaggioli@suse.com> wrote: > > > > Signed-off-by: Dario Faggioli <dfaggioli@suse.com> > > Thanks Dario, > Hi! > I applied it. > Great! Thanks... This will help me, because I want to do: %build make %{?_smp_mflags} prefix=%{_prefix} trace-cmd gui in the openSUSE package for KernelShark (see the spec file here: https://build.opensuse.org/package/show/openSUSE:Factory/kernelshark ) which is now packaging version 1.2 > But note, we will be releasing kernelshark soon in its > own repository (a new and improved version!) > Yep, sure! I did followed the talks you've been giving about it at events, and I even played a little with what you have here: https://github.com/yordan-karadzhov/kernel-shark-2.alpha The host-guest tracing part, as I think you can guess. > Stay tuned! > Looking forward to it! Thanks again and Regards -- Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere) [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Fix `make -jN trace-cmd gui` 2020-11-20 12:06 ` Dario Faggioli @ 2020-11-20 12:32 ` Yordan Karadzhov (VMware) 2020-11-20 13:43 ` Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] Dario Faggioli 0 siblings, 1 reply; 8+ messages in thread From: Yordan Karadzhov (VMware) @ 2020-11-20 12:32 UTC (permalink / raw) To: Dario Faggioli, Steven Rostedt; +Cc: linux-trace-devel On 20.11.20 г. 14:06 ч., Dario Faggioli wrote: > On Thu, 2020-11-19 at 22:09 -0500, Steven Rostedt wrote: >> On Tue, 17 Nov 2020 21:16:45 +0000 >> Dario Faggioli <dfaggioli@suse.com> wrote: >>> >>> Signed-off-by: Dario Faggioli <dfaggioli@suse.com> >> >> Thanks Dario, >> > Hi! > >> I applied it. >> > Great! Thanks... This will help me, because I want to do: > > %build > make %{?_smp_mflags} prefix=%{_prefix} trace-cmd gui > > in the openSUSE package for KernelShark (see the spec file here: > https://build.opensuse.org/package/show/openSUSE:Factory/kernelshark ) > which is now packaging version 1.2 > >> But note, we will be releasing kernelshark soon in its >> own repository (a new and improved version!) >> > Yep, sure! I did followed the talks you've been giving about it at > events, and I even played a little with what you have here: > > https://github.com/yordan-karadzhov/kernel-shark-2.alpha > > The host-guest tracing part, as I think you can guess. Ciao Dario, I am very happy to hear that. The timestamp synchronization patches for trace-cmd are almost ready to go upstream. And we are currently reviewing a beta version of KernelShark_2 that will include the guest/host visualization. As Steven said: stay tuned! cheers, Yordan > >> Stay tuned! >> > Looking forward to it! > > Thanks again and Regards > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] 2020-11-20 12:32 ` Yordan Karadzhov (VMware) @ 2020-11-20 13:43 ` Dario Faggioli 2020-11-20 14:08 ` Steven Rostedt 0 siblings, 1 reply; 8+ messages in thread From: Dario Faggioli @ 2020-11-20 13:43 UTC (permalink / raw) To: Yordan Karadzhov (VMware), Steven Rostedt; +Cc: linux-trace-devel [-- Attachment #1: Type: text/plain, Size: 2004 bytes --] On Fri, 2020-11-20 at 14:32 +0200, Yordan Karadzhov (VMware) wrote: > On 20.11.20 г. 14:06 ч., Dario Faggioli wrote: > > > > > Yep, sure! I followed the talks you've been giving about it at > > events, and I even played a little with what you have here: > > > > https://github.com/yordan-karadzhov/kernel-shark-2.alpha > > > > The host-guest tracing part, as I think you can guess. > > Ciao Dario, > Hey! > I am very happy to hear that. The timestamp synchronization patches > for > trace-cmd are almost ready to go upstream. > Yes, I tried those patches! In fact, now that I have you here, do you mind if I change the subject and ask a quick question about them? So, you often say that "the accuracy of the synchronization protocol is XX ms". Now, I guess that means that an event in the guest and the corresponding event in the host (or vice versa) are XX ms apart. And that's even after the synchronization of the two traces, is that right? Question is, how do you measure that? Sure, I can look manually for an occurrence of the pattern that I described above: i.e., an event in the guest, then the corresponding one in the host and compute the difference between the timestamps. But do you have a way to do so automatically, or with a script/program, etc? I saw that the series included a patch which was meant at debugging and profiling PTP, but even with that one applied and having it generate the graphs, I have not been able to get that info without manual inspection. > And we are currently reviewing a beta version of KernelShark_2 that > will > include the guest/host visualization. > As Steven said: stay tuned! > You bet I will. :-P Thanks and Regards -- Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere) [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] 2020-11-20 13:43 ` Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] Dario Faggioli @ 2020-11-20 14:08 ` Steven Rostedt 2020-11-26 0:15 ` Dario Faggioli 0 siblings, 1 reply; 8+ messages in thread From: Steven Rostedt @ 2020-11-20 14:08 UTC (permalink / raw) To: Dario Faggioli; +Cc: Yordan Karadzhov (VMware), linux-trace-devel On Fri, 20 Nov 2020 14:43:21 +0100 Dario Faggioli <dfaggioli@suse.com> wrote: > On Fri, 2020-11-20 at 14:32 +0200, Yordan Karadzhov (VMware) wrote: > > On 20.11.20 г. 14:06 ч., Dario Faggioli wrote: > > > > > > > Yep, sure! I followed the talks you've been giving about it at > > > events, and I even played a little with what you have here: > > > > > > https://github.com/yordan-karadzhov/kernel-shark-2.alpha > > > > > > The host-guest tracing part, as I think you can guess. > > > > Ciao Dario, > > > Hey! > > > I am very happy to hear that. The timestamp synchronization patches > > for > > trace-cmd are almost ready to go upstream. > > > Yes, I tried those patches! > > In fact, now that I have you here, do you mind if I change the subject > and ask a quick question about them? > > So, you often say that "the accuracy of the synchronization protocol is > XX ms". Now, I guess that means that an event in the guest and the Note, we are usually microsecond (us) apart, not millisecond (ms) ;-) > corresponding event in the host (or vice versa) are XX ms apart. And > that's even after the synchronization of the two traces, is that right? At plumbers we talked with Thomas Gleixner and he suggested ideas of how to get to the actual shifts used in the hardware that should give us exact timestamp offsets. We are currently working on that. But in the mean time, the P2P is giving us somewhere between 5 and 10 us accuracy. And that's simply because the jitter of the vsock connection (which is used for the synchronization at start and end of the traces) has a 5 to 10 us jitter, and it's not possible to get a more accurate than the medium that is being used. > > Question is, how do you measure that? Sure, I can look manually for an > occurrence of the pattern that I described above: i.e., an event in the > guest, then the corresponding one in the host and compute the > difference between the timestamps. You mean, how we measure the accuracy? It's usually done by seeing when we have events from the guest showing up when we should be in the host (it's like seeing events from userspace when you are in the kernel). > > But do you have a way to do so automatically, or with a script/program, > etc? We have talked about having something scan to find cases where the guest event happens in kernel and do some post processing shifting, but haven't gotten there yet. If the hardware settings can work, then there will be no need to do so. -- Steve > > I saw that the series included a patch which was meant at debugging and > profiling PTP, but even with that one applied and having it generate > the graphs, I have not been able to get that info without manual > inspection. > > > And we are currently reviewing a beta version of KernelShark_2 that > > will > > include the guest/host visualization. > > As Steven said: stay tuned! > > > You bet I will. :-P > > Thanks and Regards ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] 2020-11-20 14:08 ` Steven Rostedt @ 2020-11-26 0:15 ` Dario Faggioli 2020-11-26 3:53 ` Tzvetomir Stoyanov 0 siblings, 1 reply; 8+ messages in thread From: Dario Faggioli @ 2020-11-26 0:15 UTC (permalink / raw) To: Steven Rostedt; +Cc: Yordan Karadzhov (VMware), linux-trace-devel [-- Attachment #1: Type: text/plain, Size: 3154 bytes --] On Fri, 2020-11-20 at 09:08 -0500, Steven Rostedt wrote: > On Fri, 20 Nov 2020 14:43:21 +0100 > Dario Faggioli <dfaggioli@suse.com> wrote: > > > > So, you often say that "the accuracy of the synchronization > > protocol is > > XX ms". Now, I guess that means that an event in the guest and the > > Note, we are usually microsecond (us) apart, not millisecond (ms) ;-) > Ah, yes, sure... And sorry about that! I know it us, I'm not sure how I ended up writing ms. That would be quite terrible indeed! :-D > > corresponding event in the host (or vice versa) are XX ms apart. > > And > > that's even after the synchronization of the two traces, is that > > right? > > At plumbers we talked with Thomas Gleixner and he suggested ideas of > how to > get to the actual shifts used in the hardware that should give us > exact > timestamp offsets. We are currently working on that. > Yes, I remember that, I attended the BoF. > But in the mean time, > the P2P is giving us somewhere between 5 and 10 us accuracy. And > that's > simply because the jitter of the vsock connection (which is used for > the > synchronization at start and end of the traces) has a 5 to 10 us > jitter, > and it's not possible to get a more accurate than the medium that is > being > used. > Yes, with a student that I was helping with his thesis, we applied one debug patch to trace-cmd that you have on this list, and we tried the different synchronization strategies, frequency, etc. > > Question is, how do you measure that? Sure, I can look manually for > > an > > occurrence of the pattern that I described above: i.e., an event in > > the > > guest, then the corresponding one in the host and compute the > > difference between the timestamps. > > You mean, how we measure the accuracy? It's usually done by seeing > when we > have events from the guest showing up when we should be in the host > (it's > like seeing events from userspace when you are in the kernel). > Ok, makes sense. I need to try it first hand to make sure I've properly understood it, though. I'll collect some more tracing and looks for situations like these. Thanks! > > But do you have a way to do so automatically, or with a > > script/program, > > etc? > > We have talked about having something scan to find cases where the > guest > event happens in kernel and do some post processing shifting, but > haven't > gotten there yet. > Yep, as said, I was thinking at it as a way to measure how accurately the traces are synched, but indeed once one has it, it can even use it to actually synch them better. But I understand how it's rather tricky. > If the hardware settings can work, then there will be no > need to do so. > Indeed. Well, perhaps it could still be useful, as a test/check whether things are working? :-) Regards -- Dario Faggioli, Ph.D http://about.me/dario.faggioli Virtualization Software Engineer SUSE Labs, SUSE https://www.suse.com/ ------------------------------------------------------------------- <<This happens because _I_ choose it to happen!>> (Raistlin Majere) [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] 2020-11-26 0:15 ` Dario Faggioli @ 2020-11-26 3:53 ` Tzvetomir Stoyanov 0 siblings, 0 replies; 8+ messages in thread From: Tzvetomir Stoyanov @ 2020-11-26 3:53 UTC (permalink / raw) To: Dario Faggioli Cc: Steven Rostedt, Yordan Karadzhov (VMware), Linux Trace Devel On Thu, Nov 26, 2020 at 2:15 AM Dario Faggioli <dfaggioli@suse.com> wrote: > > On Fri, 2020-11-20 at 09:08 -0500, Steven Rostedt wrote: > > On Fri, 20 Nov 2020 14:43:21 +0100 > > Dario Faggioli <dfaggioli@suse.com> wrote: > > > > > > So, you often say that "the accuracy of the synchronization > > > protocol is > > > XX ms". Now, I guess that means that an event in the guest and the > > > > Note, we are usually microsecond (us) apart, not millisecond (ms) ;-) > > > Ah, yes, sure... And sorry about that! I know it us, I'm not sure how I > ended up writing ms. That would be quite terrible indeed! :-D > > > > corresponding event in the host (or vice versa) are XX ms apart. > > > And > > > that's even after the synchronization of the two traces, is that > > > right? > > > > At plumbers we talked with Thomas Gleixner and he suggested ideas of > > how to > > get to the actual shifts used in the hardware that should give us > > exact > > timestamp offsets. We are currently working on that. > > > Yes, I remember that, I attended the BoF. > > > But in the mean time, > > the P2P is giving us somewhere between 5 and 10 us accuracy. And > > that's > > simply because the jitter of the vsock connection (which is used for > > the > > synchronization at start and end of the traces) has a 5 to 10 us > > jitter, > > and it's not possible to get a more accurate than the medium that is > > being > > used. > > > Yes, with a student that I was helping with his thesis, we applied one > debug patch to trace-cmd that you have on this list, and we tried the > different synchronization strategies, frequency, etc. > > > > Question is, how do you measure that? Sure, I can look manually for > > > an > > > occurrence of the pattern that I described above: i.e., an event in > > > the > > > guest, then the corresponding one in the host and compute the > > > difference between the timestamps. > > > > You mean, how we measure the accuracy? It's usually done by seeing > > when we > > have events from the guest showing up when we should be in the host > > (it's > > like seeing events from userspace when you are in the kernel). > > > Ok, makes sense. I need to try it first hand to make sure I've properly > understood it, though. I'll collect some more tracing and looks for > situations like these. > > Thanks! > > > > But do you have a way to do so automatically, or with a > > > script/program, > > > etc? > > > > We have talked about having something scan to find cases where the > > guest > > event happens in kernel and do some post processing shifting, but > > haven't > > gotten there yet. > > > Yep, as said, I was thinking at it as a way to measure how accurately > the traces are synched, but indeed once one has it, it can even use it > to actually synch them better. Hi Dario There is one approach to measure the accuracy of the synchronisation. May be you remember the suggestions proposed in the BoF - KVM exposes the clock offset and scaling factor in the /proc FS. In the last version of the time sync patch set, v25, I've implemented such logic - when x86-tsc is used for trace clock source, the offset is read from the KVM debug FS. In theory this should bring the best synchronisation, but I still see some guest events in wrong order according to the host events. I use the offset exposed from KVM as a reference - run PTP algorithm using the x86-tsc trace clock and compare the calculated offset with the one from the KVM debug FS. We still need to find a way to improve the synchronisation accuracy for cases when other hypervisors and trace clocks are used. > > But I understand how it's rather tricky. > > > If the hardware settings can work, then there will be no > > need to do so. > > > Indeed. Well, perhaps it could still be useful, as a test/check whether > things are working? :-) > > Regards > -- > Dario Faggioli, Ph.D > http://about.me/dario.faggioli > Virtualization Software Engineer > SUSE Labs, SUSE https://www.suse.com/ > ------------------------------------------------------------------- > <<This happens because _I_ choose it to happen!>> (Raistlin Majere) -- Tzvetomir (Ceco) Stoyanov VMware Open Source Technology Center ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-11-26 3:53 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-17 21:16 [PATCH] Fix `make -jN trace-cmd gui` Dario Faggioli 2020-11-20 3:09 ` Steven Rostedt 2020-11-20 12:06 ` Dario Faggioli 2020-11-20 12:32 ` Yordan Karadzhov (VMware) 2020-11-20 13:43 ` Accuracy of traces sync [Was: Re: [PATCH] Fix `make -jN trace-cmd gui`] Dario Faggioli 2020-11-20 14:08 ` Steven Rostedt 2020-11-26 0:15 ` Dario Faggioli 2020-11-26 3:53 ` Tzvetomir Stoyanov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.