From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 262B3C433DB for ; Fri, 19 Mar 2021 11:56:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D1CC864F6E for ; Fri, 19 Mar 2021 11:56:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229524AbhCSLzg (ORCPT ); Fri, 19 Mar 2021 07:55:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229805AbhCSLzb (ORCPT ); Fri, 19 Mar 2021 07:55:31 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABB2BC06174A for ; Fri, 19 Mar 2021 04:55:31 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id j25so5721567pfe.2 for ; Fri, 19 Mar 2021 04:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mOoIz1/ZJKBbsTOcVPrYRpnLg8mFDEZxCXkTJkCh1gc=; b=WhxWxV+Oxr7g6FM77v/qzMFO77P/h9zZDGPsqaq97cRRJTzyuIh1/SHXqa8pfZCeEH wi4IDZit5LUEsZqJN3V6QSrVcXaNloWtoQVR7n1r9OWtEOo9/CelrGEzNo65F5WTMbQw U4w6NLGhhEdrMoPMGD+Ldd5LF004ypM7HAqQ2maTfrGeOncRi0cyViQEjh4V3Mr/E/z1 DVICdj05Tyf/nOTPjMhbnv3vA28+QdwBkvF4dpJ1P5iOi3RTR8sKKkO7scewXzLOYr8d EzrkgNt0My2w4bDigaPhkfW1F8Q4psheo11shP503a9krnXnpdYQCz9gn4zisW2W2T99 IKdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mOoIz1/ZJKBbsTOcVPrYRpnLg8mFDEZxCXkTJkCh1gc=; b=iyb0v0/VgzdxXtQNlEhqENq9KjsC2QGNV1dtvauyZHXfqHyR06vMYGuChc9PUF8bWl jwUW0EFNYsrV/AXORFmVmTl8cTJwQQriX/9f5hd6VJibvxBxBr6GPAhGBaLnFXmKlpRm A7aBOrfd0qNL8tU//HNrmgDwvSMtqPZFSFgKhn6tqNM0Livz37WcD2i6NwqNkvYRfc9Y 6mtSdNsam6GsRpArPCSahBNPt8kZUVupAWdF0OUNqBNoIQjD558OH2FkZbVdMJd9itbo U1p9DdBBZqx1BacrCQ6j7KfjHOlixfq6FH23My/8eLvMnKiI6G9mmjLHHgOC9ixlbkSS F/Dw== X-Gm-Message-State: AOAM530SyOodiMq/mmcNMF3XZwZn+LWMNTTpym+f1KLQSqB+21t+Emff L4iIPwKRxXXuxk0D/dItNydA13Jpdk3Mjqsx76M= X-Google-Smtp-Source: ABdhPJzcGfwGPKMJ2EqLOp2BeaLX/xNUclSGju/ktOSFJyTGc1+MuV7dUcQRB8ru+ztSHQcYBu9tNNWdz7l+DiwwBgI= X-Received: by 2002:a63:c143:: with SMTP id p3mr2345799pgi.44.1616154931273; Fri, 19 Mar 2021 04:55:31 -0700 (PDT) MIME-Version: 1.0 References: <20210315061857.168570-1-tz.stoyanov@gmail.com> <6796a75b-6a59-767f-08ec-ee0cbfd8c825@gmail.com> In-Reply-To: <6796a75b-6a59-767f-08ec-ee0cbfd8c825@gmail.com> From: Tzvetomir Stoyanov Date: Fri, 19 Mar 2021 13:55:14 +0200 Message-ID: Subject: Re: [PATCH v32 0/5]Timestamp synchronization of host - guest tracing session To: Stefano De Venuto Cc: Steven Rostedt , Linux Trace Devel , Dario Faggioli Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-trace-devel@vger.kernel.org Hi Stefano, On Fri, Mar 19, 2021 at 12:08 PM Stefano De Venuto wrote: > > Hello, > > On 3/15/21 7:18 AM, Tzvetomir Stoyanov (VMware) wrote: > > Basic infrastructure for host - guest timestamp synchronization and a > > PoC implementation of PTP-like and KVM algorithms. > I'm trying to make some analysis on traces generated by host and guest, > so I applied this series and the related dependencies. > > However, when the KVM algorithm is used, the merging process > of the two traces doesn't seem to happen properly. > > The output of `trace-cmd report -i trace.dat -i trace-tumbleweed.dat` is: > > trace.dat: cpus=4 > trace-tumbleweed.dat: cpus=1 > trace.dat: sleep-23948 [002]61124758463654: > write_msr: c0000100, value 7f35092875c0 > trace.dat: -0 [000]61124760841508: > write_msr: c0000100, value 7f6136b75640 > trace.dat: CPU 0/KVM-1640 [000]61124760851620: > write_msr: c0011020, value 40000000000000 > trace.dat: CPU 0/KVM-1640 [000]61124760861778: > read_msr: c0000101, value ffff9b1b76c00000 > trace.dat: CPU 0/KVM-1640 [000]61124760862832: > read_msr: c0000081, value 23001000000000 > trace.dat: CPU 0/KVM-1640 [000]61124760864347: > read_msr: c0000082, value ffffffffb5400010 > trace.dat: CPU 0/KVM-1640 [000]61124760865225: > read_msr: c0000083, value ffffffffb5401670 > . > . > . > trace-tumbleweed.dat: IPC I/O Parent-4313 [000]8446232460209009653: > write_msr: c0000100, value 7f94535c7640 > trace-tumbleweed.dat: IPC I/O Child-4503 [000]8446348845232796661: > write_msr: c001011f, value 4 > trace-tumbleweed.dat: IPC I/O Child-4503 [000]8447532508154803189: > write_msr: c0000100, value 7f945e8d5780 > trace-tumbleweed.dat: -0 [000]8452336523564659701: > write_msr: c001011f, value 0 > trace-tumbleweed.dat: -0 [000]8452515224268944373: > write_msr: 6e0, value 731478a3bf62 > trace-tumbleweed.dat: -0 [000]8452610851715789813: > write_msr: 6e0, value 731478968f37 > trace-tumbleweed.dat: -0 [000]8555757214338495477: > write_msr: 6e0, value 731478853ec1 > trace-tumbleweed.dat: -0 [000]8555904475882173429: > write_msr: 6e0, value 7314778a60e8 > trace-tumbleweed.dat: kworker/0:1-7351 [000]8556355589182174197: > write_msr: c0000100, value 7f8373573740 > > The TimeShift information is inside the guest trace file, > so I think this is not a transfer problem. > > I also tried to remove the last patch, the one related to the KVM clock, > in order to use PTP and the two traces are merged correctly. > > The commands used to record are: > > Host: > # trace-cmd record -C x86-tsc -e kvm:* -e msr:* -A tumbleweed:823 -e > msr:* -C x86-tsc sleep 1 The guest trace clock is set automatically as the host, so this command should be enough: # trace-cmd record -C x86-tsc -e kvm:* -e msr:* -A tumbleweed:823 -e msr:* sleep 1 > > Guest: > # echo x86-tsc > /sys/kernel/tracing/trace_clock There is no need to set manually the guest clock, it will be overwritten by trace-cmd agent. > # trace-cmd agent > > Am I doing something wrong? All your commands and workflow look good, nothing wrong. > > If necessary, I can provide more info about my setup, or do more tests. Yes, please can you send me both host and guest trace files ? Also, it will be useful to send me the content of the KVM debug files: /sys/kernel/debug/kvm//vcpu<*>/tsc-offset > > > Thanks and Regards, > > Stefano Thanks for testing this code! -- Tzvetomir (Ceco) Stoyanov VMware Open Source Technology Center