From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAE88C3A5A6 for ; Thu, 19 Sep 2019 22:21:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A528920678 for ; Thu, 19 Sep 2019 22:21:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1568931662; bh=IaBM54VjXjLvBbyNVHfirjB5tFsoeGZwZDIjgVaGGhE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=IjDCN+E+SRdiJELCP3H1SKL9ibbG3AeQZBAaoQcBj9iYF28EK/jY2H9tgZfVTNPEE UbbVambOVYNry+/8rBY9LOr774ihBBn+mgL1BAbWx9XJT2ucJ6RyMc+kzvX4Q8xy5S LagcvWW08645+hyvD2dsDcYSMKWMHY3rUWZw3Er4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2406667AbfISWVB (ORCPT ); Thu, 19 Sep 2019 18:21:01 -0400 Received: from mail.kernel.org ([198.145.29.99]:35120 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2405750AbfISWUy (ORCPT ); Thu, 19 Sep 2019 18:20:54 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AB1A6217D6; Thu, 19 Sep 2019 22:20:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1568931654; bh=IaBM54VjXjLvBbyNVHfirjB5tFsoeGZwZDIjgVaGGhE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=c2gbgvRzEJDFpeiSx5oZ5QuLYAV41mesf20ZDz9JJDrSbvcZcnaQxqbfYKvyNa+GF xRvmOJ/BA1MBRS5tI6eU/e3AJBG9paE8HYdYTRHXEaRJxvILS35aPNpNQjNz5SzabL D0stU5wh3ASOWONJxARfm2LI2XZreeq87P61EFX4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Kim Phillips , "Peter Zijlstra (Intel)" , "Arnaldo Carvalho de Melo" , x86@kernel.org, Ingo Molnar , Ingo Molnar , Jiri Olsa , Thomas Gleixner , "Borislav Petkov" , Stephane Eranian , Alexander Shishkin , "Namhyung Kim" , "H. Peter Anvin" , Sasha Levin Subject: [PATCH 4.9 64/74] perf/x86/amd/ibs: Fix sample bias for dispatched micro-ops Date: Fri, 20 Sep 2019 00:04:17 +0200 Message-Id: <20190919214810.780897502@linuxfoundation.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190919214800.519074117@linuxfoundation.org> References: <20190919214800.519074117@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Kim Phillips [ Upstream commit 0f4cd769c410e2285a4e9873a684d90423f03090 ] When counting dispatched micro-ops with cnt_ctl=1, in order to prevent sample bias, IBS hardware preloads the least significant 7 bits of current count (IbsOpCurCnt) with random values, such that, after the interrupt is handled and counting resumes, the next sample taken will be slightly perturbed. The current count bitfield is in the IBS execution control h/w register, alongside the maximum count field. Currently, the IBS driver writes that register with the maximum count, leaving zeroes to fill the current count field, thereby overwriting the random bits the hardware preloaded for itself. Fix the driver to actually retain and carry those random bits from the read of the IBS control register, through to its write, instead of overwriting the lower current count bits with zeroes. Tested with: perf record -c 100001 -e ibs_op/cnt_ctl=1/pp -a -C 0 taskset -c 0 'perf annotate' output before: 15.70 65: addsd %xmm0,%xmm1 17.30 add $0x1,%rax 15.88 cmp %rdx,%rax je 82 17.32 72: test $0x1,%al jne 7c 7.52 movapd %xmm1,%xmm0 5.90 jmp 65 8.23 7c: sqrtsd %xmm1,%xmm0 12.15 jmp 65 'perf annotate' output after: 16.63 65: addsd %xmm0,%xmm1 16.82 add $0x1,%rax 16.81 cmp %rdx,%rax je 82 16.69 72: test $0x1,%al jne 7c 8.30 movapd %xmm1,%xmm0 8.13 jmp 65 8.24 7c: sqrtsd %xmm1,%xmm0 8.39 jmp 65 Tested on Family 15h and 17h machines. Machines prior to family 10h Rev. C don't have the RDWROPCNT capability, and have the IbsOpCurCnt bitfield reserved, so this patch shouldn't affect their operation. It is unknown why commit db98c5faf8cb ("perf/x86: Implement 64-bit counter support for IBS") ignored the lower 4 bits of the IbsOpCurCnt field; the number of preloaded random bits has always been 7, AFAICT. Signed-off-by: Kim Phillips Signed-off-by: Peter Zijlstra (Intel) Cc: "Arnaldo Carvalho de Melo" Cc: Cc: Ingo Molnar Cc: Ingo Molnar Cc: Jiri Olsa Cc: Thomas Gleixner Cc: "Borislav Petkov" Cc: Stephane Eranian Cc: Alexander Shishkin Cc: "Namhyung Kim" Cc: "H. Peter Anvin" Link: https://lkml.kernel.org/r/20190826195730.30614-1-kim.phillips@amd.com Signed-off-by: Sasha Levin --- arch/x86/events/amd/ibs.c | 13 ++++++++++--- arch/x86/include/asm/perf_event.h | 12 ++++++++---- 2 files changed, 18 insertions(+), 7 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index fd4484ae3ffca..112e3c4636b4f 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -671,10 +671,17 @@ fail: throttle = perf_event_overflow(event, &data, ®s); out: - if (throttle) + if (throttle) { perf_ibs_stop(event, 0); - else - perf_ibs_enable_event(perf_ibs, hwc, period >> 4); + } else { + period >>= 4; + + if ((ibs_caps & IBS_CAPS_RDWROPCNT) && + (*config & IBS_OP_CNT_CTL)) + period |= *config & IBS_OP_CUR_CNT_RAND; + + perf_ibs_enable_event(perf_ibs, hwc, period); + } perf_event_update_userpage(event); diff --git a/arch/x86/include/asm/perf_event.h b/arch/x86/include/asm/perf_event.h index f353061bba1d0..81d5ea71bbe94 100644 --- a/arch/x86/include/asm/perf_event.h +++ b/arch/x86/include/asm/perf_event.h @@ -200,16 +200,20 @@ struct x86_pmu_capability { #define IBSCTL_LVT_OFFSET_VALID (1ULL<<8) #define IBSCTL_LVT_OFFSET_MASK 0x0F -/* ibs fetch bits/masks */ +/* IBS fetch bits/masks */ #define IBS_FETCH_RAND_EN (1ULL<<57) #define IBS_FETCH_VAL (1ULL<<49) #define IBS_FETCH_ENABLE (1ULL<<48) #define IBS_FETCH_CNT 0xFFFF0000ULL #define IBS_FETCH_MAX_CNT 0x0000FFFFULL -/* ibs op bits/masks */ -/* lower 4 bits of the current count are ignored: */ -#define IBS_OP_CUR_CNT (0xFFFF0ULL<<32) +/* + * IBS op bits/masks + * The lower 7 bits of the current count are random bits + * preloaded by hardware and ignored in software + */ +#define IBS_OP_CUR_CNT (0xFFF80ULL<<32) +#define IBS_OP_CUR_CNT_RAND (0x0007FULL<<32) #define IBS_OP_CNT_CTL (1ULL<<19) #define IBS_OP_VAL (1ULL<<18) #define IBS_OP_ENABLE (1ULL<<17) -- 2.20.1