From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1756730AbcGYWsf (ORCPT <rfc822;w@1wt.eu>);
	Mon, 25 Jul 2016 18:48:35 -0400
Received: from mail-yw0-f177.google.com ([209.85.161.177]:35404 "EHLO
	mail-yw0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1754288AbcGYWr5 (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Mon, 25 Jul 2016 18:47:57 -0400
MIME-Version: 1.0
In-Reply-To: <20160725180531.GA4116@intel.com>
References: <1468371785-53231-1-git-send-email-fenghua.yu@intel.com>
 <1468371785-53231-6-git-send-email-fenghua.yu@intel.com> <CACbG308=AXHTnM187Jd=J=oG3Uwo3ek8AXsFYLuN7p8MB4VZ8A@mail.gmail.com>
 <20160725180531.GA4116@intel.com>
From: David Carrillo-Cisneros <davidcc@google.com>
Date: Mon, 25 Jul 2016 15:47:54 -0700
Message-ID: <CALcN6mid=SBG9DL-ZYW5OZNUuo8D9fXGnxwqgkQdD-R_XuPKKw@mail.gmail.com>
Subject: Re: [PATCH 05/32] x86/intel_rdt: Implement scheduling support for
 Intel RDT
To: "Luck, Tony" <tony.luck@intel.com>, Nilay Vaish <nilayvaish@gmail.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>, Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@elte.hu>,
        "H. Peter Anvin" <h.peter.anvin@intel.com>, Tejun Heo <tj@kernel.org>,
        Borislav Petkov <bp@suse.de>, Stephane Eranian <eranian@google.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Marcelo Tosatti <mtosatti@redhat.com>,
        Ravi V Shankar <ravi.v.shankar@intel.com>,
        Vikas Shivappa <vikas.shivappa@linux.intel.com>,
        Sai Prakhya <sai.praneeth.prakhya@intel.com>,
        linux-kernel <linux-kernel@vger.kernel.org>, x86 <x86@kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Jul 25, 2016 at 11:05 AM Luck, Tony <tony.luck@intel.com> wrote:
>
> On Mon, Jul 25, 2016 at 11:31:24AM -0500, Nilay Vaish wrote:
> > I was thinking more about this software caching of CLOSids.  How
> > likely do you think these CLOSids would be found cached?  I think the
> > software cache would be very infrequently accessed, so it seems you
> > are likely to miss these in all levels of cache hierarchy and more
> > likely to have to fetch these from the main memory, which itself might
> > cost ~250 cycles.
>
> We need to avoid reading the PQR_ASSOC MSR (which would cost far
> more than 250 cycles).  Life is complicated here because this
> MSR contains the CLOSID in the upper half, and the RMID (owned
> by the perf code to measure cache occupancy and memory bandwidth)
> in the lower half.


On my Haswell machine, writing PQR_ASSOC_MSR takes about 380 cycles.
As Tony said, CQM/CMT writes to the same register, and it does it
twice (once to delete the old event, once to add the new one).

So, if a CQM/CMT or MBM is used with CAT, there will be 3 writes to
PQR_ASSOC_MSR per context switch and it's quite likely that the
software cache's cache line will be there for the 2 last writes.