From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43149C4332F for ; Wed, 16 Nov 2022 13:20:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233321AbiKPNUp (ORCPT ); Wed, 16 Nov 2022 08:20:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233652AbiKPNUc (ORCPT ); Wed, 16 Nov 2022 08:20:32 -0500 Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com [IPv6:2607:f8b0:4864:20::112f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91FDD27FF7 for ; Wed, 16 Nov 2022 05:20:31 -0800 (PST) Received: by mail-yw1-x112f.google.com with SMTP id 00721157ae682-367b8adf788so167186747b3.2 for ; Wed, 16 Nov 2022 05:20:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=vxKHoXY+iMzMLFmzEA3zKKda4HHbNhaI8yIydro4NOQ=; b=lhFPWDuRoDRCeijubbdr/Y9pvTp9yAsFihpcTQlnUEYTDf8vu6z8gYbHkTM7meeaH/ 7EMbbRuywzmQE/nT/gLSGWv0kgd/YWRH9teiIlKWBMMYbWgTcBg0Hocg3GAym/msyvxc C9OZ4q55GJIH/324+/gJNnbYsEfTEo5/ut5ehVJvbw9u2l+FfFkSQ5h+5hmIkie7VeqQ 912s3/1G/1qN/MalciigbCPrPAm4aPxxr9n80jtc95Upjj3V4KaNOwemtfk66ZNrHgxK y5CEhvDJhmnu7C/Ib9uvnLsbPf3ZE0cfB6qAGpPVNIHZJxOclz4f7tYq/pmC+RNTxip1 t3WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vxKHoXY+iMzMLFmzEA3zKKda4HHbNhaI8yIydro4NOQ=; b=reR2/6PB7X24cSOanh+0ImzkZTu250L6r8lsChkbo1UpQL/yVasXw6Q0XWvPudYZXp gf8zjMtiK0qDM12bo1qpNupZeVHOUqfP1AX66OELuCz28Dz4qvk09QHJBhjiidbilK66 NNR7fEwMzb1wBDd+2RGNBAXNc3I2o1MNz4SRz/o5x4Om7VFzhFcxIvfW/AChSTbgw581 b9XghEMmOC7WjdGXJwkd1eqCWh0RHhV9bBOdWczu2s021odB2BWjupjZN4tHujTUgsCz yGqWCvqW82/mCliFnkhxZe5ORcieJTnybUVZC4MivGtvWpczDBfdnSyZLe5AdXgZfv4G zCyA== X-Gm-Message-State: ANoB5pnGYgaWTDwvK7CivLYCtCIu6YkJW08NLc1xMXZ+X6jUp6whR3G8 jPOWvESSNkGuoO1CZp8XWf3o5vhd84CG+/hEByT8PQ== X-Google-Smtp-Source: AA0mqf7fxhZX2Ti4VYrXYUNql55200Xi4FUdmmZSj+5XeSmW8WdRRiZEAc2N8Mc/qQYApnU5S48ByzdhifO2l9mfc5E= X-Received: by 2002:a05:690c:29e:b0:38b:fa10:cc71 with SMTP id bf30-20020a05690c029e00b0038bfa10cc71mr1519614ywb.185.1668604830743; Wed, 16 Nov 2022 05:20:30 -0800 (PST) MIME-Version: 1.0 References: <81a7b4f6-fbb5-380e-532d-f2c1fc49b515@intel.com> <76bb4dc9-ab7c-4cb6-d1bf-26436c88c6e2@arm.com> <835d769b-3662-7be5-dcdd-804cb1f3999a@arm.com> <09029c7a-489a-7054-1ab5-01fa879fb42f@intel.com> <39fe80dc-713d-9ed2-a5aa-5c84376917f3@arm.com> In-Reply-To: <39fe80dc-713d-9ed2-a5aa-5c84376917f3@arm.com> From: Peter Newman Date: Wed, 16 Nov 2022 14:20:19 +0100 Message-ID: Subject: Re: [RFD] resctrl: reassigning a running container's CTRL_MON group To: James Morse Cc: Reinette Chatre , Tony Luck , "Yu, Fenghua" , "Eranian, Stephane" , "linux-kernel@vger.kernel.org" , Thomas Gleixner , Babu Moger , Gaurang Upasani Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi James, On Fri, Nov 11, 2022 at 7:38 PM James Morse wrote: > On 09/11/2022 19:11, Reinette Chatre wrote: > > On 11/9/2022 1:50 AM, Peter Newman wrote: > >> Using this we can permanently pin RMIDs to CPUs and read the > >> counters on every task switch to implement MBM RMIDs in software. > > >> This has the caveats that evictions while one task is running could have > >> resulted from a previous task on the current CPU, but will be counted > >> against the new task's software-RMID, and that CMT doesn't work. > > (Sounds like the best thing to do in a bad situation) > > > >> I will propose making this available as a mount option for cloud container > >> use cases which need to monitor a large number of tasks on B/W counter-poor > >> systems, and of course don't need CMT. > > Why does it need to be a mount option? > > If this is the only way of using the counters on this platform, then the skid from the > counters is just a property of the platform. It can be advertised to user-space via some > file in 'info'. No, it's not the only way of using the counters. The limitation is much more problematic for users who monitor all tasks all the time. RMIDs would be more likely to remain in use on systems that only monitor select tasks, so they should be able to benefit from skid-free bandwidth counts and CMT, so I think the proposed mode should be opt-in. > Architecture specific mount options are a bad idea, platform specific ones are even worse! It is already the case today in resctrlfs that the platform's features will determine which mount options are available to the user. I believe the same implementation would work on Intel platforms, but it would just be silly to enable when these platforms have enough counters to back all their RMIDs. Also I believe it's fine for this option to be missing on MPAM until someone is interested in paying the tradeoffs to monitor more groups and is motivated enough to implement it. -Peter