[PATCH][RFC] x86/intel_rdt: Do not display size for non-CAT resource

[PATCH][RFC] x86/intel_rdt: Do not display size for non-CAT resource

[Chen Yu]

On a platform with MB resource enabled, a divided-by-zero
exception is triggered when accessing 'size':

[  151.193447] divide error: 0000 [#1] SMP PTI
[  151.197743] CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
[  151.205070] Hardware name: Dell Inc. PowerEdge R640/0CRT1G, BIOS 1.3.7 02/08/2018
[  151.212783] RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
[  151.237172] RSP: 0018:ffffb3454f90bd88 EFLAGS: 00010246
[  151.242538] RAX: 00000000023c0000 RBX: 0000000000000000 RCX: 0000000000000003
[  151.249878] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000003
[  151.257213] RBP: ffff96ff0089e000 R08: 0000000000000000 R09: 0000000000aaaaaa
[  151.264544] R10: ffffb3454f90bd8c R11: 00000000ffffffff R12: ffffffffb5028910
[  151.271887] R13: ffffffffb5028910 R14: 0000000000000064 R15: ffff96ff0089e000
[  151.279217] FS:  00007f95a623a500(0000) GS:ffff97170f9c0000(0000) knlGS:0000000000000000
[  151.287532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  151.293432] CR2: 00007f95a6217000 CR3: 00000023f696c003 CR4: 00000000007606e0
[  151.300766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  151.308094] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  151.315426] PKRU: 55555554
[  151.318212] Call Trace:
[  151.320732]  rdtgroup_size_show+0x11a/0x1d0
[  151.325039]  seq_read+0xd8/0x3b0
[  151.328363]  __vfs_read+0x36/0x170
[  151.331857]  vfs_read+0x89/0x130
[  151.335179]  ksys_read+0x52/0xc0
[  151.338500]  do_syscall_64+0x5b/0x180
[  151.342261]  entry_SYSCALL_64_after_hwframe+0x44/0xa9

This is because for MB resource, the r->cache.cbm_len is zero, thus
calculating size in rdtgroup_cbm_to_size() will trigger the exception.

Fix this issue by not exposing 'size' for non-CAT resources.

Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups'
allocations in bytes")
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b799c00bef09..53fd07b2f61a 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -1329,7 +1329,7 @@ static struct rftype res_common_files[] = {
 		.mode		= 0444,
 		.kf_ops		= &rdtgroup_kf_single_ops,
 		.seq_show	= rdtgroup_size_show,
-		.fflags		= RF_CTRL_BASE,
+		.fflags		= RF_CTRL_INFO | RFTYPE_RES_CACHE,
 	},
 
 };
-- 
2.17.1

Re: [PATCH][RFC] x86/intel_rdt: Do not display size for non-CAT resource

[Reinette Chatre]

Hi Chen Yu,

On 9/4/2018 10:46 AM, Chen Yu wrote:
> On a platform with MB resource enabled, a divided-by-zero
> exception is triggered when accessing 'size':
> 
> [  151.193447] divide error: 0000 [#1] SMP PTI
> [  151.197743] CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
> [  151.205070] Hardware name: Dell Inc. PowerEdge R640/0CRT1G, BIOS 1.3.7 02/08/2018
> [  151.212783] RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
> [  151.237172] RSP: 0018:ffffb3454f90bd88 EFLAGS: 00010246
> [  151.242538] RAX: 00000000023c0000 RBX: 0000000000000000 RCX: 0000000000000003
> [  151.249878] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000003
> [  151.257213] RBP: ffff96ff0089e000 R08: 0000000000000000 R09: 0000000000aaaaaa
> [  151.264544] R10: ffffb3454f90bd8c R11: 00000000ffffffff R12: ffffffffb5028910
> [  151.271887] R13: ffffffffb5028910 R14: 0000000000000064 R15: ffff96ff0089e000
> [  151.279217] FS:  00007f95a623a500(0000) GS:ffff97170f9c0000(0000) knlGS:0000000000000000
> [  151.287532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  151.293432] CR2: 00007f95a6217000 CR3: 00000023f696c003 CR4: 00000000007606e0
> [  151.300766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [  151.308094] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [  151.315426] PKRU: 55555554
> [  151.318212] Call Trace:
> [  151.320732]  rdtgroup_size_show+0x11a/0x1d0
> [  151.325039]  seq_read+0xd8/0x3b0
> [  151.328363]  __vfs_read+0x36/0x170
> [  151.331857]  vfs_read+0x89/0x130
> [  151.335179]  ksys_read+0x52/0xc0
> [  151.338500]  do_syscall_64+0x5b/0x180
> [  151.342261]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> 
> This is because for MB resource, the r->cache.cbm_len is zero, thus
> calculating size in rdtgroup_cbm_to_size() will trigger the exception.
> 
> Fix this issue by not exposing 'size' for non-CAT resources.
> 
> Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups'
> allocations in bytes")
> Cc: Reinette Chatre <reinette.chatre@intel.com>
> Cc: Fenghua Yu <fenghua.yu@intel.com>
> Cc: Tony Luck <tony.luck@intel.com>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Chen Yu <yu.c.chen@intel.com>
> ---
>  arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> index b799c00bef09..53fd07b2f61a 100644
> --- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> @@ -1329,7 +1329,7 @@ static struct rftype res_common_files[] = {
>  		.mode		= 0444,
>  		.kf_ops		= &rdtgroup_kf_single_ops,
>  		.seq_show	= rdtgroup_size_show,
> -		.fflags		= RF_CTRL_BASE,
> +		.fflags		= RF_CTRL_INFO | RFTYPE_RES_CACHE,
>  	},
>  
>  };
> 

Thank you very much for catching this.

I think we need to change the fix a bit because from that I can tell the
above would cause the "size" file to be relocated to the system wide
"info" directory while we would like to have this file remain associated
with the resource group - but just not apply to a MB resource.

A similar fix may also be needed for the resource group's "mode" file
that was also recently introduced.

I am taking a closer look now.

Reinette

Re: [PATCH][RFC] x86/intel_rdt: Do not display size for non-CAT resource

[Reinette Chatre]

Hi Chen Yu,

On 9/4/2018 1:24 PM, Reinette Chatre wrote:
> On 9/4/2018 10:46 AM, Chen Yu wrote:
>> On a platform with MB resource enabled, a divided-by-zero
>> exception is triggered when accessing 'size':
>>
>> [  151.193447] divide error: 0000 [#1] SMP PTI
>> [  151.197743] CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
>> [  151.205070] Hardware name: Dell Inc. PowerEdge R640/0CRT1G, BIOS 1.3.7 02/08/2018
>> [  151.212783] RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
>> [  151.237172] RSP: 0018:ffffb3454f90bd88 EFLAGS: 00010246
>> [  151.242538] RAX: 00000000023c0000 RBX: 0000000000000000 RCX: 0000000000000003
>> [  151.249878] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000003
>> [  151.257213] RBP: ffff96ff0089e000 R08: 0000000000000000 R09: 0000000000aaaaaa
>> [  151.264544] R10: ffffb3454f90bd8c R11: 00000000ffffffff R12: ffffffffb5028910
>> [  151.271887] R13: ffffffffb5028910 R14: 0000000000000064 R15: ffff96ff0089e000
>> [  151.279217] FS:  00007f95a623a500(0000) GS:ffff97170f9c0000(0000) knlGS:0000000000000000
>> [  151.287532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [  151.293432] CR2: 00007f95a6217000 CR3: 00000023f696c003 CR4: 00000000007606e0
>> [  151.300766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [  151.308094] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>> [  151.315426] PKRU: 55555554
>> [  151.318212] Call Trace:
>> [  151.320732]  rdtgroup_size_show+0x11a/0x1d0
>> [  151.325039]  seq_read+0xd8/0x3b0
>> [  151.328363]  __vfs_read+0x36/0x170
>> [  151.331857]  vfs_read+0x89/0x130
>> [  151.335179]  ksys_read+0x52/0xc0
>> [  151.338500]  do_syscall_64+0x5b/0x180
>> [  151.342261]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>
>> This is because for MB resource, the r->cache.cbm_len is zero, thus
>> calculating size in rdtgroup_cbm_to_size() will trigger the exception.
>>
>> Fix this issue by not exposing 'size' for non-CAT resources.
>>
>> Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups'
>> allocations in bytes")
>> Cc: Reinette Chatre <reinette.chatre@intel.com>
>> Cc: Fenghua Yu <fenghua.yu@intel.com>
>> Cc: Tony Luck <tony.luck@intel.com>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Signed-off-by: Chen Yu <yu.c.chen@intel.com>
>> ---
>>  arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
>> index b799c00bef09..53fd07b2f61a 100644
>> --- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
>> @@ -1329,7 +1329,7 @@ static struct rftype res_common_files[] = {
>>  		.mode		= 0444,
>>  		.kf_ops		= &rdtgroup_kf_single_ops,
>>  		.seq_show	= rdtgroup_size_show,
>> -		.fflags		= RF_CTRL_BASE,
>> +		.fflags		= RF_CTRL_INFO | RFTYPE_RES_CACHE,
>>  	},
>>  
>>  };
>>
> 
> Thank you very much for catching this.
> 
> I think we need to change the fix a bit because from that I can tell the
> above would cause the "size" file to be relocated to the system wide
> "info" directory while we would like to have this file remain associated
> with the resource group - but just not apply to a MB resource.
> 
> A similar fix may also be needed for the resource group's "mode" file
> that was also recently introduced.
> 
> I am taking a closer look now.

The "size" file is intended to be associated with a resource group and
to list the size in bytes of the cache allocations. It does not
currently accommodate the memory bandwidth allocations as you
discovered. A system may have multiple resources to be managed via RDT,
it could include cache as well as memory, and to thus not expose the
"size" file if memory bandwidth allocation is supported is not ideal
since the user would not be able to see this information for the cache
resources.

So, instead of not exposing the "size" file when memory bandwidth
allocation is in use I think that we could just include the memory
bandwidth allocation information in the existing file. This would be in
the currently active bandwidth granularity that would essentially
duplicate the schemata information.

While looking further at how the new files (size and mode) will behave
when a MBA resource is present I think I discovered a few more issues:
- the "exclusive" mode should not apply to a MBA resource
- it should not be possible to pseudo-lock a MBA resource

I attempt to address the above issues with the change below. Could you
please try it out with what you are currently testing? I do not have
access to a system with a MBA resource - could you please let me know
what system you are testing on so I can try out more tests?

Thanks!

Reinette


-->8----
diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
index af358ca05160..434dd93f915a 100644
--- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
+++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
@@ -200,6 +200,12 @@ static int parse_line(char *line, struct
rdt_resource *r,
        struct rdt_domain *d;
        unsigned long dom_id;

+       if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
+           r->rid == RDT_RESOURCE_MBA) {
+               rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
+               return -EINVAL;
+       }
+
 next:
        if (!line || line[0] == '\0')
                return 0;
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b799c00bef09..2bc4a01536bc 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -1027,6 +1027,8 @@ static bool rdtgroup_mode_test_exclusive(struct
rdtgroup *rdtgrp)
        struct rdt_domain *d;

        for_each_alloc_enabled_rdt_resource(r) {
+               if (r->rid == RDT_RESOURCE_MBA)
+                       continue;
                list_for_each_entry(d, &r->domains, list) {
                        if (rdtgroup_cbm_overlaps(r, d, d->ctrl_val[closid],
                                                  rdtgrp->closid, false))
@@ -1156,7 +1158,7 @@ static int rdtgroup_size_show(struct
kernfs_open_file *of,
        struct rdt_domain *d;
        unsigned int size;
        bool sep = false;
-       u32 cbm;
+       u32 ctrl;

        rdtgrp = rdtgroup_kn_lock_live(of->kn);
        if (!rdtgrp) {
@@ -1181,8 +1183,13 @@ static int rdtgroup_size_show(struct
kernfs_open_file *of,
                        if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
                                size = 0;
                        } else {
-                               cbm = d->ctrl_val[rdtgrp->closid];
-                               size = rdtgroup_cbm_to_size(r, d, cbm);
+                               ctrl = (!is_mba_sc(r) ?
+
d->ctrl_val[rdtgrp->closid] :
+
d->mbps_val[rdtgrp->closid]);
+                               if (r->rid == RDT_RESOURCE_MBA)
+                                       size = ctrl;
+                               else
+                                       size = rdtgroup_cbm_to_size(r,
d, ctrl);
                        }
                        seq_printf(s, "%d=%u", d->id, size);
                        sep = true;

Re: [PATCH][RFC] x86/intel_rdt: Do not display size for non-CAT resource

[Yu Chen]

Hi Reinette,
Thanks for looking at this.
On Tue, Sep 04, 2018 at 03:36:01PM -0700, Reinette Chatre wrote:
> Hi Chen Yu,
> 
> On 9/4/2018 1:24 PM, Reinette Chatre wrote:
> > On 9/4/2018 10:46 AM, Chen Yu wrote:
> >> On a platform with MB resource enabled, a divided-by-zero
> >> exception is triggered when accessing 'size':
> >>
> >> [  151.193447] divide error: 0000 [#1] SMP PTI
> >> [  151.197743] CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
> >> [  151.205070] Hardware name: Dell Inc. PowerEdge R640/0CRT1G, BIOS 1.3.7 02/08/2018
> >> [  151.212783] RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
> >> [  151.237172] RSP: 0018:ffffb3454f90bd88 EFLAGS: 00010246
> >> [  151.242538] RAX: 00000000023c0000 RBX: 0000000000000000 RCX: 0000000000000003
> >> [  151.249878] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000003
> >> [  151.257213] RBP: ffff96ff0089e000 R08: 0000000000000000 R09: 0000000000aaaaaa
> >> [  151.264544] R10: ffffb3454f90bd8c R11: 00000000ffffffff R12: ffffffffb5028910
> >> [  151.271887] R13: ffffffffb5028910 R14: 0000000000000064 R15: ffff96ff0089e000
> >> [  151.279217] FS:  00007f95a623a500(0000) GS:ffff97170f9c0000(0000) knlGS:0000000000000000
> >> [  151.287532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >> [  151.293432] CR2: 00007f95a6217000 CR3: 00000023f696c003 CR4: 00000000007606e0
> >> [  151.300766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >> [  151.308094] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> >> [  151.315426] PKRU: 55555554
> >> [  151.318212] Call Trace:
> >> [  151.320732]  rdtgroup_size_show+0x11a/0x1d0
> >> [  151.325039]  seq_read+0xd8/0x3b0
> >> [  151.328363]  __vfs_read+0x36/0x170
> >> [  151.331857]  vfs_read+0x89/0x130
> >> [  151.335179]  ksys_read+0x52/0xc0
> >> [  151.338500]  do_syscall_64+0x5b/0x180
> >> [  151.342261]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >>
> >> This is because for MB resource, the r->cache.cbm_len is zero, thus
> >> calculating size in rdtgroup_cbm_to_size() will trigger the exception.
> >>
> >> Fix this issue by not exposing 'size' for non-CAT resources.
> >>
> >> Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups'
> >> allocations in bytes")
> >> Cc: Reinette Chatre <reinette.chatre@intel.com>
> >> Cc: Fenghua Yu <fenghua.yu@intel.com>
> >> Cc: Tony Luck <tony.luck@intel.com>
> >> Cc: Thomas Gleixner <tglx@linutronix.de>
> >> Signed-off-by: Chen Yu <yu.c.chen@intel.com>
> >> ---
> >>  arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> >> index b799c00bef09..53fd07b2f61a 100644
> >> --- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> >> +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> >> @@ -1329,7 +1329,7 @@ static struct rftype res_common_files[] = {
> >>  		.mode		= 0444,
> >>  		.kf_ops		= &rdtgroup_kf_single_ops,
> >>  		.seq_show	= rdtgroup_size_show,
> >> -		.fflags		= RF_CTRL_BASE,
> >> +		.fflags		= RF_CTRL_INFO | RFTYPE_RES_CACHE,
> >>  	},
> >>  
> >>  };
> >>
> > 
> > Thank you very much for catching this.
> > 
> > I think we need to change the fix a bit because from that I can tell the
> > above would cause the "size" file to be relocated to the system wide
> > "info" directory 
Okay. Just curious, if changing like this, will it be moved to resctrl/info/L3/size or
resctrl/info/size?
>> while we would like to have this file remain associated
> > with the resource group - but just not apply to a MB resource.
> > 
> > A similar fix may also be needed for the resource group's "mode" file
> > that was also recently introduced.
> > 
> > I am taking a closer look now.
> 
> The "size" file is intended to be associated with a resource group and
> to list the size in bytes of the cache allocations. It does not
> currently accommodate the memory bandwidth allocations as you
> discovered. A system may have multiple resources to be managed via RDT,
> it could include cache as well as memory, and to thus not expose the
> "size" file if memory bandwidth allocation is supported is not ideal
> since the user would not be able to see this information for the cache
> resources.
> 
> So, instead of not exposing the "size" file when memory bandwidth
> allocation is in use I think that we could just include the memory
> bandwidth allocation information in the existing file. This would be in
> the currently active bandwidth granularity that would essentially
> duplicate the schemata information.
> 
I might understand incorrectly, but in this way, the size in resctrl top
dir will display all the domain ranges within all the resources, say, the
size for MB, L3 will be displayed in one file, right? Will the 'size' be
displayed under each resource dir in info dir?
> While looking further at how the new files (size and mode) will behave
> when a MBA resource is present I think I discovered a few more issues:
> - the "exclusive" mode should not apply to a MBA resource
> - it should not be possible to pseudo-lock a MBA resource
> 
> I attempt to address the above issues with the change below. Could you
> please try it out with what you are currently testing?
This patch works.
Tested-by: Chen Yu <yu.c.chen@intel.com>
> I do not have
> access to a system with a MBA resource - could you please let me know
> what system you are testing on so I can try out more tests?
> 
I'm using SKYLAKE-X, of cpu stepping 4, so l3cat might be disabled
due to errorta.

BTW, may I know the scope of CBM? It seems that in this patch all
the other resource than MBA could leverage CBM to calculate their
resource size. What if other resources are added in the future?

Best,
Yu
> Thanks!
> 
> Reinette
> 
> 
> -->8----
> diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
> b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
> index af358ca05160..434dd93f915a 100644
> --- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
> +++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
> @@ -200,6 +200,12 @@ static int parse_line(char *line, struct
> rdt_resource *r,
>         struct rdt_domain *d;
>         unsigned long dom_id;
> 
> +       if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
> +           r->rid == RDT_RESOURCE_MBA) {
> +               rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
> +               return -EINVAL;
> +       }
> +
>  next:
>         if (!line || line[0] == '\0')
>                 return 0;
> diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> index b799c00bef09..2bc4a01536bc 100644
> --- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
> @@ -1027,6 +1027,8 @@ static bool rdtgroup_mode_test_exclusive(struct
> rdtgroup *rdtgrp)
>         struct rdt_domain *d;
> 
>         for_each_alloc_enabled_rdt_resource(r) {
> +               if (r->rid == RDT_RESOURCE_MBA)
> +                       continue;
>                 list_for_each_entry(d, &r->domains, list) {
>                         if (rdtgroup_cbm_overlaps(r, d, d->ctrl_val[closid],
>                                                   rdtgrp->closid, false))
> @@ -1156,7 +1158,7 @@ static int rdtgroup_size_show(struct
> kernfs_open_file *of,
>         struct rdt_domain *d;
>         unsigned int size;
>         bool sep = false;
> -       u32 cbm;
> +       u32 ctrl;
> 
>         rdtgrp = rdtgroup_kn_lock_live(of->kn);
>         if (!rdtgrp) {
> @@ -1181,8 +1183,13 @@ static int rdtgroup_size_show(struct
> kernfs_open_file *of,
>                         if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
>                                 size = 0;
>                         } else {
> -                               cbm = d->ctrl_val[rdtgrp->closid];
> -                               size = rdtgroup_cbm_to_size(r, d, cbm);
> +                               ctrl = (!is_mba_sc(r) ?
> +
> d->ctrl_val[rdtgrp->closid] :
> +
> d->mbps_val[rdtgrp->closid]);
> +                               if (r->rid == RDT_RESOURCE_MBA)
> +                                       size = ctrl;
> +                               else
> +                                       size = rdtgroup_cbm_to_size(r,
> d, ctrl);
>                         }
>                         seq_printf(s, "%d=%u", d->id, size);
>                         sep = true;
> 

Re: [PATCH][RFC] x86/intel_rdt: Do not display size for non-CAT resource

[Reinette Chatre]

Hi Yu,

On 9/4/2018 11:28 PM, Yu Chen wrote:
> On Tue, Sep 04, 2018 at 03:36:01PM -0700, Reinette Chatre wrote:
>> On 9/4/2018 1:24 PM, Reinette Chatre wrote:
>>> On 9/4/2018 10:46 AM, Chen Yu wrote:
>>>> On a platform with MB resource enabled, a divided-by-zero
>>>> exception is triggered when accessing 'size':
>>>>
>>>> [  151.193447] divide error: 0000 [#1] SMP PTI
>>>> [  151.197743] CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
>>>> [  151.205070] Hardware name: Dell Inc. PowerEdge R640/0CRT1G, BIOS 1.3.7 02/08/2018
>>>> [  151.212783] RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
>>>> [  151.237172] RSP: 0018:ffffb3454f90bd88 EFLAGS: 00010246
>>>> [  151.242538] RAX: 00000000023c0000 RBX: 0000000000000000 RCX: 0000000000000003
>>>> [  151.249878] RDX: 0000000000000000 RSI: 0000000000000004 RDI: 0000000000000003
>>>> [  151.257213] RBP: ffff96ff0089e000 R08: 0000000000000000 R09: 0000000000aaaaaa
>>>> [  151.264544] R10: ffffb3454f90bd8c R11: 00000000ffffffff R12: ffffffffb5028910
>>>> [  151.271887] R13: ffffffffb5028910 R14: 0000000000000064 R15: ffff96ff0089e000
>>>> [  151.279217] FS:  00007f95a623a500(0000) GS:ffff97170f9c0000(0000) knlGS:0000000000000000
>>>> [  151.287532] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [  151.293432] CR2: 00007f95a6217000 CR3: 00000023f696c003 CR4: 00000000007606e0
>>>> [  151.300766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>> [  151.308094] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>>> [  151.315426] PKRU: 55555554
>>>> [  151.318212] Call Trace:
>>>> [  151.320732]  rdtgroup_size_show+0x11a/0x1d0
>>>> [  151.325039]  seq_read+0xd8/0x3b0
>>>> [  151.328363]  __vfs_read+0x36/0x170
>>>> [  151.331857]  vfs_read+0x89/0x130
>>>> [  151.335179]  ksys_read+0x52/0xc0
>>>> [  151.338500]  do_syscall_64+0x5b/0x180
>>>> [  151.342261]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>
>>>> This is because for MB resource, the r->cache.cbm_len is zero, thus
>>>> calculating size in rdtgroup_cbm_to_size() will trigger the exception.
>>>>
>>>> Fix this issue by not exposing 'size' for non-CAT resources.
>>>>
>>>> Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups'
>>>> allocations in bytes")
>>>> Cc: Reinette Chatre <reinette.chatre@intel.com>
>>>> Cc: Fenghua Yu <fenghua.yu@intel.com>
>>>> Cc: Tony Luck <tony.luck@intel.com>
>>>> Cc: Thomas Gleixner <tglx@linutronix.de>
>>>> Signed-off-by: Chen Yu <yu.c.chen@intel.com>
>>>> ---
>>>>  arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
>>>> index b799c00bef09..53fd07b2f61a 100644
>>>> --- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
>>>> +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
>>>> @@ -1329,7 +1329,7 @@ static struct rftype res_common_files[] = {
>>>>  		.mode		= 0444,
>>>>  		.kf_ops		= &rdtgroup_kf_single_ops,
>>>>  		.seq_show	= rdtgroup_size_show,
>>>> -		.fflags		= RF_CTRL_BASE,
>>>> +		.fflags		= RF_CTRL_INFO | RFTYPE_RES_CACHE,
>>>>  	},
>>>>  
>>>>  };
>>>>
>>>
>>> Thank you very much for catching this.
>>>
>>> I think we need to change the fix a bit because from that I can tell the
>>> above would cause the "size" file to be relocated to the system wide
>>> "info" directory 
> Okay. Just curious, if changing like this, will it be moved to resctrl/info/L3/size or
> resctrl/info/size?

This change should attempt to create this file in all cache resources
within the info subdirectory. So resctrl/info/L3/size,
resctrl/info/L2/size, etc.

Take care though that the file handling routine (rdtgroup_size_show())
assumes that it is associated with a resource group.

>>> while we would like to have this file remain associated
>>> with the resource group - but just not apply to a MB resource.
>>>
>>> A similar fix may also be needed for the resource group's "mode" file
>>> that was also recently introduced.
>>>
>>> I am taking a closer look now.
>>
>> The "size" file is intended to be associated with a resource group and
>> to list the size in bytes of the cache allocations. It does not
>> currently accommodate the memory bandwidth allocations as you
>> discovered. A system may have multiple resources to be managed via RDT,
>> it could include cache as well as memory, and to thus not expose the
>> "size" file if memory bandwidth allocation is supported is not ideal
>> since the user would not be able to see this information for the cache
>> resources.
>>
>> So, instead of not exposing the "size" file when memory bandwidth
>> allocation is in use I think that we could just include the memory
>> bandwidth allocation information in the existing file. This would be in
>> the currently active bandwidth granularity that would essentially
>> duplicate the schemata information.
>>
> I might understand incorrectly, but in this way, the size in resctrl top
> dir will display all the domain ranges within all the resources, say, the
> size for MB, L3 will be displayed in one file, right? Will the 'size' be
> displayed under each resource dir in info dir?

Yes (answering your first question), the size file is intended to
reflect the allocations of all resources associated with the associated
resource group. It does follow the display of the schemata file in this
regard and is indeed a different visualization of the same content - for
cache it is size in bytes instead of a bitmask. The top resctrl
directory is the default resource group but a user could create more
resource groups with "mkdir" in the top directory.

>> While looking further at how the new files (size and mode) will behave
>> when a MBA resource is present I think I discovered a few more issues:
>> - the "exclusive" mode should not apply to a MBA resource
>> - it should not be possible to pseudo-lock a MBA resource
>>
>> I attempt to address the above issues with the change below. Could you
>> please try it out with what you are currently testing?
> This patch works.
> Tested-by: Chen Yu <yu.c.chen@intel.com>

Thank you very much for trying it out.

>> I do not have
>> access to a system with a MBA resource - could you please let me know
>> what system you are testing on so I can try out more tests?
>>
> I'm using SKYLAKE-X, of cpu stepping 4, so l3cat might be disabled
> due to errorta.

Thank you. I now have access to a similar system and will test these
changes more before resubmitting.


> BTW, may I know the scope of CBM? It seems that in this patch all
> the other resource than MBA could leverage CBM to calculate their
> resource size. What if other resources are added in the future?

Yes, at this time I am only aware of the MBA control interface that does
not use a CBM and it is distinguished specifically because of this.
Indeed, if other non-CBM allocation/control resources are added in the
future we may consider to abstract how (if at all - like MBA) the size
is computed.

Reinette

[PATCH V2 0/9] x86/intel_rdt: MBA integration fixes

[Fenghua Yu]

Chen Yu reported an issue where reading the resctrl "size" file results in
a divide-by-zero issue on a system with a MBA resource. Further
investigation revealed more issues where the recent RDT features are not
well integrated with the MBA resource handling.

This series consists out of:
 - One helper function in support of fixes that need to know the number of
   supported CLOSIDs on the system (the minimum of all CLOSIDs of all
   resources).
 - The fix to the issue reported by Chen Yu  - now reading a resource
   group's "size" file will show a MB resource's allocation as its size.
 - A fix from Xiaochen Shen to the MB parsing callback that was never
   changed to accept a new parameter format.
 - Functions that iterate over the number of CLOSIDs need to take care
   whether it is using a particular resource's supported CLOSIDs or the
   number of CLOSIDs supported by the system. This was done incorrectly in
   a few places and fixed here.
 - When a new resource group is created it is intended to be configured
   with sane defaults. This new feature blindly assumed that the resource
   group only consists out of cache resources - make this explicit to not
   cause invalid register writes on a system with MBA resources.
 - The new "exclusive" mode assumes that all resources support a CBM, while
   a MBA resource does not. Since the MBA resource allocations cannot be
   done in a way to specify whether allocations can overlap or not the
   "exclusive" mode of a resource group will only apply to the cache
   resources within the group, if only a MBA resource is present then
   "exclusive" mode will not be allowed.

Changelog
v2:
 - Patch 1: Change 'data' from void pointer to struct rdt_parse_data
   pointer in parsing callbacks based on suggestion from Thomas.
 - Patch 2, 4, 5: Condense backtraces in commit description to minimum
   based on suggestion from Thomas

Reinette Chatre (8):
  x86/intel_rdt: Fix size reporting of MBA resource
  x86/intel_rdt: Global closid helper to support future fixes
  x86/intel_rdt: Fix invalid mode warning when multiple resources are
    managed
  x86/intel_rdt: Fix unchecked MSR access
  x86/intel_rdt: Do not allow pseudo-locking of MBA resource
  x86/intel_rdt: Fix incorrect loop end condition
  x86/intel_rdt: Fix exclusive mode handling of MBA resource
  x86/intel_rdt: Fix incorrect loop end condition

Xiaochen Shen (1):
  x86/intel_rdt: Fix data type in parsing callbacks

 arch/x86/kernel/cpu/intel_rdt.h             | 17 +++++--
 arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 27 ++++++-----
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c    | 53 +++++++++++++++++----
 3 files changed, 71 insertions(+), 26 deletions(-)

-- 
2.19.0

[PATCH V2 1/9] x86/intel_rdt: Fix data type in parsing callbacks

[Fenghua Yu]

From: Xiaochen Shen <xiaochen.shen@intel.com>

Each resource is associated with a parsing callback to parse the data
provided from user space when writing schemata file.

The 'data' parameter in the callbacks is defined as a void pointer which
is error prone due to lack of type check.

parse_bw() processes the 'data' parameter as a string while its caller
actually passes the parameter as a pointer to struct rdt_cbm_parse_data.
Thus, parse_bw() takes wrong data and causes failure of parsing MBA
throttle value.

To fix the issue, the 'data' parameter in the parsing callbacks is
defined and handled as a pointer to struct rdt_parse_data (renamed from
struct rdt_cbm_parse_data).

Fixes: 7604df6e16ae ("x86/intel_rdt: Support flexible data to parsing callbacks")
Fixes: 9ab9aa15c309 ("x86/intel_rdt: Ensure requested schemata respects mode")

Signed-off-by: Xiaochen Shen <xiaochen.shen@intel.com>
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt.h             | 16 ++++++++++++----
 arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 21 ++++++++-------------
 2 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt.h b/arch/x86/kernel/cpu/intel_rdt.h
index 4e588f36228f..78266c798280 100644
--- a/arch/x86/kernel/cpu/intel_rdt.h
+++ b/arch/x86/kernel/cpu/intel_rdt.h
@@ -382,6 +382,11 @@ static inline bool is_mbm_event(int e)
 		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
 }
 
+struct rdt_parse_data {
+	struct rdtgroup		*rdtgrp;
+	char			*buf;
+};
+
 /**
  * struct rdt_resource - attributes of an RDT resource
  * @rid:		The index of the resource
@@ -423,16 +428,19 @@ struct rdt_resource {
 	struct rdt_cache	cache;
 	struct rdt_membw	membw;
 	const char		*format_str;
-	int (*parse_ctrlval)	(void *data, struct rdt_resource *r,
-				 struct rdt_domain *d);
+	int (*parse_ctrlval)(struct rdt_parse_data *data,
+			     struct rdt_resource *r,
+			     struct rdt_domain *d);
 	struct list_head	evt_list;
 	int			num_rmid;
 	unsigned int		mon_scale;
 	unsigned long		fflags;
 };
 
-int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d);
-int parse_bw(void *_buf, struct rdt_resource *r,  struct rdt_domain *d);
+int parse_cbm(struct rdt_parse_data *data, struct rdt_resource *r,
+	      struct rdt_domain *d);
+int parse_bw(struct rdt_parse_data *data, struct rdt_resource *r,
+	     struct rdt_domain *d);
 
 extern struct mutex rdtgroup_mutex;
 
diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
index af358ca05160..edd5761f7336 100644
--- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
+++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
@@ -64,19 +64,19 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
 	return true;
 }
 
-int parse_bw(void *_buf, struct rdt_resource *r, struct rdt_domain *d)
+int parse_bw(struct rdt_parse_data *data, struct rdt_resource *r,
+	     struct rdt_domain *d)
 {
-	unsigned long data;
-	char *buf = _buf;
+	unsigned long bw_val;
 
 	if (d->have_new_ctrl) {
 		rdt_last_cmd_printf("duplicate domain %d\n", d->id);
 		return -EINVAL;
 	}
 
-	if (!bw_validate(buf, &data, r))
+	if (!bw_validate(data->buf, &bw_val, r))
 		return -EINVAL;
-	d->new_ctrl = data;
+	d->new_ctrl = bw_val;
 	d->have_new_ctrl = true;
 
 	return 0;
@@ -123,18 +123,13 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
 	return true;
 }
 
-struct rdt_cbm_parse_data {
-	struct rdtgroup		*rdtgrp;
-	char			*buf;
-};
-
 /*
  * Read one cache bit mask (hex). Check that it is valid for the current
  * resource type.
  */
-int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d)
+int parse_cbm(struct rdt_parse_data *data, struct rdt_resource *r,
+	      struct rdt_domain *d)
 {
-	struct rdt_cbm_parse_data *data = _data;
 	struct rdtgroup *rdtgrp = data->rdtgrp;
 	u32 cbm_val;
 
@@ -195,7 +190,7 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d)
 static int parse_line(char *line, struct rdt_resource *r,
 		      struct rdtgroup *rdtgrp)
 {
-	struct rdt_cbm_parse_data data;
+	struct rdt_parse_data data;
 	char *dom = NULL, *id;
 	struct rdt_domain *d;
 	unsigned long dom_id;
-- 
2.19.0

[PATCH V2 2/9] x86/intel_rdt: Fix size reporting of MBA resource

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

Chen Yu reported a divide-by-zero error when accessing the 'size'
resctrl file when a MBA resource is enabled.

divide error: 0000 [#1] SMP PTI
CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
Call Trace:
rdtgroup_size_show+0x11a/0x1d0
seq_read+0xd8/0x3b0

Quoting Chen Yu's report: This is because for MB resource,
the r->cache.cbm_len is zero, thus calculating size in
rdtgroup_cbm_to_size() will trigger the exception.

Fix this issue in the 'size' file by getting correct memory bandwidth
value which is in MBps when MBA software controller is enabled or in
percentage when MBA software controller is disabled.

Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups' allocations in bytes")
Link: https://lkml.kernel.org/r/20180904174614.26682-1-yu.c.chen@intel.com
Reported-by: Chen Yu <yu.c.chen@intel.com>
Tested-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b799c00bef09..32e8bbdf2400 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -1155,8 +1155,8 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 	struct rdt_resource *r;
 	struct rdt_domain *d;
 	unsigned int size;
-	bool sep = false;
-	u32 cbm;
+	bool sep;
+	u32 ctrl;
 
 	rdtgrp = rdtgroup_kn_lock_live(of->kn);
 	if (!rdtgrp) {
@@ -1174,6 +1174,7 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 	}
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		sep = false;
 		seq_printf(s, "%*s:", max_name_width, r->name);
 		list_for_each_entry(d, &r->domains, list) {
 			if (sep)
@@ -1181,8 +1182,13 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 			if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
 				size = 0;
 			} else {
-				cbm = d->ctrl_val[rdtgrp->closid];
-				size = rdtgroup_cbm_to_size(r, d, cbm);
+				ctrl = (!is_mba_sc(r) ?
+						d->ctrl_val[rdtgrp->closid] :
+						d->mbps_val[rdtgrp->closid]);
+				if (r->rid == RDT_RESOURCE_MBA)
+					size = ctrl;
+				else
+					size = rdtgroup_cbm_to_size(r, d, ctrl);
 			}
 			seq_printf(s, "%d=%u", d->id, size);
 			sep = true;
-- 
2.19.0

[PATCH V2 3/9] x86/intel_rdt: Global closid helper to support future fixes

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

The number of CLOSIDs supported by a system is the minimum number of
CLOSIDs supported by any of its resources. Care should be taken when
iterating over the CLOSIDs of a resource since it may be that the number
of CLOSIDs supported on the system is less than the number of CLOSIDs
supported by the resource.

Introduce a helper function that can be used to query the number of
CLOSIDs that is supported by all resources, irrespective of how many
CLOSIDs are supported by a particular resource.

Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt.h          | 1 +
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt.h b/arch/x86/kernel/cpu/intel_rdt.h
index 78266c798280..285eb3ec4200 100644
--- a/arch/x86/kernel/cpu/intel_rdt.h
+++ b/arch/x86/kernel/cpu/intel_rdt.h
@@ -544,6 +544,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
 void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
 struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);
 int update_domains(struct rdt_resource *r, int closid);
+int closids_supported(void);
 void closid_free(int closid);
 int alloc_rmid(void);
 void free_rmid(u32 rmid);
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 32e8bbdf2400..b372923eb209 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -97,6 +97,12 @@ void rdt_last_cmd_printf(const char *fmt, ...)
  *   limited as the number of resources grows.
  */
 static int closid_free_map;
+static int closid_free_map_len;
+
+int closids_supported(void)
+{
+	return closid_free_map_len;
+}
 
 static void closid_init(void)
 {
@@ -111,6 +117,7 @@ static void closid_init(void)
 
 	/* CLOSID 0 is always reserved for the default group */
 	closid_free_map &= ~1;
+	closid_free_map_len = rdt_min_closid;
 }
 
 static int closid_alloc(void)
-- 
2.19.0

[PATCH V2 4/9] x86/intel_rdt: Fix invalid mode warning when multiple resources are managed

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

When multiple resources are managed by RDT, the number of CLOSIDs used
is the minimum of the CLOSIDs supported by each resource. In the function
rdt_bit_usage_show(), the annotated bitmask is created to depict how the
CAT supporting caches are being used. During this annotated bitmask
creation, each resource group is queried for its mode that is used as a
label in the annotated bitmask.

The maximum number of resource groups is currently assumed to be the
number of CLOSIDs supported by the resource for which the information is
being displayed. This is incorrect since the number of active CLOSIDs is
the minimum across all resources.

If information for a cache instance with more CLOSIDs than another is
being generated we thus encounter a warning like:

invalid mode for closid 8
WARNING: CPU: 88 PID: 1791 at [SNIP]/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
:827 rdt_bit_usage_show+0x221/0x2b0

Fix this by ensuring that only the number of supported CLOSIDs are
considered.

Fixes: e651901187ab8 ("x86/intel_rdt: Introduce "bit_usage" to display cache allocations details")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b372923eb209..ea91750ba27f 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -809,7 +809,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,
 		sw_shareable = 0;
 		exclusive = 0;
 		seq_printf(seq, "%d=", dom->id);
-		for (i = 0; i < r->num_closid; i++, ctrl++) {
+		for (i = 0; i < closids_supported(); i++, ctrl++) {
 			if (!closid_allocated(i))
 				continue;
 			mode = rdtgroup_mode_by_closid(i);
-- 
2.19.0

[PATCH V2 5/9] x86/intel_rdt: Fix unchecked MSR access

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

When a new resource group is created, it is initialized with sane
defaults that currently assume the resource being initialized is a CAT
resource. This code path is also followed by a MBA resource that is not
allocated the same as a CAT resource and as a result we encounter the
following unchecked MSR access error:

unchecked MSR access error: WRMSR to 0xd51 (tried to write 0x0000
000000000064) at rIP: 0xffffffffae059994 (native_write_msr+0x4/0x20)
Call Trace:
mba_wrmsr+0x41/0x80
update_domains+0x125/0x130
rdtgroup_mkdir+0x270/0x500

Fix the above by ensuring the initial allocation is only attempted on a
CAT resource.

Fixes: 95f0b77ef ("x86/intel_rdt: Initialize new resource group with sane defaults")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index ea91750ba27f..74821bc457c0 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -2349,6 +2349,12 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
 	u32 *ctrl;
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		/*
+		 * Only initialize default allocations for CBM cache
+		 * resources
+		 */
+		if (r->rid == RDT_RESOURCE_MBA)
+			continue;
 		list_for_each_entry(d, &r->domains, list) {
 			d->have_new_ctrl = false;
 			d->new_ctrl = r->cache.shareable_bits;
@@ -2386,6 +2392,12 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
 	}
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		/*
+		 * Only initialize default allocations for CBM cache
+		 * resources
+		 */
+		if (r->rid == RDT_RESOURCE_MBA)
+			continue;
 		ret = update_domains(r, rdtgrp->closid);
 		if (ret < 0) {
 			rdt_last_cmd_puts("failed to initialize allocations\n");
-- 
2.19.0

[PATCH V2 6/9] x86/intel_rdt: Do not allow pseudo-locking of MBA resource

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

A system supporting pseudo-locking may have MBA as well as CAT
resources of which only the CAT resources could support cache
pseudo-locking. When the schemata to be pseudo-locked is provided it
should be checked that that schemata does not attempt to pseudo-lock a
MBA resource.

Fixes: e0bdfe8e3 ("x86/intel_rdt: Support creation/removal of pseudo-locked region")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
index edd5761f7336..0f53049719cd 100644
--- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
+++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
@@ -195,6 +195,12 @@ static int parse_line(char *line, struct rdt_resource *r,
 	struct rdt_domain *d;
 	unsigned long dom_id;
 
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
+	    r->rid == RDT_RESOURCE_MBA) {
+		rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
+		return -EINVAL;
+	}
+
 next:
 	if (!line || line[0] == '\0')
 		return 0;
-- 
2.19.0

[PATCH V2 7/9] x86/intel_rdt: Fix incorrect loop end condition

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

A loop is used to check if a CAT resource's CBM of one CLOSID
overlaps with the CBM of another CLOSID of the same resource. The loop
is run over all CLOSIDs supported by the resource.

The problem with running the loop over all CLOSIDs supported by the
resource is that its number of supported CLOSIDs may be more than the
number of supported CLOSIDs on the system, which is the minimum number of
CLOSIDs supported across all resources.

Fix the loop to only consider the number of system supported CLOSIDs,
not all that are supported by the resource.

Fixes: 49f7b4efa ("x86/intel_rdt: Enable setting of exclusive mode")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 74821bc457c0..afd93d45e21b 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -996,7 +996,7 @@ bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
 
 	/* Check for overlap with other resource groups */
 	ctrl = d->ctrl_val;
-	for (i = 0; i < r->num_closid; i++, ctrl++) {
+	for (i = 0; i < closids_supported(); i++, ctrl++) {
 		ctrl_b = (unsigned long *)ctrl;
 		mode = rdtgroup_mode_by_closid(i);
 		if (closid_allocated(i) && i != closid &&
-- 
2.19.0

[PATCH V2 8/9] x86/intel_rdt: Fix exclusive mode handling of MBA resource

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

It is possible for a resource group to consist out of MBA as well as
CAT/CDP resources. The "exclusive" resource mode only applies to the
CAT/CDP resources since MBA allocations cannot be specified to overlap
or not. When a user requests a resource group to become "exclusive" then it
can only be successful if there are CAT/CDP resources in the group
and none of their CBMs associated with the group's CLOSID overlaps with
any other resource group.

Fix the "exclusive" mode setting by failing if there isn't any CAT/CDP
resource in the group and ensuring that the CBM checking is only done on
CAT/CDP resources.

Fixes: 49f7b4efa ("x86/intel_rdt: Enable setting of exclusive mode")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index afd93d45e21b..f3231f78d69b 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -1031,16 +1031,27 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
 {
 	int closid = rdtgrp->closid;
 	struct rdt_resource *r;
+	bool has_cache = false;
 	struct rdt_domain *d;
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		if (r->rid == RDT_RESOURCE_MBA)
+			continue;
+		has_cache = true;
 		list_for_each_entry(d, &r->domains, list) {
 			if (rdtgroup_cbm_overlaps(r, d, d->ctrl_val[closid],
-						  rdtgrp->closid, false))
+						  rdtgrp->closid, false)) {
+				rdt_last_cmd_puts("schemata overlaps\n");
 				return false;
+			}
 		}
 	}
 
+	if (!has_cache) {
+		rdt_last_cmd_puts("cannot be exclusive without CAT/CDP\n");
+		return false;
+	}
+
 	return true;
 }
 
@@ -1092,7 +1103,6 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
 		rdtgrp->mode = RDT_MODE_SHAREABLE;
 	} else if (!strcmp(buf, "exclusive")) {
 		if (!rdtgroup_mode_test_exclusive(rdtgrp)) {
-			rdt_last_cmd_printf("schemata overlaps\n");
 			ret = -EINVAL;
 			goto out;
 		}
-- 
2.19.0

[PATCH V2 9/9] x86/intel_rdt: Fix incorrect loop end condition

[Fenghua Yu]

From: Reinette Chatre <reinette.chatre@intel.com>

In order to determine a sane default cache allocation for a new CAT/CDP
resource group, all resource groups are checked to determine which cache
portions are available to share. At this time all possible CLOSIDs
that can be supported by the resource is checked. This is problematic
if the resource supports more CLOSIDs than another CAT/CDP resource. In
this case, the number of CLOSIDs that could be allocated are fewer than
the number of CLOSIDs that can be supported by the resource.

Limit the check of closids to that what is supported by the system based
on the minimum across all resources.

Fixes: 95f0b77ef ("x86/intel_rdt: Initialize new resource group with sane defaults")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index f3231f78d69b..1b8e86a5d5e1 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -2370,7 +2370,7 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
 			d->new_ctrl = r->cache.shareable_bits;
 			used_b = r->cache.shareable_bits;
 			ctrl = d->ctrl_val;
-			for (i = 0; i < r->num_closid; i++, ctrl++) {
+			for (i = 0; i < closids_supported(); i++, ctrl++) {
 				if (closid_allocated(i) && i != closid) {
 					mode = rdtgroup_mode_by_closid(i);
 					if (mode == RDT_MODE_PSEUDO_LOCKSETUP)
-- 
2.19.0

[tip:x86/urgent] x86/intel_rdt: Fix data type in parsing callbacks

[tip-bot for Xiaochen Shen]

Commit-ID:  753694a8df318f204a0ac1303de136def16f2e9c
Gitweb:     https://git.kernel.org/tip/753694a8df318f204a0ac1303de136def16f2e9c
Author:     Xiaochen Shen <xiaochen.shen@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:19 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:05 +0200

x86/intel_rdt: Fix data type in parsing callbacks

Each resource is associated with a parsing callback to parse the data
provided from user space when writing schemata file.

The 'data' parameter in the callbacks is defined as a void pointer which
is error prone due to lack of type check.

parse_bw() processes the 'data' parameter as a string while its caller
actually passes the parameter as a pointer to struct rdt_cbm_parse_data.
Thus, parse_bw() takes wrong data and causes failure of parsing MBA
throttle value.

To fix the issue, the 'data' parameter in all parsing callbacks is defined
and handled as a pointer to struct rdt_parse_data (renamed from struct
rdt_cbm_parse_data).

Fixes: 7604df6e16ae ("x86/intel_rdt: Support flexible data to parsing callbacks")
Fixes: 9ab9aa15c309 ("x86/intel_rdt: Ensure requested schemata respects mode")
Signed-off-by: Xiaochen Shen <xiaochen.shen@intel.com>
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-2-git-send-email-fenghua.yu@intel.com


---
 arch/x86/kernel/cpu/intel_rdt.h             | 16 ++++++++++++----
 arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 21 ++++++++-------------
 2 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt.h b/arch/x86/kernel/cpu/intel_rdt.h
index 4e588f36228f..78266c798280 100644
--- a/arch/x86/kernel/cpu/intel_rdt.h
+++ b/arch/x86/kernel/cpu/intel_rdt.h
@@ -382,6 +382,11 @@ static inline bool is_mbm_event(int e)
 		e <= QOS_L3_MBM_LOCAL_EVENT_ID);
 }
 
+struct rdt_parse_data {
+	struct rdtgroup		*rdtgrp;
+	char			*buf;
+};
+
 /**
  * struct rdt_resource - attributes of an RDT resource
  * @rid:		The index of the resource
@@ -423,16 +428,19 @@ struct rdt_resource {
 	struct rdt_cache	cache;
 	struct rdt_membw	membw;
 	const char		*format_str;
-	int (*parse_ctrlval)	(void *data, struct rdt_resource *r,
-				 struct rdt_domain *d);
+	int (*parse_ctrlval)(struct rdt_parse_data *data,
+			     struct rdt_resource *r,
+			     struct rdt_domain *d);
 	struct list_head	evt_list;
 	int			num_rmid;
 	unsigned int		mon_scale;
 	unsigned long		fflags;
 };
 
-int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d);
-int parse_bw(void *_buf, struct rdt_resource *r,  struct rdt_domain *d);
+int parse_cbm(struct rdt_parse_data *data, struct rdt_resource *r,
+	      struct rdt_domain *d);
+int parse_bw(struct rdt_parse_data *data, struct rdt_resource *r,
+	     struct rdt_domain *d);
 
 extern struct mutex rdtgroup_mutex;
 
diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
index af358ca05160..edd5761f7336 100644
--- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
+++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
@@ -64,19 +64,19 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r)
 	return true;
 }
 
-int parse_bw(void *_buf, struct rdt_resource *r, struct rdt_domain *d)
+int parse_bw(struct rdt_parse_data *data, struct rdt_resource *r,
+	     struct rdt_domain *d)
 {
-	unsigned long data;
-	char *buf = _buf;
+	unsigned long bw_val;
 
 	if (d->have_new_ctrl) {
 		rdt_last_cmd_printf("duplicate domain %d\n", d->id);
 		return -EINVAL;
 	}
 
-	if (!bw_validate(buf, &data, r))
+	if (!bw_validate(data->buf, &bw_val, r))
 		return -EINVAL;
-	d->new_ctrl = data;
+	d->new_ctrl = bw_val;
 	d->have_new_ctrl = true;
 
 	return 0;
@@ -123,18 +123,13 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r)
 	return true;
 }
 
-struct rdt_cbm_parse_data {
-	struct rdtgroup		*rdtgrp;
-	char			*buf;
-};
-
 /*
  * Read one cache bit mask (hex). Check that it is valid for the current
  * resource type.
  */
-int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d)
+int parse_cbm(struct rdt_parse_data *data, struct rdt_resource *r,
+	      struct rdt_domain *d)
 {
-	struct rdt_cbm_parse_data *data = _data;
 	struct rdtgroup *rdtgrp = data->rdtgrp;
 	u32 cbm_val;
 
@@ -195,7 +190,7 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d)
 static int parse_line(char *line, struct rdt_resource *r,
 		      struct rdtgroup *rdtgrp)
 {
-	struct rdt_cbm_parse_data data;
+	struct rdt_parse_data data;
 	char *dom = NULL, *id;
 	struct rdt_domain *d;
 	unsigned long dom_id;

[tip:x86/urgent] x86/intel_rdt: Fix size reporting of MBA resource

[tip-bot for Reinette Chatre]

Commit-ID:  f968dc119a159a95628a20de2a2dcc913d0a82d7
Gitweb:     https://git.kernel.org/tip/f968dc119a159a95628a20de2a2dcc913d0a82d7
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:20 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:05 +0200

x86/intel_rdt: Fix size reporting of MBA resource

Chen Yu reported a divide-by-zero error when accessing the 'size'
resctrl file when a MBA resource is enabled.

divide error: 0000 [#1] SMP PTI
CPU: 93 PID: 1929 Comm: cat Not tainted 4.19.0-rc2-debug-rdt+ #25
RIP: 0010:rdtgroup_cbm_to_size+0x7e/0xa0
Call Trace:
rdtgroup_size_show+0x11a/0x1d0
seq_read+0xd8/0x3b0

Quoting Chen Yu's report: This is because for MB resource, the
r->cache.cbm_len is zero, thus calculating size in rdtgroup_cbm_to_size()
will trigger the exception.

Fix this issue in the 'size' file by getting correct memory bandwidth value
which is in MBps when MBA software controller is enabled or in percentage
when MBA software controller is disabled.

Fixes: d9b48c86eb38 ("x86/intel_rdt: Display resource groups' allocations in bytes")
Reported-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Chen Yu <yu.c.chen@intel.com>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Link: https://lkml.kernel.org/r/20180904174614.26682-1-yu.c.chen@intel.com
Link: https://lkml.kernel.org/r/1537048707-76280-3-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b799c00bef09..32e8bbdf2400 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -1155,8 +1155,8 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 	struct rdt_resource *r;
 	struct rdt_domain *d;
 	unsigned int size;
-	bool sep = false;
-	u32 cbm;
+	bool sep;
+	u32 ctrl;
 
 	rdtgrp = rdtgroup_kn_lock_live(of->kn);
 	if (!rdtgrp) {
@@ -1174,6 +1174,7 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 	}
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		sep = false;
 		seq_printf(s, "%*s:", max_name_width, r->name);
 		list_for_each_entry(d, &r->domains, list) {
 			if (sep)
@@ -1181,8 +1182,13 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,
 			if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) {
 				size = 0;
 			} else {
-				cbm = d->ctrl_val[rdtgrp->closid];
-				size = rdtgroup_cbm_to_size(r, d, cbm);
+				ctrl = (!is_mba_sc(r) ?
+						d->ctrl_val[rdtgrp->closid] :
+						d->mbps_val[rdtgrp->closid]);
+				if (r->rid == RDT_RESOURCE_MBA)
+					size = ctrl;
+				else
+					size = rdtgroup_cbm_to_size(r, d, ctrl);
 			}
 			seq_printf(s, "%d=%u", d->id, size);
 			sep = true;

[tip:x86/urgent] x86/intel_rdt: Global closid helper to support future fixes

[tip-bot for Reinette Chatre]

Commit-ID:  c793da8e4c62d2c002a79c47f44efead450cbcae
Gitweb:     https://git.kernel.org/tip/c793da8e4c62d2c002a79c47f44efead450cbcae
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:21 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:05 +0200

x86/intel_rdt: Global closid helper to support future fixes

The number of CLOSIDs supported by a system is the minimum number of
CLOSIDs supported by any of its resources. Care should be taken when
iterating over the CLOSIDs of a resource since it may be that the number
of CLOSIDs supported on the system is less than the number of CLOSIDs
supported by the resource.

Introduce a helper function that can be used to query the number of
CLOSIDs that is supported by all resources, irrespective of how many
CLOSIDs are supported by a particular resource.

Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-4-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt.h          | 1 +
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt.h b/arch/x86/kernel/cpu/intel_rdt.h
index 78266c798280..285eb3ec4200 100644
--- a/arch/x86/kernel/cpu/intel_rdt.h
+++ b/arch/x86/kernel/cpu/intel_rdt.h
@@ -544,6 +544,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp);
 void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp);
 struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r);
 int update_domains(struct rdt_resource *r, int closid);
+int closids_supported(void);
 void closid_free(int closid);
 int alloc_rmid(void);
 void free_rmid(u32 rmid);
diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 32e8bbdf2400..b372923eb209 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -97,6 +97,12 @@ void rdt_last_cmd_printf(const char *fmt, ...)
  *   limited as the number of resources grows.
  */
 static int closid_free_map;
+static int closid_free_map_len;
+
+int closids_supported(void)
+{
+	return closid_free_map_len;
+}
 
 static void closid_init(void)
 {
@@ -111,6 +117,7 @@ static void closid_init(void)
 
 	/* CLOSID 0 is always reserved for the default group */
 	closid_free_map &= ~1;
+	closid_free_map_len = rdt_min_closid;
 }
 
 static int closid_alloc(void)

[tip:x86/urgent] x86/intel_rdt: Fix invalid mode warning when multiple resources are managed

[tip-bot for Reinette Chatre]

Commit-ID:  47d53b184aee983ab9492503da11b0a81b19145b
Gitweb:     https://git.kernel.org/tip/47d53b184aee983ab9492503da11b0a81b19145b
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:22 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:05 +0200

x86/intel_rdt: Fix invalid mode warning when multiple resources are managed

When multiple resources are managed by RDT, the number of CLOSIDs used
is the minimum of the CLOSIDs supported by each resource. In the function
rdt_bit_usage_show(), the annotated bitmask is created to depict how the
CAT supporting caches are being used. During this annotated bitmask
creation, each resource group is queried for its mode that is used as a
label in the annotated bitmask.

The maximum number of resource groups is currently assumed to be the
number of CLOSIDs supported by the resource for which the information is
being displayed. This is incorrect since the number of active CLOSIDs is
the minimum across all resources.

If information for a cache instance with more CLOSIDs than another is
being generated we thus encounter a warning like:

invalid mode for closid 8
WARNING: CPU: 88 PID: 1791 at [SNIP]/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
:827 rdt_bit_usage_show+0x221/0x2b0

Fix this by ensuring that only the number of supported CLOSIDs are
considered.

Fixes: e651901187ab8 ("x86/intel_rdt: Introduce "bit_usage" to display cache allocations details")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-5-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index b372923eb209..ea91750ba27f 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -809,7 +809,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,
 		sw_shareable = 0;
 		exclusive = 0;
 		seq_printf(seq, "%d=", dom->id);
-		for (i = 0; i < r->num_closid; i++, ctrl++) {
+		for (i = 0; i < closids_supported(); i++, ctrl++) {
 			if (!closid_allocated(i))
 				continue;
 			mode = rdtgroup_mode_by_closid(i);

[tip:x86/urgent] x86/intel_rdt: Fix unchecked MSR access

[tip-bot for Reinette Chatre]

Commit-ID:  70479c012b67b89e219c40eddc5dc338b7c447a3
Gitweb:     https://git.kernel.org/tip/70479c012b67b89e219c40eddc5dc338b7c447a3
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:23 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:06 +0200

x86/intel_rdt: Fix unchecked MSR access

When a new resource group is created, it is initialized with sane
defaults that currently assume the resource being initialized is a CAT
resource. This code path is also followed by a MBA resource that is not
allocated the same as a CAT resource and as a result we encounter the
following unchecked MSR access error:

unchecked MSR access error: WRMSR to 0xd51 (tried to write 0x0000
000000000064) at rIP: 0xffffffffae059994 (native_write_msr+0x4/0x20)
Call Trace:
mba_wrmsr+0x41/0x80
update_domains+0x125/0x130
rdtgroup_mkdir+0x270/0x500

Fix the above by ensuring the initial allocation is only attempted on a
CAT resource.

Fixes: 95f0b77ef ("x86/intel_rdt: Initialize new resource group with sane defaults")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-6-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index ea91750ba27f..74821bc457c0 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -2349,6 +2349,12 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
 	u32 *ctrl;
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		/*
+		 * Only initialize default allocations for CBM cache
+		 * resources
+		 */
+		if (r->rid == RDT_RESOURCE_MBA)
+			continue;
 		list_for_each_entry(d, &r->domains, list) {
 			d->have_new_ctrl = false;
 			d->new_ctrl = r->cache.shareable_bits;
@@ -2386,6 +2392,12 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
 	}
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		/*
+		 * Only initialize default allocations for CBM cache
+		 * resources
+		 */
+		if (r->rid == RDT_RESOURCE_MBA)
+			continue;
 		ret = update_domains(r, rdtgrp->closid);
 		if (ret < 0) {
 			rdt_last_cmd_puts("failed to initialize allocations\n");

[tip:x86/urgent] x86/intel_rdt: Do not allow pseudo-locking of MBA resource

[tip-bot for Reinette Chatre]

Commit-ID:  32d736abed4febff4b6bf85d5d240ee24d254322
Gitweb:     https://git.kernel.org/tip/32d736abed4febff4b6bf85d5d240ee24d254322
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:24 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:06 +0200

x86/intel_rdt: Do not allow pseudo-locking of MBA resource

A system supporting pseudo-locking may have MBA as well as CAT
resources of which only the CAT resources could support cache
pseudo-locking. When the schemata to be pseudo-locked is provided it
should be checked that that schemata does not attempt to pseudo-lock a
MBA resource.

Fixes: e0bdfe8e3 ("x86/intel_rdt: Support creation/removal of pseudo-locked region")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-7-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
index edd5761f7336..0f53049719cd 100644
--- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
+++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c
@@ -195,6 +195,12 @@ static int parse_line(char *line, struct rdt_resource *r,
 	struct rdt_domain *d;
 	unsigned long dom_id;
 
+	if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP &&
+	    r->rid == RDT_RESOURCE_MBA) {
+		rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n");
+		return -EINVAL;
+	}
+
 next:
 	if (!line || line[0] == '\0')
 		return 0;

[tip:x86/urgent] x86/intel_rdt: Fix incorrect loop end condition

[tip-bot for Reinette Chatre]

Commit-ID:  f0df4e1acf3d721958dcafb2c9c0bdf25189068d
Gitweb:     https://git.kernel.org/tip/f0df4e1acf3d721958dcafb2c9c0bdf25189068d
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:25 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:06 +0200

x86/intel_rdt: Fix incorrect loop end condition

A loop is used to check if a CAT resource's CBM of one CLOSID
overlaps with the CBM of another CLOSID of the same resource. The loop
is run over all CLOSIDs supported by the resource.

The problem with running the loop over all CLOSIDs supported by the
resource is that its number of supported CLOSIDs may be more than the
number of supported CLOSIDs on the system, which is the minimum number of
CLOSIDs supported across all resources.

Fix the loop to only consider the number of system supported CLOSIDs,
not all that are supported by the resource.

Fixes: 49f7b4efa ("x86/intel_rdt: Enable setting of exclusive mode")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-8-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index 74821bc457c0..afd93d45e21b 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -996,7 +996,7 @@ bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d,
 
 	/* Check for overlap with other resource groups */
 	ctrl = d->ctrl_val;
-	for (i = 0; i < r->num_closid; i++, ctrl++) {
+	for (i = 0; i < closids_supported(); i++, ctrl++) {
 		ctrl_b = (unsigned long *)ctrl;
 		mode = rdtgroup_mode_by_closid(i);
 		if (closid_allocated(i) && i != closid &&

[tip:x86/urgent] x86/intel_rdt: Fix exclusive mode handling of MBA resource

[tip-bot for Reinette Chatre]

Commit-ID:  939b90b20bc87e199b6b53942764b987289b87ce
Gitweb:     https://git.kernel.org/tip/939b90b20bc87e199b6b53942764b987289b87ce
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:26 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:07 +0200

x86/intel_rdt: Fix exclusive mode handling of MBA resource

It is possible for a resource group to consist out of MBA as well as
CAT/CDP resources. The "exclusive" resource mode only applies to the
CAT/CDP resources since MBA allocations cannot be specified to overlap
or not. When a user requests a resource group to become "exclusive" then it
can only be successful if there are CAT/CDP resources in the group
and none of their CBMs associated with the group's CLOSID overlaps with
any other resource group.

Fix the "exclusive" mode setting by failing if there isn't any CAT/CDP
resource in the group and ensuring that the CBM checking is only done on
CAT/CDP resources.

Fixes: 49f7b4efa ("x86/intel_rdt: Enable setting of exclusive mode")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-9-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index afd93d45e21b..f3231f78d69b 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -1031,16 +1031,27 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp)
 {
 	int closid = rdtgrp->closid;
 	struct rdt_resource *r;
+	bool has_cache = false;
 	struct rdt_domain *d;
 
 	for_each_alloc_enabled_rdt_resource(r) {
+		if (r->rid == RDT_RESOURCE_MBA)
+			continue;
+		has_cache = true;
 		list_for_each_entry(d, &r->domains, list) {
 			if (rdtgroup_cbm_overlaps(r, d, d->ctrl_val[closid],
-						  rdtgrp->closid, false))
+						  rdtgrp->closid, false)) {
+				rdt_last_cmd_puts("schemata overlaps\n");
 				return false;
+			}
 		}
 	}
 
+	if (!has_cache) {
+		rdt_last_cmd_puts("cannot be exclusive without CAT/CDP\n");
+		return false;
+	}
+
 	return true;
 }
 
@@ -1092,7 +1103,6 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of,
 		rdtgrp->mode = RDT_MODE_SHAREABLE;
 	} else if (!strcmp(buf, "exclusive")) {
 		if (!rdtgroup_mode_test_exclusive(rdtgrp)) {
-			rdt_last_cmd_printf("schemata overlaps\n");
 			ret = -EINVAL;
 			goto out;
 		}

[tip:x86/urgent] x86/intel_rdt: Fix incorrect loop end condition

[tip-bot for Reinette Chatre]

Commit-ID:  ffb2315fd22c2568747402eecdc581a245a2f5ba
Gitweb:     https://git.kernel.org/tip/ffb2315fd22c2568747402eecdc581a245a2f5ba
Author:     Reinette Chatre <reinette.chatre@intel.com>
AuthorDate: Sat, 15 Sep 2018 14:58:27 -0700
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 18 Sep 2018 23:38:07 +0200

x86/intel_rdt: Fix incorrect loop end condition

In order to determine a sane default cache allocation for a new CAT/CDP
resource group, all resource groups are checked to determine which cache
portions are available to share. At this time all possible CLOSIDs
that can be supported by the resource is checked. This is problematic
if the resource supports more CLOSIDs than another CAT/CDP resource. In
this case, the number of CLOSIDs that could be allocated are fewer than
the number of CLOSIDs that can be supported by the resource.

Limit the check of closids to that what is supported by the system based
on the minimum across all resources.

Fixes: 95f0b77ef ("x86/intel_rdt: Initialize new resource group with sane defaults")
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "H Peter Anvin" <hpa@zytor.com>
Cc: "Tony Luck" <tony.luck@intel.com>
Cc: "Xiaochen Shen" <xiaochen.shen@intel.com>
Cc: "Chen Yu" <yu.c.chen@intel.com>
Link: https://lkml.kernel.org/r/1537048707-76280-10-git-send-email-fenghua.yu@intel.com

---
 arch/x86/kernel/cpu/intel_rdt_rdtgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
index f3231f78d69b..1b8e86a5d5e1 100644
--- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
+++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c
@@ -2370,7 +2370,7 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp)
 			d->new_ctrl = r->cache.shareable_bits;
 			used_b = r->cache.shareable_bits;
 			ctrl = d->ctrl_val;
-			for (i = 0; i < r->num_closid; i++, ctrl++) {
+			for (i = 0; i < closids_supported(); i++, ctrl++) {
 				if (closid_allocated(i) && i != closid) {
 					mode = rdtgroup_mode_by_closid(i);
 					if (mode == RDT_MODE_PSEUDO_LOCKSETUP)