what the meaning of the per cpu variable cpu_core_mask and cpu_sibling_mask?

what the meaning of the per cpu variable cpu_core_mask and cpu_sibling_mask?

[Demon King]

[-- Attachment #1.1: Type: text/plain, Size: 815 bytes --]

I am a new comer of xen-devel. When I read the source code of
sched_credit.c. I was confused by such codes below:

if ( cpumask_test_cpu(cpu, per_cpu(cpu_core_mask, nxt)) )
        {
            /* We're on the same socket, so check the busy-ness of threads.
             * Migrate if # of idlers is less at all */
            ASSERT( cpumask_test_cpu(nxt, per_cpu(cpu_core_mask, cpu)) );
            migrate_factor = 1;
            cpumask_and(&cpu_idlers, &idlers, per_cpu(cpu_sibling_mask,
cpu));
            cpumask_and(&nxt_idlers, &idlers, per_cpu(cpu_sibling_mask,
nxt));
        }

I don't know much about xen or linux. So I get confused in the per cpu
variable: cpu_core_mask and cpu_sibling_mask
Is anyone willing to tell me? Thank you! And may I ask you where to get
answers about this kind of question?

[-- Attachment #1.2: Type: text/html, Size: 1019 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: what the meaning of the per cpu variable cpu_core_mask and cpu_sibling_mask?

[George Dunlap]

On Thu, Mar 21, 2013 at 9:53 AM, Demon King <kdmxen@gmail.com> wrote:
> I am a new comer of xen-devel. When I read the source code of
> sched_credit.c. I was confused by such codes below:
>
> if ( cpumask_test_cpu(cpu, per_cpu(cpu_core_mask, nxt)) )
>         {
>             /* We're on the same socket, so check the busy-ness of threads.
>              * Migrate if # of idlers is less at all */
>             ASSERT( cpumask_test_cpu(nxt, per_cpu(cpu_core_mask, cpu)) );
>             migrate_factor = 1;
>             cpumask_and(&cpu_idlers, &idlers, per_cpu(cpu_sibling_mask,
> cpu));
>             cpumask_and(&nxt_idlers, &idlers, per_cpu(cpu_sibling_mask,
> nxt));
>         }
>
> I don't know much about xen or linux. So I get confused in the per cpu
> variable: cpu_core_mask and cpu_sibling_mask
> Is anyone willing to tell me? Thank you! And may I ask you where to get
> answers about this kind of question?

A handy tool for looking through the hypervisor source code is as follows:

$ find . -name "*.[cSh]" | xargs grep -H [rexexp]

This will look for [regexp] in every .c, .h, and .S file.

Doing this with "cpu_sibling_mask" turns up a couple of places, one of which is:

xen/arch/x86/smpboot.c

Which has the following comments:

/* representing HT siblings of each logical CPU */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
/* representing HT and core siblings of each logical CPU */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);

So if per_cpu(cpu_core_mask, cpu1) has bit cpu2 set, then cpu2 is a
'core sibling' of cpu1.  Same with cpu_sibling_mask, except then it's
hyperthreads.

To be honest, the code you're looking at now is very complicated and
not very well commented, so it's not surprising you found it difficult
to understand.  I had a plan at some point to update the comments to
make it easier to follow.

 -George

Re: what the meaning of the per cpu variable cpu_core_mask and cpu_sibling_mask?

[Demon King]

[-- Attachment #1.1: Type: text/plain, Size: 2289 bytes --]

Thank you very much !
Thanks to your tools, I now understand how to trace some variable in the
source code.But I still don't know what is  HT siblings or  HT and core
siblings?Could you explain it a little more detail?
Thanks again!


On Thu, Mar 21, 2013 at 8:12 PM, George Dunlap
<George.Dunlap@eu.citrix.com>wrote:

> On Thu, Mar 21, 2013 at 9:53 AM, Demon King <kdmxen@gmail.com> wrote:
> > I am a new comer of xen-devel. When I read the source code of
> > sched_credit.c. I was confused by such codes below:
> >
> > if ( cpumask_test_cpu(cpu, per_cpu(cpu_core_mask, nxt)) )
> >         {
> >             /* We're on the same socket, so check the busy-ness of
> threads.
> >              * Migrate if # of idlers is less at all */
> >             ASSERT( cpumask_test_cpu(nxt, per_cpu(cpu_core_mask, cpu)) );
> >             migrate_factor = 1;
> >             cpumask_and(&cpu_idlers, &idlers, per_cpu(cpu_sibling_mask,
> > cpu));
> >             cpumask_and(&nxt_idlers, &idlers, per_cpu(cpu_sibling_mask,
> > nxt));
> >         }
> >
> > I don't know much about xen or linux. So I get confused in the per cpu
> > variable: cpu_core_mask and cpu_sibling_mask
> > Is anyone willing to tell me? Thank you! And may I ask you where to get
> > answers about this kind of question?
>
> A handy tool for looking through the hypervisor source code is as follows:
>
> $ find . -name "*.[cSh]" | xargs grep -H [rexexp]
>
> This will look for [regexp] in every .c, .h, and .S file.
>
> Doing this with "cpu_sibling_mask" turns up a couple of places, one of
> which is:
>
> xen/arch/x86/smpboot.c
>
> Which has the following comments:
>
> /* representing HT siblings of each logical CPU */
> DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
> /* representing HT and core siblings of each logical CPU */
> DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
>
> So if per_cpu(cpu_core_mask, cpu1) has bit cpu2 set, then cpu2 is a
> 'core sibling' of cpu1.  Same with cpu_sibling_mask, except then it's
> hyperthreads.
>
> To be honest, the code you're looking at now is very complicated and
> not very well commented, so it's not surprising you found it difficult
> to understand.  I had a plan at some point to update the comments to
> make it easier to follow.
>
>  -George
>

[-- Attachment #1.2: Type: text/html, Size: 3433 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: what the meaning of the per cpu variable cpu_core_mask and cpu_sibling_mask?

[George Dunlap]

[-- Attachment #1.1: Type: text/plain, Size: 2755 bytes --]

On 21/03/13 13:49, Demon King wrote:
> Thank you very much !
> Thanks to your tools, I now understand how to trace some variable in 
> the source code.But I still don't know what is HT siblings or HT and 
> core siblings?Could you explain it a little more detail?
> Thanks again!

Please don't top-post.

Hyperthreads (HT) in the same core are siblings, and cores in the same 
socket / numa node are siblings.

  -George

>
>
> On Thu, Mar 21, 2013 at 8:12 PM, George Dunlap 
> <George.Dunlap@eu.citrix.com <mailto:George.Dunlap@eu.citrix.com>> wrote:
>
>     On Thu, Mar 21, 2013 at 9:53 AM, Demon King <kdmxen@gmail.com
>     <mailto:kdmxen@gmail.com>> wrote:
>     > I am a new comer of xen-devel. When I read the source code of
>     > sched_credit.c. I was confused by such codes below:
>     >
>     > if ( cpumask_test_cpu(cpu, per_cpu(cpu_core_mask, nxt)) )
>     >         {
>     >             /* We're on the same socket, so check the busy-ness
>     of threads.
>     >              * Migrate if # of idlers is less at all */
>     >             ASSERT( cpumask_test_cpu(nxt, per_cpu(cpu_core_mask,
>     cpu)) );
>     >             migrate_factor = 1;
>     >             cpumask_and(&cpu_idlers, &idlers,
>     per_cpu(cpu_sibling_mask,
>     > cpu));
>     >             cpumask_and(&nxt_idlers, &idlers,
>     per_cpu(cpu_sibling_mask,
>     > nxt));
>     >         }
>     >
>     > I don't know much about xen or linux. So I get confused in the
>     per cpu
>     > variable: cpu_core_mask and cpu_sibling_mask
>     > Is anyone willing to tell me? Thank you! And may I ask you where
>     to get
>     > answers about this kind of question?
>
>     A handy tool for looking through the hypervisor source code is as
>     follows:
>
>     $ find . -name "*.[cSh]" | xargs grep -H [rexexp]
>
>     This will look for [regexp] in every .c, .h, and .S file.
>
>     Doing this with "cpu_sibling_mask" turns up a couple of places,
>     one of which is:
>
>     xen/arch/x86/smpboot.c
>
>     Which has the following comments:
>
>     /* representing HT siblings of each logical CPU */
>     DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_mask);
>     /* representing HT and core siblings of each logical CPU */
>     DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_mask);
>
>     So if per_cpu(cpu_core_mask, cpu1) has bit cpu2 set, then cpu2 is a
>     'core sibling' of cpu1.  Same with cpu_sibling_mask, except then it's
>     hyperthreads.
>
>     To be honest, the code you're looking at now is very complicated and
>     not very well commented, so it's not surprising you found it difficult
>     to understand.  I had a plan at some point to update the comments to
>     make it easier to follow.
>
>      -George
>
>


[-- Attachment #1.2: Type: text/html, Size: 5636 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: what the meaning of the per cpu variable cpu_core_mask and cpu_sibling_mask?

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 599 bytes --]

On gio, 2013-03-21 at 21:49 +0800, Demon King wrote:
> Thank you very much !
> 
> Thanks to your tools, I now understand how to trace some variable in
> the source code.
>
cscope is also your friend:

cd <xen-source-tree>
cscope -R

And for everything else:

man cscope
http://cscope.sourceforge.net/

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel