perf events (node-loads, node-load-misses, ...)

perf events (node-loads, node-load-misses, ...)

[Hyojong Kim]

Hello,

My processor has several hardware cache events, of which I am
interested in node-loads, node-load-misses, node-stores,
node-store-misses. I think it is kind of obvious that node-loads and
node-stores represent references to local memory (memory physically
attached to a processor package). However, I have no clue as to what
the rest represents. I have some theory that "node-load-misses"
represent the number of misses in local memory, or the number of
accesses to disk. Based on the profiled results I have
(page-faults:16M << node-load-misses+node-store-misses:748M), however,
this cannot be the raw number of disk accesses. Maybe some kind of
piggybacking mechanism (by some MSHR kind of structure in memory?)
could be the reason behind this discrepancy, but no clue. If anyone
has some idea on this, please share it with me.

I appreciate your help in advance!

Sincerely,
- Hyojong

Re: perf events (node-loads, node-load-misses, ...)

[Taeung Song]

Hi Hyojong,

On 05/02/2017 02:03 AM, Hyojong Kim wrote:
> Hello,
>
> My processor has several hardware cache events, of which I am
> interested in node-loads, node-load-misses, node-stores,
> node-store-misses. I think it is kind of obvious that node-loads and
> node-stores represent references to local memory (memory physically
> attached to a processor package). However, I have no clue as to what
> the rest represents. I have some theory that "node-load-misses"
> represent the number of misses in local memory, or the number of
> accesses to disk. Based on the profiled results I have
> (page-faults:16M << node-load-misses+node-store-misses:748M), however,
> this cannot be the raw number of disk accesses. Maybe some kind of
> piggybacking mechanism (by some MSHR kind of structure in memory?)
> could be the reason behind this discrepancy, but no clue. If anyone
> has some idea on this, please share it with me.
>
> I appreciate your help in advance!
>
> Sincerely,
> - Hyojong

Could I detailedly know your test case ?
(e.g options of perf-record, perf.data, test environment, etc)

Of course, you used the perf events (i.e. node-load-misses, etc),
but if you show the correct command and options when using perf-record
and share a link of the perf.data as the profiled results,
it'd be more easy to investigate this situation.


Thanks,
Taeung

Re: perf events (node-loads, node-load-misses, ...)

[Andi Kleen]

Hyojong Kim <hkim606@gmail.com> writes:

> Hello,
>
> My processor has several hardware cache events, of which I am
> interested in node-loads, node-load-misses, node-stores,
> node-store-misses. I think it is kind of obvious that node-loads and
> node-stores represent references to local memory (memory physically
> attached to a processor package). However, I have no clue as to what
> the rest represents. I have some theory that "node-load-misses"
> represent the number of misses in local memory, or the number of
> accesses to disk. Based on the profiled results I have
> (page-faults:16M << node-load-misses+node-store-misses:748M), however,
> this cannot be the raw number of disk accesses. Maybe some kind of
> piggybacking mechanism (by some MSHR kind of structure in memory?)
> could be the reason behind this discrepancy, but no clue. If anyone
> has some idea on this, please share it with me.

node-loads is the number of memory reads served by the local NUMA node.
node-loads-misses is the number of reads served by a remote NUMA node.
Similar for stores.

It has nothing to do with the disk.

-Andi

Re: perf events (node-loads, node-load-misses, ...)

[Hyojong Kim]

Hi Taeung,

Thank you for trying to help, but I believe it has been cleared up by
Andi Kleen.

Sincerely,
- Hyojong


On Mon, May 1, 2017 at 7:29 PM, Taeung Song <treeze.taeung@gmail.com> wrote:
> Hi Hyojong,
>
>
> On 05/02/2017 02:03 AM, Hyojong Kim wrote:
>>
>> Hello,
>>
>> My processor has several hardware cache events, of which I am
>> interested in node-loads, node-load-misses, node-stores,
>> node-store-misses. I think it is kind of obvious that node-loads and
>> node-stores represent references to local memory (memory physically
>> attached to a processor package). However, I have no clue as to what
>> the rest represents. I have some theory that "node-load-misses"
>> represent the number of misses in local memory, or the number of
>> accesses to disk. Based on the profiled results I have
>> (page-faults:16M << node-load-misses+node-store-misses:748M), however,
>> this cannot be the raw number of disk accesses. Maybe some kind of
>> piggybacking mechanism (by some MSHR kind of structure in memory?)
>> could be the reason behind this discrepancy, but no clue. If anyone
>> has some idea on this, please share it with me.
>>
>> I appreciate your help in advance!
>>
>> Sincerely,
>> - Hyojong
>
>
> Could I detailedly know your test case ?
> (e.g options of perf-record, perf.data, test environment, etc)
>
> Of course, you used the perf events (i.e. node-load-misses, etc),
> but if you show the correct command and options when using perf-record
> and share a link of the perf.data as the profiled results,
> it'd be more easy to investigate this situation.
>
>
> Thanks,
> Taeung

Re: perf events (node-loads, node-load-misses, ...)

[Hyojong Kim]

Hi Andi,

Thank you for clearing up the confusion. It makes more sense to
interpret node-loads-misses that way than attribute it to disk
accesses.

Sincerely,
- Hyojong


On Mon, May 1, 2017 at 11:25 PM, Andi Kleen <andi@firstfloor.org> wrote:
> Hyojong Kim <hkim606@gmail.com> writes:
>
>> Hello,
>>
>> My processor has several hardware cache events, of which I am
>> interested in node-loads, node-load-misses, node-stores,
>> node-store-misses. I think it is kind of obvious that node-loads and
>> node-stores represent references to local memory (memory physically
>> attached to a processor package). However, I have no clue as to what
>> the rest represents. I have some theory that "node-load-misses"
>> represent the number of misses in local memory, or the number of
>> accesses to disk. Based on the profiled results I have
>> (page-faults:16M << node-load-misses+node-store-misses:748M), however,
>> this cannot be the raw number of disk accesses. Maybe some kind of
>> piggybacking mechanism (by some MSHR kind of structure in memory?)
>> could be the reason behind this discrepancy, but no clue. If anyone
>> has some idea on this, please share it with me.
>
> node-loads is the number of memory reads served by the local NUMA node.
> node-loads-misses is the number of reads served by a remote NUMA node.
> Similar for stores.
>
> It has nothing to do with the disk.
>
> -Andi