All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC] hugepages on sparc
@ 2016-01-27 22:29 Nitin Gupta
  2016-01-27 23:17 ` David Miller
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Nitin Gupta @ 2016-01-27 22:29 UTC (permalink / raw)
  To: sparclinux

Hi,

I wanted to use 4M hugepages but found that kernel only supports
non-HW backed 8M hugepages on SPARC. I see the change was made by this
commit: http://permalink.gmane.org/gmane.linux.ports.sparc/18407

I'm not a hugepage expert but as I understand, this change in default
hugepage size is due to the way transparent hugepage (THP) is
implemented, which looks at memory in PMD size increments and PMD
happens to be 8M aligned. Correct? If this THP design is the only
reason why we have moved from 4M to 8M as default hugepage size then I
think it would be better to fix THP instead.

THP code in mm/huge_memory.c also seems to assume an x86 style page
table (depends on "PMD" size etc.). This seems not ideal for
architectures like SPARC where any data structure like hash table can
be used for page tables. So, shouldn't THP code be using some API to
figure out hugepage backing status of a memory area?

Do you think it would be worth fixing THP as above (remove constrain
for hugepage to be PMD aligned) to at least allow enabling hw-backed
hugepage size as default again?

The whole notion of "default hugepage size" looks weird to me. Ideally
there should be equivalent support for all (most?) hw-backed
hugepapges and THP should be able to work with all kernel supported
pages.

Thanks,
Nitin

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [RFC] hugepages on sparc
  2016-01-27 22:29 [RFC] hugepages on sparc Nitin Gupta
@ 2016-01-27 23:17 ` David Miller
  2016-01-28 19:22 ` Nitin Gupta
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2016-01-27 23:17 UTC (permalink / raw)
  To: sparclinux

From: Nitin Gupta <nitin.m.gupta@oracle.com>
Date: Wed, 27 Jan 2016 14:29:49 -0800

> I'm not a hugepage expert but as I understand, this change in default
> hugepage size is due to the way transparent hugepage (THP) is
> implemented, which looks at memory in PMD size increments and PMD
> happens to be 8M aligned. Correct?

Correct.  It is 1,000 times easier to implement THP support if the
PMD boundary matches the huge page size.  We can encode the huge PTEs
directly in the PMDs, so it's the best implementation.

> THP code in mm/huge_memory.c also seems to assume an x86 style page
> table (depends on "PMD" size etc.). This seems not ideal for
> architectures like SPARC where any data structure like hash table can
> be used for page tables. So, shouldn't THP code be using some API to
> figure out hugepage backing status of a memory area?

Making APIs easier to use for architecture other than x86 is not a
priority for anyone.

> Do you think it would be worth fixing THP as above (remove constrain
> for hugepage to be PMD aligned) to at least allow enabling hw-backed
> hugepage size as default again?

I don't see what the problem with 8MB huge pages are, they work really
well and the implementation is incredibly straightforward.

You have to demonstrate a performance of behavioral problem to justify
such work.  Then you have to come up with an implementation of the
THP ABI changes, and satisfy the THP and mm maintainers.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [RFC] hugepages on sparc
  2016-01-27 22:29 [RFC] hugepages on sparc Nitin Gupta
  2016-01-27 23:17 ` David Miller
@ 2016-01-28 19:22 ` Nitin Gupta
  2016-01-28 21:47 ` Sam Ravnborg
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Nitin Gupta @ 2016-01-28 19:22 UTC (permalink / raw)
  To: sparclinux

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 41979 bytes --]



On 01/27/2016 03:17 PM, David Miller wrote:
> From: Nitin Gupta <nitin.m.gupta@oracle.com>
> Date: Wed, 27 Jan 2016 14:29:49 -0800
> 

> 
> I don't see what the problem with 8MB huge pages are, they work really
> well and the implementation is incredibly straightforward.
> 
> You have to demonstrate a performance of behavioral problem to justify
> such work.  Then you have to come up with an implementation of the
> THP ABI changes, and satisfy the THP and mm maintainers.
> 

8M pages don't seem to work too well:

I ran stream benchmark:
https://www.cs.virginia.edu/stream/ref.html

On a SPARC T4 (128 CPUs, 128G RAM) running 4.5.0-rc1,  I got CPU
lockups and the test seems to get stuck near initialization. The
same test completed on the same hardware in 1m53s with normal 8K
pages.  The test allocates a total of 48G of RAM and I forced it
to use hugepages with 'hugectl --heap=8M'. I passed hugepages92
as kernel boot param to make sure all 48G could be allocated with
hugepages.

Since 8M seems not quite functional, I can work on getting back 4M
pages with hopefully decent performance. This would temporarily
break THP on sparc/linux but it would allow me to split the work
in phases.


= For reference, following are logs dumped out during CPU lockup =

NMI watchdog: BUG: soft lockup - CPU#40 stuck for 65s! [stream:4509]
Modules linked in: autofs4(E) xt_REDIRECT(E) nf_nat_redirect(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) nf_conntrack(E) ip_tables(E) bnx2fc(E) cnic(E) uio(E) fcoe(E) libfcoe(E) 8021q(E) libfc(E) garp(E) stp(E) llc(E) ipv6(E) dm_mirror(E) dm_region_hash(E) dm_log(E) openprom(E) cdc_ether(E) usbnet(E) mii(E) sg(E) ixgbe(E) mdio(E) igb(E) i2c_algo_bit(E) i2c_core(E) ptp(E) pps_core(E) hwmon(E) ext4(E) jbd2(E) mbcache(E) usb_storage(E) sr_mod(E) cdrom(E) sd_mod(E) lpfc(E) scsi_transport_fc(E) des_sparc64(E) des_generic(E) sha512_sparc64(E) camellia_sparc64(E) aes_sparc64(E) crc32c_sparc64(E) sha1_sparc64(E) md5_sparc64(E) sha256_sparc64(E) sunvnet(E) sunvdc(E) dm_mod(E)
CPU: 40 PID: 4509 Comm: stream Tainted: G E 4.5.0-rc1-hp4m-ng #5
task: fff8000dcc7c1980 ti: fff8000dc9568000 task.ti: fff8000dc9568000
TSTATE: 0000004411001605 TPC: 0000000000a19224 TNPC: 0000000000a1921c Y: 00000000 Tainted: G E
TPC: <_raw_spin_trylock_bh+0x44/0x120>
g0: 0000000011001606 g1: 00000000000000ff g2: 0000000000000000 g3: 0000000000000017
g4: fff8000dcc7c1980 g5: fff8001dda6d2000 g6: fff8000dc9568000 g7: 0000000801000000
o0: 0000000000000000 o1: 0000000000000710 o2: 00000000000017d3 o3: 0000000000000001
o4: 0000000000000000 o5: 00000000024200ca sp: fff8000dc956b1c1 ret_pc: 00000000005a3008
RPC: <__vma_reservation_common+0xc8/0x100>
l0: fff8000dc7ff96c0 l1: 0000000000000400 l2: 0000000000fddc00 l3: 0000000000bd2240
l4: 0000000000000000 l5: 0000000000000000 l6: 0000000000000001 l7: 0000000000000000
i0: fff800006403d1e4 i1: fff80000644901c8 i2: 00000000000007d1 i3: 0000000000000002
i4: 0000000000fe0080 i5: 0000000000fe0000 i6: fff8000dc956b271 i7: 00000000005a5624
I7: <hugetlb_no_page+0x164/0x3a0>
Call Trace:
[00000000005a5624] hugetlb_no_page+0x164/0x3a0
[00000000005a5b68] hugetlb_fault+0x308/0x460
[00000000005851f4] __handle_mm_fault+0x134/0x180
[0000000000585314] handle_mm_fault+0xd4/0x1a0
[0000000000a1ac34] do_sparc64_fault+0x434/0x700
[0000000000407c2c] sparc64_realfault_common+0x10/0x20
NMI watchdog: BUG: soft lockup - CPU#41 stuck for 67s! [stream:4388]
Modules linked in: autofs4(E) xt_REDIRECT(E) nf_nat_redirect(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) nf_conntrack(E) ip_tables(E) bnx2fc(E) cnic(E) uio(E) fcoe(E) libfcoe(E) 8021q(E) libfc(E) garp(E) stp(E) llc(E) ipv6(E) dm_mirror(E) dm_region_hash(E) dm_log(E) openprom(E) cdc_ether(E) usbnet(E) mii(E) sg(E) ixgbe(E) mdio(E) igb(E) i2c_algo_bit(E) i2c_core(E) ptp(E) pps_core(E) hwmon(E) ext4(E) jbd2(E) mbcache(E) usb_storage(E) sr_mod(E) cdrom(E) sd_mod(E) lpfc(E) scsi_transport_fc(E) des_sparc64(E) des_generic(E) sha512_sparc64(E) camellia_sparc64(E) aes_sparc64(E) crc32c_sparc64(E) sha1_sparc64(E) md5_sparc64(E) sha256_sparc64(E) sunvnet(E) sunvdc(E) dm_mod(E)
CPU: 41 PID: 4388 Comm: stream Tainted: G EL 4.5.0-rc1-hp4m-ng #5
task: fff8000dcc77c960 ti: fff8001dd3470000 task.ti: fff8001dd3470000
TSTATE: 0000000011001606 TPC: 0000000000a19224 TNPC: 0000000000a1921c Y: 00000000 Tainted: G EL
TPC: <_raw_spin_trylock_bh+0x44/0x120>
g0: 0000000421fe8000 g1: 00000000000000ff g2: 0000000000000001 g3: 0000000000000020
g4: fff8000dcc77c960 g5: fff8001dda6f2000 g6: fff8001dd3470000 g7: 0000000000000000
o0: 0006000000e3a978 o1: 000000000000000f o2: 0000000020800040 o3: 00000000041ffff8
o4: 00000000041ffff9 o5: fff8001dfffffb40 sp: fff8001dd3473141 ret_pc: 0000000000458b08
RPC: <pte_alloc_one+0x28/0x40>
l0: 0006000000e3a978 l1: 00000008218001c0 l2: fff8000100357918 l3: fff80001003579b8
l4: 0000000000000450 l5: 0000000000000448 l6: 0000000000000388 l7: fff8000100356d20
i0: fff800006403d1e4 i1: 0000000021800000 i2: 0000000000000000 i3: 0000000000000000
i4: 0000000000000000 i5: 0000000000000388 i6: fff8001dd34731f1 i7: 0000000000585068
I7: <__pte_alloc+0x28/0x80>
Call Trace:
[0000000000585068] __pte_alloc+0x28/0x80
[0000000000459b48] huge_pte_alloc+0xe8/0x120
[00000000005a5c50] hugetlb_fault+0x3f0/0x460
[00000000005851f4] __handle_mm_fault+0x134/0x180
[0000000000585314] handle_mm_fault+0xd4/0x1a0
[0000000000a1ac34] do_sparc64_fault+0x434/0x700
[0000000000407c2c] sparc64_realfault_common+0x10/0x20
INFO: rcu_sched self-detected stall on CPU
INFO: rcu_sched self-detected stall on CPU
47-...: (14998 ticks this GP) idleÇ3/140000000000001/0 softirqg43/6743 fqs\x13966
(t\x15000 jiffies g995 c994 q$49)
Task dump for CPU 41:
stream R running task 0 4388 3846 0x208000301000c08
Call Trace:
[0000000000a1ac34] do_sparc64_fault+0x434/0x700
[0000000000407c2c] sparc64_realfault_common+0x10/0x20
Task dump for CPU 47:
stream R running task 0 4496 3846 0x208000301000c08
Call Trace:
[0000000000494d7c] dump_cpu_task+0x3c/0x60
[00000000004c247c] rcu_dump_cpu_stacks+0x9c/0x100
[00000000004c26e0] print_cpu_stall+0x100/0x180
[00000000004c2b80] check_cpu_stall+0x100/0x120
[00000000004c47b0] rcu_check_callbacks+0x110/0x360
[00000000004ca1a4] update_process_times+0x24/0x60
[00000000004d8868] tick_sched_handle+0x28/0xa0
[00000000004d89e8] tick_sched_timer+0x48/0xa0
[00000000004cad00] __run_hrtimer+0x80/0x200
[00000000004caef0] __hrtimer_run_queues+0x70/0xa0
[00000000004cb08c] hrtimer_interrupt+0x8c/0x1c0
[0000000000a1b160] timer_interrupt+0x80/0xc0
[00000000004209d4] tl0_irq14+0x14/0x20
[0000000000a19220] _raw_spin_trylock_bh+0x40/0x120
[0000000000585068] __pte_alloc+0x28/0x80
[0000000000459b48] huge_pte_alloc+0xe8/0x120

41-...: (14903 ticks this GP) idlevd/140000000000001/0 softirq\x19831/19831 fqs\x13966
(t\x15061 jiffies g995 c994 q$49)
Task dump for CPU 41:
stream R running task 0 4388 3846 0x208000301000c08
Call Trace:
[0000000000494d7c] dump_cpu_task+0x3c/0x60
[00000000004c247c] rcu_dump_cpu_stacks+0x9c/0x100
[00000000004c26e0] print_cpu_stall+0x100/0x180
[00000000004c2b80] check_cpu_stall+0x100/0x120
[00000000004c47b0] rcu_check_callbacks+0x110/0x360
[00000000004ca1a4] update_process_times+0x24/0x60
[00000000004d8868] tick_sched_handle+0x28/0xa0
[00000000004d89e8] tick_sched_timer+0x48/0xa0
[00000000004cad00] __run_hrtimer+0x80/0x200
[00000000004caef0] __hrtimer_run_queues+0x70/0xa0
[00000000004cb08c] hrtimer_interrupt+0x8c/0x1c0
[0000000000a1b160] timer_interrupt+0x80/0xc0
[00000000004209d4] tl0_irq14+0x14/0x20
[0000000000a19224] _raw_spin_trylock_bh+0x44/0x120
[0000000000585068] __pte_alloc+0x28/0x80
[0000000000459b48] huge_pte_alloc+0xe8/0x120
Task dump for CPU 47:
stream R running task 0 4496 3846 0x208000101000c08
Call Trace:
[0000000000a1ac34] do_sparc64_fault+0x434/0x700
[0000000000407c2c] sparc64_realfault_common+0x10/0x20
NMI watchdog: BUG: soft lockup - CPU#47 stuck for 71s! [stream:4496]
Modules linked in: autofs4(E) xt_REDIRECT(E) nf_nat_redirect(E) iptable_nat(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) nf_nat_ipv4(E) nf_nat(E) nf_conntrack(E) ip_tables(E) bnx2fc(E) cnic(E) uio(E) fcoe(E) libfcoe(E) 8021q(E) libfc(E) garp(E) stp(E) llc(E) ipv6(E) dm_mirror(E) dm_region_hash(E) dm_log(E) openprom(E) cdc_ether(E) usbnet(E) mii(E) sg(E) ixgbe(E) mdio(E) igb(E) i2c_algo_bit(E) i2c_core(E) ptp(E) pps_core(E) hwmon(E) ext4(E) jbd2(E) mbcache(E) usb_storage(E) sr_mod(E) cdrom(E) sd_mod(E) lpfc(E) scsi_transport_fc(E) des_sparc64(E) des_generic(E) sha512_sparc64(E) camellia_sparc64(E) aes_sparc64(E) crc32c_sparc64(E) sha1_sparc64(E) md5_sparc64(E) sha256_sparc64(E) sunvnet(E) sunvdc(E) dm_mod(E)
CPU: 47 PID: 4496 Comm: stream Tainted: G EL 4.5.0-rc1-hp4m-ng #5
task: fff8000065318e40 ti: fff8000dc7d18000 task.ti: fff8000dc7d18000
TSTATE: 0000000011001606 TPC: 0000000000a19224 TNPC: 0000000000a1921c Y: 00000000 Tainted: G EL
TPC: <_raw_spin_trylock_bh+0x44/0x120>
g0: 0000000781ff0000 g1: 00000000000000ff g2: 0000000000000001 g3: 0000000000000020
g4: fff8000065318e40 g5: fff8001dda7b2000 g6: fff8000dc7d18000 g7: 0000000000000000
o0: 000600001f050ba0 o1: 000000000000000f o2: 0000000380800040 o3: 00000000701ffff8
o4: 00000000701ffff9 o5: fff8001dfffffb40 sp: fff8000dc7d1b141 ret_pc: 0000000000458b08
RPC: <pte_alloc_one+0x28/0x40>
l0: 000600001f050ba0 l1: 0000000b818001c0 l2: fff8000100357918 l3: fff80001003579b8
l4: 0000000000000450 l5: 0000000000000448 l6: 0000000000000388 l7: fff8000100356d20
i0: fff800006403d1e4 i1: 0000000381800000 i2: 0000000000000000 i3: 0000000000000000
i4: 0000000000000000 i5: 0000000000000388 i6: fff8000dc7d1b1f1 i7: 0000000000585068
I7: <__pte_alloc+0x28/0x80>
Call Trace:
[0000000000585068] __pte_alloc+0x28/0x80
[0000000000459b48] huge_pte_alloc+0xe8/0x120
[00000000005a5c50] hugetlb_fault+0x3f0/0x460
[00000000005851f4] __handle_mm_fault+0x134/0x180
[0000000000585314] handle_mm_fault+0xd4/0x1a0
[0000000000a1ac34] do_sparc64_fault+0x434/0x700
[0000000000407c2c] sparc64_realfault_common+0x10/0x20
INFO: task stream:4446 blocked for more than 30 seconds.
Tainted: G EL 4.5.0-rc1-hp4m-ng #5
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
stream D 0000000000a159cc 0 4446 3846 0x208000100000c00
Call Trace:
[0000000000a158e4] schedule+0x24/0xc0
[0000000000a159cc] schedule_preempt_disabled+0xc/0x20
[0000000000a16cfc] __mutex_lock_slowpath+0x7c/0x140
[0000000000a16df8] mutex_lock+0x38/0x60
[00000000005a590c] hugetlb_fault+0xac/0x460
[00000000005851f4] __handle_mm_fault+0x134/0x180
[0000000000585314] handle_mm_fault+0xd4/0x1a0
[0000000000a1ac34] do_sparc64_fault+0x434/0x700
[0000000000407c2c] sparc64_realfault_common+0x10/0x20
CPU[ 0]: TSTATE[0000004411001605] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4443]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 1]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4486]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 2]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4427]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 3]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4388]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 4]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4433]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 5]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4411]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 6]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4393]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 7]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4477]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 8]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4468]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 9]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4441]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 10]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4431]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 11]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4471]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 12]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4405]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 13]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4456]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 14]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4466]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 15]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4426]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 16]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4490]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 17]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4503]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 18]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4462]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 19]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4412]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 20]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4494]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 21]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4439]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 22]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4454]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 23]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4424]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 24]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4481]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 25]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4460]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 26]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4429]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 27]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4445]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 28]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4414]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 29]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4505]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 30]: TSTATE[0000000011001600] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4383]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 31]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4399]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 32]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4498]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 33]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4401]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 34]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4452]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 35]: TSTATE[0000004411001605] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4409]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 36]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4416]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 37]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4464]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 38]: TSTATE[0000000011001602] TPC[0000000000406420] TNPC[0000000000406424] TASK[stream:4479]
TPC[sun4v_cpu_mondo_send+0x8/0x10] O7[hypervisor_xcall_deliver+0x5c/0x220] I7[xcall_deliver+0x108/0x120] RPC[smp_flush_tlb_page+0x44/0xc0]
CPU[ 39]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4458]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 40]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/40:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 41]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4484]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 42]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4496]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 43]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4403]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 44]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4509]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 45]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4392]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 46]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4450]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 47]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4418]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 48]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4448]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 49]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4395]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 50]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4420]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 51]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4390]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 52]: TSTATE[0000004411001605] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4469]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 53]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4488]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 54]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4391]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 55]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/55:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 56]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4483]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 57]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4397]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 58]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4502]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 59]: TSTATE[0000004411001605] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4473]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 60]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/60:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 61]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/61:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 62]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4422]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 63]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4507]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 64]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4492]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 65]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4461]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 66]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4457]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 67]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4444]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 68]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4387]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 69]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4491]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 70]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4497]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 71]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4430]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 72]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4434]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
* CPU[ 73]: TSTATE[0000000000000000] TPC[0000000000000000] TNPC[0000000000000000] TASK[khungtaskd:651]
TPC[0] O7[0] I7[0] RPC[0]
CPU[ 74]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/74:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 75]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4394]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 76]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4413]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 77]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4504]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 78]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4442]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 79]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4459]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 80]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/80:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 81]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4501]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 82]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4408]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 83]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4415]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 84]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4425]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 85]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/85:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 86]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4440]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 87]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/87:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 88]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4487]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 89]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4499]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 90]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4465]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 91]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4432]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 92]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/92:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 93]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/93:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 94]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/94:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[ 95]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4389]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 96]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4428]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 97]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4404]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 98]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4419]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[ 99]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4506]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[100]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4480]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[101]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4451]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[102]: TSTATE[0000004411001605] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4447]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[103]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4396]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[104]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/104:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[105]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4406]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[106]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4476]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[107]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4436]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[108]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4410]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[109]: TSTATE[0000004411001605] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4475]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[110]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4398]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[111]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4470]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[112]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4482]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[113]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4449]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[114]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4400]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[115]: TSTATE[0000000011001606] TPC[0000000000a19224] TNPC[0000000000a1921c] TASK[stream:4463]
TPC[_raw_spin_trylock_bh+0x44/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[116]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/116:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[117]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4386]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[118]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4453]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[119]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4455]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[120]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4474]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[121]: TSTATE[0000000011001606] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4421]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[122]: TSTATE[0000000080001606] TPC[000000000042d57c] TNPC[000000000042d580] TASK[swapper/122:0]
TPC[arch_cpu_idle+0x7c/0xa0] O7[arch_cpu_idle+0x68/0xa0] I7[default_idle_call+0x28/0x60] RPC[cpuidle_idle_call+0x24/0xa0]
CPU[123]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4495]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[124]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4489]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[125]: TSTATE[0000000011001606] TPC[0000000000a1921c] TNPC[0000000000a19220] TASK[stream:4510]
TPC[_raw_spin_trylock_bh+0x3c/0x120] O7[pte_alloc_one+0x28/0x40] I7[__pte_alloc+0x28/0x80] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[126]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4493]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
CPU[127]: TSTATE[0000004411001605] TPC[0000000000a19220] TNPC[0000000000a19224] TASK[stream:4385]
TPC[_raw_spin_trylock_bh+0x40/0x120] O7[__vma_reservation_common+0xc8/0x100] I7[hugetlb_no_page+0x164/0x3a0] RPC[trace_event_raw_event_softirq+0x34/0xa0]
Kernel panic - not syncing: hung_task: blocked tasks
CPU: 73 PID: 651 Comm: khungtaskd Tainted: G EL 4.5.0-rc1-hp4m-ng #5
Call Trace:
[000000000054cf78] panic+0xd8/0x27c
[0000000000509078] check_hung_task+0x118/0x160
[00000000005091f8] watchdog+0x138/0x180
[000000000048764c] kthread+0xac/0xe0
[0000000000406104] ret_from_fork+0x1c/0x2c
[0000000000000000] (null)
Press Stop-A (L1-A) to return to the boot prom
---[ end Kernel panic - not syncing: hung_task: blocked tasks

Serial console stopped.
--
To unsubscribe from this list: send the line "unsubscribe sparclinux" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [RFC] hugepages on sparc
  2016-01-27 22:29 [RFC] hugepages on sparc Nitin Gupta
  2016-01-27 23:17 ` David Miller
  2016-01-28 19:22 ` Nitin Gupta
@ 2016-01-28 21:47 ` Sam Ravnborg
  2016-01-28 22:44 ` David Miller
  2016-01-29  5:28 ` Nitin Gupta
  4 siblings, 0 replies; 6+ messages in thread
From: Sam Ravnborg @ 2016-01-28 21:47 UTC (permalink / raw)
  To: sparclinux

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 1847 bytes --]

On Thu, Jan 28, 2016 at 11:22:52AM -0800, Nitin Gupta wrote:
> 
> 
> On 01/27/2016 03:17 PM, David Miller wrote:
> > From: Nitin Gupta <nitin.m.gupta@oracle.com>
> > Date: Wed, 27 Jan 2016 14:29:49 -0800
> > 
> 
> > 
> > I don't see what the problem with 8MB huge pages are, they work really
> > well and the implementation is incredibly straightforward.
> > 
> > You have to demonstrate a performance of behavioral problem to justify
> > such work.  Then you have to come up with an implementation of the
> > THP ABI changes, and satisfy the THP and mm maintainers.
> > 
> 
> 8M pages don't seem to work too well:
> 
> I ran stream benchmark:
> https://www.cs.virginia.edu/stream/ref.html
> 
> On a SPARC T4 (128 CPUs, 128G RAM) running 4.5.0-rc1,  I got CPU
> lockups and the test seems to get stuck near initialization. The
> same test completed on the same hardware in 1m53s with normal 8K
> pages.  The test allocates a total of 48G of RAM and I forced it
> to use hugepages with 'hugectl --heap=8M'. I passed hugepages92
> as kernel boot param to make sure all 48G could be allocated with
> hugepages.
> 
> Since 8M seems not quite functional, I can work on getting back 4M
> pages with hopefully decent performance. This would temporarily
> break THP on sparc/linux but it would allow me to split the work
> in phases.

Before introducing anything new it would be good to have the current
functionality fixed. Then you have something working that can be improved.
And you seem to know the area, have access to HW and can reproduce the bug.
So is this something you could at least try to spend some time on before
starting a bigger rewrite?

	Sam
--
To unsubscribe from this list: send the line "unsubscribe sparclinux" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [RFC] hugepages on sparc
  2016-01-27 22:29 [RFC] hugepages on sparc Nitin Gupta
                   ` (2 preceding siblings ...)
  2016-01-28 21:47 ` Sam Ravnborg
@ 2016-01-28 22:44 ` David Miller
  2016-01-29  5:28 ` Nitin Gupta
  4 siblings, 0 replies; 6+ messages in thread
From: David Miller @ 2016-01-28 22:44 UTC (permalink / raw)
  To: sparclinux

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 2389 bytes --]

From: Sam Ravnborg <sam@ravnborg.org>
Date: Thu, 28 Jan 2016 22:47:24 +0100

> On Thu, Jan 28, 2016 at 11:22:52AM -0800, Nitin Gupta wrote:
>> 
>> 
>> On 01/27/2016 03:17 PM, David Miller wrote:
>> > From: Nitin Gupta <nitin.m.gupta@oracle.com>
>> > Date: Wed, 27 Jan 2016 14:29:49 -0800
>> > 
>> 
>> > 
>> > I don't see what the problem with 8MB huge pages are, they work really
>> > well and the implementation is incredibly straightforward.
>> > 
>> > You have to demonstrate a performance of behavioral problem to justify
>> > such work.  Then you have to come up with an implementation of the
>> > THP ABI changes, and satisfy the THP and mm maintainers.
>> > 
>> 
>> 8M pages don't seem to work too well:
>> 
>> I ran stream benchmark:
>> https://www.cs.virginia.edu/stream/ref.html
>> 
>> On a SPARC T4 (128 CPUs, 128G RAM) running 4.5.0-rc1,  I got CPU
>> lockups and the test seems to get stuck near initialization. The
>> same test completed on the same hardware in 1m53s with normal 8K
>> pages.  The test allocates a total of 48G of RAM and I forced it
>> to use hugepages with 'hugectl --heap=8M'. I passed hugepages92
>> as kernel boot param to make sure all 48G could be allocated with
>> hugepages.
>> 
>> Since 8M seems not quite functional, I can work on getting back 4M
>> pages with hopefully decent performance. This would temporarily
>> break THP on sparc/linux but it would allow me to split the work
>> in phases.
> 
> Before introducing anything new it would be good to have the current
> functionality fixed.

I totally agree.

It is _ABSOLUTELY_ _WRONG_ to respond to a bug in THP support by changing
the huge page size.

That is just pure insanity.

Instead, figure out what the bug is an _FIX_ it.

I refuse to entertain any patches presented to me that change the huge
page size without:

1) Fixing the current bug.

2) Seeing if the 8MB performance meets your expectations.

Nitin, your focus is completely in the wrong place.  You cannot even
see how 8MB performs yet you think it's not appropriate to use that as
the THP page size.

That's not logical, and that's not good engineering.

So please stop this obsession with changing the THP page size now,
thanks.
--
To unsubscribe from this list: send the line "unsubscribe sparclinux" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [RFC] hugepages on sparc
  2016-01-27 22:29 [RFC] hugepages on sparc Nitin Gupta
                   ` (3 preceding siblings ...)
  2016-01-28 22:44 ` David Miller
@ 2016-01-29  5:28 ` Nitin Gupta
  4 siblings, 0 replies; 6+ messages in thread
From: Nitin Gupta @ 2016-01-29  5:28 UTC (permalink / raw)
  To: sparclinux

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 2451 bytes --]

On 1/28/16 1:47 PM, Sam Ravnborg wrote:
> On Thu, Jan 28, 2016 at 11:22:52AM -0800, Nitin Gupta wrote:
>>
>>
>> On 01/27/2016 03:17 PM, David Miller wrote:
>>> From: Nitin Gupta <nitin.m.gupta@oracle.com>
>>> Date: Wed, 27 Jan 2016 14:29:49 -0800
>>>
>>
>>>
>>> I don't see what the problem with 8MB huge pages are, they work really
>>> well and the implementation is incredibly straightforward.
>>>
>>> You have to demonstrate a performance of behavioral problem to justify
>>> such work.  Then you have to come up with an implementation of the
>>> THP ABI changes, and satisfy the THP and mm maintainers.
>>>
>>
>> 8M pages don't seem to work too well:
>>
>> I ran stream benchmark:
>> https://www.cs.virginia.edu/stream/ref.html
>>
>> On a SPARC T4 (128 CPUs, 128G RAM) running 4.5.0-rc1,  I got CPU
>> lockups and the test seems to get stuck near initialization. The
>> same test completed on the same hardware in 1m53s with normal 8K
>> pages.  The test allocates a total of 48G of RAM and I forced it
>> to use hugepages with 'hugectl --heap=8M'. I passed hugepages92
>> as kernel boot param to make sure all 48G could be allocated with
>> hugepages.
>>
>> Since 8M seems not quite functional, I can work on getting back 4M
>> pages with hopefully decent performance. This would temporarily
>> break THP on sparc/linux but it would allow me to split the work
>> in phases.
> 
> Before introducing anything new it would be good to have the current
> functionality fixed. Then you have something working that can be improved.
> And you seem to know the area, have access to HW and can reproduce the bug.
> So is this something you could at least try to spend some time on before
> starting a bigger rewrite?
> 


I was just concerned that fixing 8M might be more work than getting
hw-supported 4M pages to work and since 4M->8M transition did not
seem to be based on very fundamental reasons, I believed directly
going for 4M pages while disabling THP temporarily (currently
it doesn't really work great on sparc/linux anyways) would be
worth it.

Anyways, I did make a quick hack to enable 4M pages and I see
CPU lockups with same stack trace so I guess they both
have same underlying cause, so I'm now okay with fixing 8M
case first.

Thanks,
Nitin


--
To unsubscribe from this list: send the line "unsubscribe sparclinux" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2016-01-29  5:28 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-27 22:29 [RFC] hugepages on sparc Nitin Gupta
2016-01-27 23:17 ` David Miller
2016-01-28 19:22 ` Nitin Gupta
2016-01-28 21:47 ` Sam Ravnborg
2016-01-28 22:44 ` David Miller
2016-01-29  5:28 ` Nitin Gupta

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.