bad rss-counter message in 3.14rc5

bad rss-counter message in 3.14rc5

[Dave Jones]

I just saw this on my box that's been running trinity..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)

There's nothing else, no trace, nothing.  Any ideas where to begin with this?

	Dave

bad rss-counter message in 3.14rc5

[Dave Jones]

I just saw this on my box that's been running trinity..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)

There's nothing else, no trace, nothing.  Any ideas where to begin with this?

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote:
 > I just saw this on my box that's been running trinity..
 > 
 > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 > 
 > There's nothing else, no trace, nothing.  Any ideas where to begin with this?

ah, on the serial console there was also this truncated warning..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
[48924.133273] ------------[ cut here ]------------
[48924.133391] kernel BUG at include/linux/swapops.h:131!

	Dave

124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
125 {
126         struct page *p = pfn_to_page(swp_offset(entry));
127         /*
128          * Any use of migration entries may only occur while the
129          * corresponding page is locked
130          */
131         BUG_ON(!PageLocked(p));
132         return p;
133 }

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote:
 > I just saw this on my box that's been running trinity..
 > 
 > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 > 
 > There's nothing else, no trace, nothing.  Any ideas where to begin with this?

ah, on the serial console there was also this truncated warning..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
[48924.133273] ------------[ cut here ]------------
[48924.133391] kernel BUG at include/linux/swapops.h:131!

	Dave

124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
125 {
126         struct page *p = pfn_to_page(swp_offset(entry));
127         /*
128          * Any use of migration entries may only occur while the
129          * corresponding page is locked
130          */
131         BUG_ON(!PageLocked(p));
132         return p;
133 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote:
 > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote:
 >  > I just saw this on my box that's been running trinity..
 >  > 
 >  > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 >  > 
 >  > There's nothing else, no trace, nothing.  Any ideas where to begin with this?
 > 
 > ah, on the serial console there was also this truncated warning..
 > 
 > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 > [48924.133273] ------------[ cut here ]------------
 > [48924.133391] kernel BUG at include/linux/swapops.h:131!
 > 
 > 	Dave
 > 
 > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
 > 125 {
 > 126         struct page *p = pfn_to_page(swp_offset(entry));
 > 127         /*
 > 128          * Any use of migration entries may only occur while the
 > 129          * corresponding page is locked
 > 130          */
 > 131         BUG_ON(!PageLocked(p));
 > 132         return p;
 > 133 }

I hit this again, this time a full trace made it over the serial console.
This time there was no bad rss-counter message though.

kernel BUG at include/linux/swapops.h:131!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Stack:
 ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
 ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
Call Trace:
 [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
 [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
 [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
 [<ffffffff9d068d13>] mmput+0x73/0x110
 [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
 [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
 [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
 [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
 [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
 [<ffffffff9d0024c7>] do_signal+0x57/0x9d0
 [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
 [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
 [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
 [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
 [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
 [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
 [<ffffffff9d739c06>] retint_signal+0x46/0x90
Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote:
 > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote:
 >  > I just saw this on my box that's been running trinity..
 >  > 
 >  > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 >  > 
 >  > There's nothing else, no trace, nothing.  Any ideas where to begin with this?
 > 
 > ah, on the serial console there was also this truncated warning..
 > 
 > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 > [48924.133273] ------------[ cut here ]------------
 > [48924.133391] kernel BUG at include/linux/swapops.h:131!
 > 
 > 	Dave
 > 
 > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
 > 125 {
 > 126         struct page *p = pfn_to_page(swp_offset(entry));
 > 127         /*
 > 128          * Any use of migration entries may only occur while the
 > 129          * corresponding page is locked
 > 130          */
 > 131         BUG_ON(!PageLocked(p));
 > 132         return p;
 > 133 }

I hit this again, this time a full trace made it over the serial console.
This time there was no bad rss-counter message though.

kernel BUG at include/linux/swapops.h:131!
invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Stack:
 ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
 ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
Call Trace:
 [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
 [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
 [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
 [<ffffffff9d068d13>] mmput+0x73/0x110
 [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
 [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
 [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
 [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
 [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
 [<ffffffff9d0024c7>] do_signal+0x57/0x9d0
 [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
 [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
 [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
 [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
 [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
 [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
 [<ffffffff9d739c06>] retint_signal+0x46/0x90
Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Thu, Mar 06, 2014 at 07:22:10PM -0500, Dave Jones wrote:
 > On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote:
 >  > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote:
 >  >  > I just saw this on my box that's been running trinity..
 >  >  > 
 >  >  > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 >  >  > 
 >  >  > There's nothing else, no trace, nothing.  Any ideas where to begin with this?
 >  > 
 >  > ah, on the serial console there was also this truncated warning..
 >  > 
 >  > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 >  > [48924.133273] ------------[ cut here ]------------
 >  > [48924.133391] kernel BUG at include/linux/swapops.h:131!
 >  > 
 >  > 	Dave
 >  > 
 >  > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
 >  > 125 {
 >  > 126         struct page *p = pfn_to_page(swp_offset(entry));
 >  > 127         /*
 >  > 128          * Any use of migration entries may only occur while the
 >  > 129          * corresponding page is locked
 >  > 130          */
 >  > 131         BUG_ON(!PageLocked(p));
 >  > 132         return p;
 >  > 133 }
 > 
 > I hit this again, this time a full trace made it over the serial console.
 > This time there was no bad rss-counter message though.
 > 
 > kernel BUG at include/linux/swapops.h:131!
 > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
 > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
 > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
 > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
 > RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
 > RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
 > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
 > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
 > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
 > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
 > FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
 > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
 > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
 > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
 > Stack:
 >  ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
 >  ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
 >  0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
 > Call Trace:
 >  [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
 >  [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
 >  [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
 >  [<ffffffff9d068d13>] mmput+0x73/0x110
 >  [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
 >  [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
 >  [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
 >  [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
 >  [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
 >  [<ffffffff9d0024c7>] do_signal+0x57/0x9d0
 >  [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
 >  [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
 >  [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
 >  [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
 >  [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
 >  [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
 >  [<ffffffff9d739c06>] retint_signal+0x46/0x90
 > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
while trying to reproduce a different bug..

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Thu, Mar 06, 2014 at 07:22:10PM -0500, Dave Jones wrote:
 > On Wed, Mar 05, 2014 at 12:57:25PM -0500, Dave Jones wrote:
 >  > On Wed, Mar 05, 2014 at 12:45:03PM -0500, Dave Jones wrote:
 >  >  > I just saw this on my box that's been running trinity..
 >  >  > 
 >  >  > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 >  >  > 
 >  >  > There's nothing else, no trace, nothing.  Any ideas where to begin with this?
 >  > 
 >  > ah, on the serial console there was also this truncated warning..
 >  > 
 >  > [48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
 >  > [48924.133273] ------------[ cut here ]------------
 >  > [48924.133391] kernel BUG at include/linux/swapops.h:131!
 >  > 
 >  > 	Dave
 >  > 
 >  > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
 >  > 125 {
 >  > 126         struct page *p = pfn_to_page(swp_offset(entry));
 >  > 127         /*
 >  > 128          * Any use of migration entries may only occur while the
 >  > 129          * corresponding page is locked
 >  > 130          */
 >  > 131         BUG_ON(!PageLocked(p));
 >  > 132         return p;
 >  > 133 }
 > 
 > I hit this again, this time a full trace made it over the serial console.
 > This time there was no bad rss-counter message though.
 > 
 > kernel BUG at include/linux/swapops.h:131!
 > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
 > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
 > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
 > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
 > RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
 > RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
 > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
 > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
 > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
 > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
 > FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
 > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
 > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
 > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
 > Stack:
 >  ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
 >  ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
 >  0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
 > Call Trace:
 >  [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
 >  [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
 >  [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
 >  [<ffffffff9d068d13>] mmput+0x73/0x110
 >  [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
 >  [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
 >  [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
 >  [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
 >  [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
 >  [<ffffffff9d0024c7>] do_signal+0x57/0x9d0
 >  [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
 >  [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
 >  [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
 >  [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
 >  [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
 >  [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
 >  [<ffffffff9d739c06>] retint_signal+0x46/0x90
 > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
while trying to reproduce a different bug..

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Andrew Morton]

On Mon, 10 Mar 2014 22:49:06 -0400 Dave Jones <davej@redhat.com> wrote:

> ...
>
>  >  > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
>  >  > 125 {
>  >  > 126         struct page *p = pfn_to_page(swp_offset(entry));
>  >  > 127         /*
>  >  > 128          * Any use of migration entries may only occur while the
>  >  > 129          * corresponding page is locked
>  >  > 130          */
>  >  > 131         BUG_ON(!PageLocked(p));
>  >  > 132         return p;
>  >  > 133 }
>  > 
>  > I hit this again, this time a full trace made it over the serial console.
>  > This time there was no bad rss-counter message though.
>  > 
>  > kernel BUG at include/linux/swapops.h:131!
>  > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
>  > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
>  > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
>  > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
>  > RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
>  > RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
>  > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
>  > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
>  > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
>  > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
>  > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
>  > FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
>  > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
>  > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
>  > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
>  > Stack:
>  >  ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
>  >  ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
>  >  0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
>  > Call Trace:
>  >  [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
>  >  [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
>  >  [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
>  >  [<ffffffff9d068d13>] mmput+0x73/0x110
>  >  [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
>  >  [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
>  >  [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
>  >  [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
>  >  [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
>  >  [<ffffffff9d0024c7>] do_signal+0x57/0x9d0
>  >  [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
>  >  [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
>  >  [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
>  >  [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
>  >  [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
>  >  [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
>  >  [<ffffffff9d739c06>] retint_signal+0x46/0x90
>  > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8
> 
> Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
> while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not.  Cc's added.

Guys, what stops the migration target page from coming unlocked in
parallel with zap_pte_range()'s call to migration_entry_to_page()?

Re: bad rss-counter message in 3.14rc5

[Andrew Morton]

On Mon, 10 Mar 2014 22:49:06 -0400 Dave Jones <davej@redhat.com> wrote:

> ...
>
>  >  > 124 static inline struct page *migration_entry_to_page(swp_entry_t entry)
>  >  > 125 {
>  >  > 126         struct page *p = pfn_to_page(swp_offset(entry));
>  >  > 127         /*
>  >  > 128          * Any use of migration entries may only occur while the
>  >  > 129          * corresponding page is locked
>  >  > 130          */
>  >  > 131         BUG_ON(!PageLocked(p));
>  >  > 132         return p;
>  >  > 133 }
>  > 
>  > I hit this again, this time a full trace made it over the serial console.
>  > This time there was no bad rss-counter message though.
>  > 
>  > kernel BUG at include/linux/swapops.h:131!
>  > invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
>  > Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
>  > CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
>  > task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
>  > RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
>  > RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
>  > RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
>  > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
>  > RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
>  > R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
>  > R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
>  > FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
>  > CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  > CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
>  > DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
>  > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
>  > Stack:
>  >  ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
>  >  ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
>  >  0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
>  > Call Trace:
>  >  [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
>  >  [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
>  >  [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
>  >  [<ffffffff9d068d13>] mmput+0x73/0x110
>  >  [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
>  >  [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
>  >  [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
>  >  [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
>  >  [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
>  >  [<ffffffff9d0024c7>] do_signal+0x57/0x9d0
>  >  [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
>  >  [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
>  >  [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
>  >  [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
>  >  [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
>  >  [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
>  >  [<ffffffff9d739c06>] retint_signal+0x46/0x90
>  > Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8
> 
> Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
> while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not.  Cc's added.

Guys, what stops the migration target page from coming unlocked in
parallel with zap_pte_range()'s call to migration_entry_to_page()?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Andrew Morton]

On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
> > while trying to reproduce a different bug..
> 
> Damn, I thought we'd fixed that but it seems not.  Cc's added.
> 
> Guys, what stops the migration target page from coming unlocked in
> parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of.  At least, transitions of is_migration_entry()
and page_locked() happen under ptl.

I don't see any holes in regular migration.  Do you know if this is
reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

Re: bad rss-counter message in 3.14rc5

[Andrew Morton]

On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
> > while trying to reproduce a different bug..
> 
> Damn, I thought we'd fixed that but it seems not.  Cc's added.
> 
> Guys, what stops the migration target page from coming unlocked in
> parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of.  At least, transitions of is_migration_entry()
and page_locked() happen under ptl.

I don't see any holes in regular migration.  Do you know if this is
reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > 
 > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > > > while trying to reproduce a different bug..
 > > 
 > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > > 
 > > Guys, what stops the migration target page from coming unlocked in
 > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > 
 > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > and page_locked() happen under ptl.
 > 
 > I don't see any holes in regular migration.  Do you know if this is
 > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

I'll give it an overnight run and let you know tomorrow.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > 
 > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > > > while trying to reproduce a different bug..
 > > 
 > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > > 
 > > Guys, what stops the migration target page from coming unlocked in
 > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > 
 > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > and page_locked() happen under ptl.
 > 
 > I don't see any holes in regular migration.  Do you know if this is
 > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

I'll give it an overnight run and let you know tomorrow.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > 
 > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > > > while trying to reproduce a different bug..
 > > 
 > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > > 
 > > Guys, what stops the migration target page from coming unlocked in
 > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > 
 > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > and page_locked() happen under ptl.
 > 
 > I don't see any holes in regular migration.  Do you know if this is
 > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > 
 > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > > > while trying to reproduce a different bug..
 > > 
 > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > > 
 > > Guys, what stops the migration target page from coming unlocked in
 > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > 
 > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > and page_locked() happen under ptl.
 > 
 > I don't see any holes in regular migration.  Do you know if this is
 > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Andrew Morton]

On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:

> On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
>  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>  > 
>  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
>  > > > while trying to reproduce a different bug..
>  > > 
>  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
>  > > 
>  > > Guys, what stops the migration target page from coming unlocked in
>  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
>  > 
>  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
>  > and page_locked() happen under ptl.
>  > 
>  > I don't see any holes in regular migration.  Do you know if this is
>  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
> 
> CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

There probably isn't much point unless trinity is using
sys_move_pages().  Is it?  If so it would be interesting to disable
trinity's move_pages calls and see if it still fails.

Grasping at straws here, trying to reduce the amount of code to look at :(

Re: bad rss-counter message in 3.14rc5

[Andrew Morton]

On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:

> On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
>  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>  > 
>  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
>  > > > while trying to reproduce a different bug..
>  > > 
>  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
>  > > 
>  > > Guys, what stops the migration target page from coming unlocked in
>  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
>  > 
>  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
>  > and page_locked() happen under ptl.
>  > 
>  > I don't see any holes in regular migration.  Do you know if this is
>  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
> 
> CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

There probably isn't much point unless trinity is using
sys_move_pages().  Is it?  If so it would be interesting to disable
trinity's move_pages calls and see if it still fails.

Grasping at straws here, trying to reduce the amount of code to look at :(

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote:
 > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:
 > 
 > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > >  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > >  > 
 > >  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > >  > > > while trying to reproduce a different bug..
 > >  > > 
 > >  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > >  > > 
 > >  > > Guys, what stops the migration target page from coming unlocked in
 > >  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > >  > 
 > >  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > >  > and page_locked() happen under ptl.
 > >  > 
 > >  > I don't see any holes in regular migration.  Do you know if this is
 > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
 > > 
 > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
 > 
 > There probably isn't much point unless trinity is using
 > sys_move_pages().  Is it?

Trinity will do every syscall an arch has.

In the test case I have so far, I've narrowed it down to the vm group of syscalls
(so running with '-g vm' will do anything that I deemed 'vm'. Including.. sys_move_pages)
I'll try to narrow it down further tomorrow.

 >  If so it would be interesting to disable
 > trinity's move_pages calls and see if it still fails.
 
Ok, I'll try that first.

 > Grasping at straws here, trying to reduce the amount of code to look at :(

*nod*, it's not helped by the fact that the trace happens at process exit time
which could be considerably later after the syscall that buggers everything up
has happened.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote:
 > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:
 > 
 > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > >  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > >  > 
 > >  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > >  > > > while trying to reproduce a different bug..
 > >  > > 
 > >  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > >  > > 
 > >  > > Guys, what stops the migration target page from coming unlocked in
 > >  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > >  > 
 > >  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > >  > and page_locked() happen under ptl.
 > >  > 
 > >  > I don't see any holes in regular migration.  Do you know if this is
 > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
 > > 
 > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
 > 
 > There probably isn't much point unless trinity is using
 > sys_move_pages().  Is it?

Trinity will do every syscall an arch has.

In the test case I have so far, I've narrowed it down to the vm group of syscalls
(so running with '-g vm' will do anything that I deemed 'vm'. Including.. sys_move_pages)
I'll try to narrow it down further tomorrow.

 >  If so it would be interesting to disable
 > trinity's move_pages calls and see if it still fails.
 
Ok, I'll try that first.

 > Grasping at straws here, trying to reduce the amount of code to look at :(

*nod*, it's not helped by the fact that the trace happens at process exit time
which could be considerably later after the syscall that buggers everything up
has happened.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote:
 > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:
 > 
 > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > >  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > >  > 
 > >  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > >  > > > while trying to reproduce a different bug..
 > >  > > 
 > >  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > >  > > 
 > >  > > Guys, what stops the migration target page from coming unlocked in
 > >  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > >  > 
 > >  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > >  > and page_locked() happen under ptl.
 > >  > 
 > >  > I don't see any holes in regular migration.  Do you know if this is
 > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
 > > 
 > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
 > 
 > There probably isn't much point unless trinity is using
 > sys_move_pages().  Is it?  If so it would be interesting to disable
 > trinity's move_pages calls and see if it still fails.

Ok, with move_pages excluded it still oopses.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote:
 > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:
 > 
 > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
 > >  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
 > >  > 
 > >  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
 > >  > > > while trying to reproduce a different bug..
 > >  > > 
 > >  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
 > >  > > 
 > >  > > Guys, what stops the migration target page from coming unlocked in
 > >  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
 > >  > 
 > >  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
 > >  > and page_locked() happen under ptl.
 > >  > 
 > >  > I don't see any holes in regular migration.  Do you know if this is
 > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
 > > 
 > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
 > 
 > There probably isn't much point unless trinity is using
 > sys_move_pages().  Is it?  If so it would be interesting to disable
 > trinity's move_pages calls and see if it still fails.

Ok, with move_pages excluded it still oopses.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/11/2014 01:30 AM, Dave Jones wrote:
> On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote:
>   > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:
>   >
>   > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
>   > >  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>   > >  >
>   > >  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
>   > >  > > > while trying to reproduce a different bug..
>   > >  > >
>   > >  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
>   > >  > >
>   > >  > > Guys, what stops the migration target page from coming unlocked in
>   > >  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
>   > >  >
>   > >  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
>   > >  > and page_locked() happen under ptl.
>   > >  >
>   > >  > I don't see any holes in regular migration.  Do you know if this is
>   > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
>   > >
>   > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
>   >
>   > There probably isn't much point unless trinity is using
>   > sys_move_pages().  Is it?  If so it would be interesting to disable
>   > trinity's move_pages calls and see if it still fails.
>
> Ok, with move_pages excluded it still oopses.

FWIW, yes - I still see both of these issues happening. It's easy to ignore the
bad rss-counter, and I've commented out the BUG at swapops.h so that I could keep
on testing.

There are quite a few issues within mm/ right now, I think there are more than 5
different BUG()s hittable using trinity at this point without a fix.


Thanks,
Sasha

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/11/2014 01:30 AM, Dave Jones wrote:
> On Mon, Mar 10, 2014 at 10:01:58PM -0700, Andrew Morton wrote:
>   > On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <davej@redhat.com> wrote:
>   >
>   > > On Mon, Mar 10, 2014 at 09:46:12PM -0700, Andrew Morton wrote:
>   > >  > On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
>   > >  >
>   > >  > > > Anyone ? I'm hitting this trace on an almost daily basis, which is a pain
>   > >  > > > while trying to reproduce a different bug..
>   > >  > >
>   > >  > > Damn, I thought we'd fixed that but it seems not.  Cc's added.
>   > >  > >
>   > >  > > Guys, what stops the migration target page from coming unlocked in
>   > >  > > parallel with zap_pte_range()'s call to migration_entry_to_page()?
>   > >  >
>   > >  > page_table_lock, sort-of.  At least, transitions of is_migration_entry()
>   > >  > and page_locked() happen under ptl.
>   > >  >
>   > >  > I don't see any holes in regular migration.  Do you know if this is
>   > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
>   > >
>   > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
>   >
>   > There probably isn't much point unless trinity is using
>   > sys_move_pages().  Is it?  If so it would be interesting to disable
>   > trinity's move_pages calls and see if it still fails.
>
> Ok, with move_pages excluded it still oopses.

FWIW, yes - I still see both of these issues happening. It's easy to ignore the
bad rss-counter, and I've commented out the BUG at swapops.h so that I could keep
on testing.

There are quite a few issues within mm/ right now, I think there are more than 5
different BUG()s hittable using trinity at this point without a fix.


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote:
>  > >  > 
>  > >  > I don't see any holes in regular migration.  Do you know if this is
>  > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
>  > > 
>  > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
>  > 
>  > There probably isn't much point unless trinity is using
>  > sys_move_pages().  Is it?  If so it would be interesting to disable
>  > trinity's move_pages calls and see if it still fails.
> 
> Ok, with move_pages excluded it still oopses.

Dave, is it possible to somehow figure out was someone reading pagemap file
at moment of the bug triggering?

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote:
>  > >  > 
>  > >  > I don't see any holes in regular migration.  Do you know if this is
>  > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
>  > > 
>  > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
>  > 
>  > There probably isn't much point unless trinity is using
>  > sys_move_pages().  Is it?  If so it would be interesting to disable
>  > trinity's move_pages calls and see if it still fails.
> 
> Ok, with move_pages excluded it still oopses.

Dave, is it possible to somehow figure out was someone reading pagemap file
at moment of the bug triggering?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/11/2014 09:20 AM, Cyrill Gorcunov wrote:
> On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote:
>>   > >  >
>>   > >  > I don't see any holes in regular migration.  Do you know if this is
>>   > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
>>   > >
>>   > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
>>   >
>>   > There probably isn't much point unless trinity is using
>>   > sys_move_pages().  Is it?  If so it would be interesting to disable
>>   > trinity's move_pages calls and see if it still fails.
>>
>> Ok, with move_pages excluded it still oopses.
>
> Dave, is it possible to somehow figure out was someone reading pagemap file
> at moment of the bug triggering?

We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
should be close enough to confirm/deny the theory.


Thanks,
Sasha

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/11/2014 09:20 AM, Cyrill Gorcunov wrote:
> On Tue, Mar 11, 2014 at 01:30:17AM -0400, Dave Jones wrote:
>>   > >  >
>>   > >  > I don't see any holes in regular migration.  Do you know if this is
>>   > >  > reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?
>>   > >
>>   > > CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.
>>   >
>>   > There probably isn't much point unless trinity is using
>>   > sys_move_pages().  Is it?  If so it would be interesting to disable
>>   > trinity's move_pages calls and see if it still fails.
>>
>> Ok, with move_pages excluded it still oopses.
>
> Dave, is it possible to somehow figure out was someone reading pagemap file
> at moment of the bug triggering?

We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
should be close enough to confirm/deny the theory.


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
> >>
> >>Ok, with move_pages excluded it still oopses.
> >
> >Dave, is it possible to somehow figure out was someone reading pagemap file
> >at moment of the bug triggering?
> 
> We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
> should be close enough to confirm/deny the theory.

After reading some more, I suppose the idea I had is wrong, investigating.
Will ping if I find something.

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
> >>
> >>Ok, with move_pages excluded it still oopses.
> >
> >Dave, is it possible to somehow figure out was someone reading pagemap file
> >at moment of the bug triggering?
> 
> We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
> should be close enough to confirm/deny the theory.

After reading some more, I suppose the idea I had is wrong, investigating.
Will ping if I find something.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote:
 > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
 > > >>
 > > >>Ok, with move_pages excluded it still oopses.
 > > >
 > > >Dave, is it possible to somehow figure out was someone reading pagemap file
 > > >at moment of the bug triggering?
 > > 
 > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
 > > should be close enough to confirm/deny the theory.
 > 
 > After reading some more, I suppose the idea I had is wrong, investigating.
 > Will ping if I find something.

I can rule it out anyway, I can reproduce this by telling trinity to do nothing
other than mmap()'s.   I'll try and narrow down the exact parameters.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote:
 > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
 > > >>
 > > >>Ok, with move_pages excluded it still oopses.
 > > >
 > > >Dave, is it possible to somehow figure out was someone reading pagemap file
 > > >at moment of the bug triggering?
 > > 
 > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
 > > should be close enough to confirm/deny the theory.
 > 
 > After reading some more, I suppose the idea I had is wrong, investigating.
 > Will ping if I find something.

I can rule it out anyway, I can reproduce this by telling trinity to do nothing
other than mmap()'s.   I'll try and narrow down the exact parameters.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote:
> On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote:
>  > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
>  > > >>
>  > > >>Ok, with move_pages excluded it still oopses.
>  > > >
>  > > >Dave, is it possible to somehow figure out was someone reading pagemap file
>  > > >at moment of the bug triggering?
>  > > 
>  > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
>  > > should be close enough to confirm/deny the theory.
>  > 
>  > After reading some more, I suppose the idea I had is wrong, investigating.
>  > Will ping if I find something.
> 
> I can rule it out anyway, I can reproduce this by telling trinity to do nothing
> other than mmap()'s.   I'll try and narrow down the exact parameters.

Dave, iirc trinity can write log file pointing which exactly syscall sequence
was passed, right? Share it too please.

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote:
> On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote:
>  > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
>  > > >>
>  > > >>Ok, with move_pages excluded it still oopses.
>  > > >
>  > > >Dave, is it possible to somehow figure out was someone reading pagemap file
>  > > >at moment of the bug triggering?
>  > > 
>  > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
>  > > should be close enough to confirm/deny the theory.
>  > 
>  > After reading some more, I suppose the idea I had is wrong, investigating.
>  > Will ping if I find something.
> 
> I can rule it out anyway, I can reproduce this by telling trinity to do nothing
> other than mmap()'s.   I'll try and narrow down the exact parameters.

Dave, iirc trinity can write log file pointing which exactly syscall sequence
was passed, right? Share it too please.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/11/2014 10:37 AM, Cyrill Gorcunov wrote:
> On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote:
>> On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote:
>>   > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
>>   > > >>
>>   > > >>Ok, with move_pages excluded it still oopses.
>>   > > >
>>   > > >Dave, is it possible to somehow figure out was someone reading pagemap file
>>   > > >at moment of the bug triggering?
>>   > >
>>   > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
>>   > > should be close enough to confirm/deny the theory.
>>   >
>>   > After reading some more, I suppose the idea I had is wrong, investigating.
>>   > Will ping if I find something.
>>
>> I can rule it out anyway, I can reproduce this by telling trinity to do nothing
>> other than mmap()'s.   I'll try and narrow down the exact parameters.
>
> Dave, iirc trinity can write log file pointing which exactly syscall sequence
> was passed, right? Share it too please.

I've sent one of those last time I reported this issue:

	https://lkml.org/lkml/2014/1/22/625


Thanks,
Sasha

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/11/2014 10:37 AM, Cyrill Gorcunov wrote:
> On Tue, Mar 11, 2014 at 10:28:17AM -0400, Dave Jones wrote:
>> On Tue, Mar 11, 2014 at 05:41:58PM +0400, Cyrill Gorcunov wrote:
>>   > On Tue, Mar 11, 2014 at 09:23:05AM -0400, Sasha Levin wrote:
>>   > > >>
>>   > > >>Ok, with move_pages excluded it still oopses.
>>   > > >
>>   > > >Dave, is it possible to somehow figure out was someone reading pagemap file
>>   > > >at moment of the bug triggering?
>>   > >
>>   > > We can sprinkle printk()s wherever might be useful, might not be 100% accurate but
>>   > > should be close enough to confirm/deny the theory.
>>   >
>>   > After reading some more, I suppose the idea I had is wrong, investigating.
>>   > Will ping if I find something.
>>
>> I can rule it out anyway, I can reproduce this by telling trinity to do nothing
>> other than mmap()'s.   I'll try and narrow down the exact parameters.
>
> Dave, iirc trinity can write log file pointing which exactly syscall sequence
> was passed, right? Share it too please.

I've sent one of those last time I reported this issue:

	https://lkml.org/lkml/2014/1/22/625


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 11, 2014 at 06:37:50PM +0400, Cyrill Gorcunov wrote:
 
 > >  > After reading some more, I suppose the idea I had is wrong, investigating.
 > >  > Will ping if I find something.
 > > 
 > > I can rule it out anyway, I can reproduce this by telling trinity to do nothing
 > > other than mmap()'s.   I'll try and narrow down the exact parameters.
 > 
 > Dave, iirc trinity can write log file pointing which exactly syscall sequence
 > was passed, right? Share it too please.

Hm, I may have been mistaken, and the damage was done by a previous run.
I went from being able to reproduce it almost instantly to now not being able
to reproduce it at all.  Will keep trying.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 11, 2014 at 06:37:50PM +0400, Cyrill Gorcunov wrote:
 
 > >  > After reading some more, I suppose the idea I had is wrong, investigating.
 > >  > Will ping if I find something.
 > > 
 > > I can rule it out anyway, I can reproduce this by telling trinity to do nothing
 > > other than mmap()'s.   I'll try and narrow down the exact parameters.
 > 
 > Dave, iirc trinity can write log file pointing which exactly syscall sequence
 > was passed, right? Share it too please.

Hm, I may have been mistaken, and the damage was done by a previous run.
I went from being able to reproduce it almost instantly to now not being able
to reproduce it at all.  Will keep trying.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>  > 
>  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
>  > was passed, right? Share it too please.
> 
> Hm, I may have been mistaken, and the damage was done by a previous run.
> I went from being able to reproduce it almost instantly to now not being able
> to reproduce it at all.  Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>  > 
>  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
>  > was passed, right? Share it too please.
> 
> Hm, I may have been mistaken, and the damage was done by a previous run.
> I went from being able to reproduce it almost instantly to now not being able
> to reproduce it at all.  Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
 > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
 > >  > 
 > >  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
 > >  > was passed, right? Share it too please.
 > > 
 > > Hm, I may have been mistaken, and the damage was done by a previous run.
 > > I went from being able to reproduce it almost instantly to now not being able
 > > to reproduce it at all.  Will keep trying.
 > 
 > Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
 > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
 > >  > 
 > >  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
 > >  > was passed, right? Share it too please.
 > > 
 > > Hm, I may have been mistaken, and the damage was done by a previous run.
 > > I went from being able to reproduce it almost instantly to now not being able
 > > to reproduce it at all.  Will keep trying.
 > 
 > Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 01:39:17PM -0400, Dave Jones wrote:
>  > 
>  > Sasha already gave a link to the syscalls sequence, so no rush.
> 
> It'd be nice to get a more concise reproducer, his list had a little of everything in there.

Dave, could you please send me your config privately so I would try to reproduce
the issue locally maybe it shed some light on the problem.

	Cyrill

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 11, 2014 at 01:39:17PM -0400, Dave Jones wrote:
>  > 
>  > Sasha already gave a link to the syscalls sequence, so no rush.
> 
> It'd be nice to get a more concise reproducer, his list had a little of everything in there.

Dave, could you please send me your config privately so I would try to reproduce
the issue locally maybe it shed some light on the problem.

	Cyrill

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 11 Mar 2014, Dave Jones wrote:
> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
>  > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>  > >  > 
>  > >  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
>  > >  > was passed, right? Share it too please.
>  > > 
>  > > Hm, I may have been mistaken, and the damage was done by a previous run.
>  > > I went from being able to reproduce it almost instantly to now not being able
>  > > to reproduce it at all.  Will keep trying.
>  > 
>  > Sasha already gave a link to the syscalls sequence, so no rush.
> 
> It'd be nice to get a more concise reproducer, his list had a little of everything in there.

I've so far failed to find any explanation for your swapops.h BUG;
but believe I have identified one cause for "Bad rss-counter"s.

My hunch is that the swapops.h BUG is "nearby", but I just cannot
fit it together (the swapops.h BUG comes when rmap cannot find all
all the migration entries it inserted earlier: it's a very useful
BUG for validating rmap).

Untested patch below: I can't quite say Reported-by, because it may
not even be one that you and Sasha have been seeing; but I'm hopeful,
remap_file_pages is in the list.

Please give this a try, preferably on 3.14-rc or earlier: I've never
seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
a lot more than most of us); but have seen them on mmotm/next, so
some other trigger is coming up there, I'll worry about that once
it reaches 3.15-rc.

(Cyrill, entirely unrelated, but in preparing this patch I noticed
your soft_dirty work in install_file_pte(): which looked good at
first, until I realized that it's propagating the soft_dirty of a
pte it's about to zap completely, to the unrelated entry it's about
to insert in its place.  Which seems very odd to me.)


[PATCH] mm: fix bad rss-counter if remap_file_pages raced migration

Fix some "Bad rss-counter state" reports on exit, arising from the
interaction between page migration and remap_file_pages(): zap_pte()
must count a migration entry when zapping it.

And yes, it is possible (though very unusual) to find an anon page or
swap entry in a VM_SHARED nonlinear mapping: coming from that horrid
get_user_pages(write, force) case which COWs even in a shared mapping.

Signed-off-by: Hugh Dickins <hughd@google.com>
---

 mm/fremap.c |   28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

--- 3.14-rc7/mm/fremap.c	2014-01-19 18:40:07.000000000 -0800
+++ linux/mm/fremap.c	2014-03-18 16:32:39.288612346 -0700
@@ -23,28 +23,44 @@
 
 #include "internal.h"
 
+static int mm_counter(struct page *page)
+{
+	return PageAnon(page) ? MM_ANONPAGES : MM_FILEPAGES;
+}
+
 static void zap_pte(struct mm_struct *mm, struct vm_area_struct *vma,
 			unsigned long addr, pte_t *ptep)
 {
 	pte_t pte = *ptep;
+	struct page *page;
+	swp_entry_t entry;
 
 	if (pte_present(pte)) {
-		struct page *page;
-
 		flush_cache_page(vma, addr, pte_pfn(pte));
 		pte = ptep_clear_flush(vma, addr, ptep);
 		page = vm_normal_page(vma, addr, pte);
 		if (page) {
 			if (pte_dirty(pte))
 				set_page_dirty(page);
+			update_hiwater_rss(mm);
+			dec_mm_counter(mm, mm_counter(page));
 			page_remove_rmap(page);
 			page_cache_release(page);
+		}
+	} else {	/* zap_pte() is not called when pte_none() */
+		if (!pte_file(pte)) {
 			update_hiwater_rss(mm);
-			dec_mm_counter(mm, MM_FILEPAGES);
+			entry = pte_to_swp_entry(pte);
+			if (non_swap_entry(entry)) {
+				if (is_migration_entry(entry)) {
+					page = migration_entry_to_page(entry);
+					dec_mm_counter(mm, mm_counter(page));
+				}
+			} else {
+				free_swap_and_cache(entry);
+				dec_mm_counter(mm, MM_SWAPENTS);
+			}
 		}
-	} else {
-		if (!pte_file(pte))
-			free_swap_and_cache(pte_to_swp_entry(pte));
 		pte_clear_not_present_full(mm, addr, ptep, 0);
 	}
 }

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 11 Mar 2014, Dave Jones wrote:
> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
>  > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>  > >  > 
>  > >  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
>  > >  > was passed, right? Share it too please.
>  > > 
>  > > Hm, I may have been mistaken, and the damage was done by a previous run.
>  > > I went from being able to reproduce it almost instantly to now not being able
>  > > to reproduce it at all.  Will keep trying.
>  > 
>  > Sasha already gave a link to the syscalls sequence, so no rush.
> 
> It'd be nice to get a more concise reproducer, his list had a little of everything in there.

I've so far failed to find any explanation for your swapops.h BUG;
but believe I have identified one cause for "Bad rss-counter"s.

My hunch is that the swapops.h BUG is "nearby", but I just cannot
fit it together (the swapops.h BUG comes when rmap cannot find all
all the migration entries it inserted earlier: it's a very useful
BUG for validating rmap).

Untested patch below: I can't quite say Reported-by, because it may
not even be one that you and Sasha have been seeing; but I'm hopeful,
remap_file_pages is in the list.

Please give this a try, preferably on 3.14-rc or earlier: I've never
seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
a lot more than most of us); but have seen them on mmotm/next, so
some other trigger is coming up there, I'll worry about that once
it reaches 3.15-rc.

(Cyrill, entirely unrelated, but in preparing this patch I noticed
your soft_dirty work in install_file_pte(): which looked good at
first, until I realized that it's propagating the soft_dirty of a
pte it's about to zap completely, to the unrelated entry it's about
to insert in its place.  Which seems very odd to me.)


[PATCH] mm: fix bad rss-counter if remap_file_pages raced migration

Fix some "Bad rss-counter state" reports on exit, arising from the
interaction between page migration and remap_file_pages(): zap_pte()
must count a migration entry when zapping it.

And yes, it is possible (though very unusual) to find an anon page or
swap entry in a VM_SHARED nonlinear mapping: coming from that horrid
get_user_pages(write, force) case which COWs even in a shared mapping.

Signed-off-by: Hugh Dickins <hughd@google.com>
---

 mm/fremap.c |   28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

--- 3.14-rc7/mm/fremap.c	2014-01-19 18:40:07.000000000 -0800
+++ linux/mm/fremap.c	2014-03-18 16:32:39.288612346 -0700
@@ -23,28 +23,44 @@
 
 #include "internal.h"
 
+static int mm_counter(struct page *page)
+{
+	return PageAnon(page) ? MM_ANONPAGES : MM_FILEPAGES;
+}
+
 static void zap_pte(struct mm_struct *mm, struct vm_area_struct *vma,
 			unsigned long addr, pte_t *ptep)
 {
 	pte_t pte = *ptep;
+	struct page *page;
+	swp_entry_t entry;
 
 	if (pte_present(pte)) {
-		struct page *page;
-
 		flush_cache_page(vma, addr, pte_pfn(pte));
 		pte = ptep_clear_flush(vma, addr, ptep);
 		page = vm_normal_page(vma, addr, pte);
 		if (page) {
 			if (pte_dirty(pte))
 				set_page_dirty(page);
+			update_hiwater_rss(mm);
+			dec_mm_counter(mm, mm_counter(page));
 			page_remove_rmap(page);
 			page_cache_release(page);
+		}
+	} else {	/* zap_pte() is not called when pte_none() */
+		if (!pte_file(pte)) {
 			update_hiwater_rss(mm);
-			dec_mm_counter(mm, MM_FILEPAGES);
+			entry = pte_to_swp_entry(pte);
+			if (non_swap_entry(entry)) {
+				if (is_migration_entry(entry)) {
+					page = migration_entry_to_page(entry);
+					dec_mm_counter(mm, mm_counter(page));
+				}
+			} else {
+				free_swap_and_cache(entry);
+				dec_mm_counter(mm, MM_SWAPENTS);
+			}
 		}
-	} else {
-		if (!pte_file(pte))
-			free_swap_and_cache(pte_to_swp_entry(pte));
 		pte_clear_not_present_full(mm, addr, ptep, 0);
 	}
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Linus Torvalds]

On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins <hughd@google.com> wrote:
>
> And yes, it is possible (though very unusual) to find an anon page or
> swap entry in a VM_SHARED nonlinear mapping: coming from that horrid
> get_user_pages(write, force) case which COWs even in a shared mapping.

Hmm. Maybe we could just disallow that forced case.

It *used* to be a trivial "we can just do a COW", but that was back
when the VM was much simpler and we had no rmap's etc. So "that horrid
case" used to be a simple hack that wasn't painful. But I suspect we
could very easily just fail it instead of forcing a COW, if that would
make it simpler for the VM code.

               Linus

Re: bad rss-counter message in 3.14rc5

[Linus Torvalds]

On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins <hughd@google.com> wrote:
>
> And yes, it is possible (though very unusual) to find an anon page or
> swap entry in a VM_SHARED nonlinear mapping: coming from that horrid
> get_user_pages(write, force) case which COWs even in a shared mapping.

Hmm. Maybe we could just disallow that forced case.

It *used* to be a trivial "we can just do a COW", but that was back
when the VM was much simpler and we had no rmap's etc. So "that horrid
case" used to be a simple hack that wasn't painful. But I suspect we
could very easily just fail it instead of forcing a COW, if that would
make it simpler for the VM code.

               Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Linus Torvalds wrote:
> On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins <hughd@google.com> wrote:
> >
> > And yes, it is possible (though very unusual) to find an anon page or
> > swap entry in a VM_SHARED nonlinear mapping: coming from that horrid
> > get_user_pages(write, force) case which COWs even in a shared mapping.
> 
> Hmm. Maybe we could just disallow that forced case.
> 
> It *used* to be a trivial "we can just do a COW", but that was back
> when the VM was much simpler and we had no rmap's etc. So "that horrid
> case" used to be a simple hack that wasn't painful. But I suspect we
> could very easily just fail it instead of forcing a COW, if that would
> make it simpler for the VM code.

I'd love that, if we can get away with it now: depends very
much on whether we then turn out to break userspace or not.

If I remember correctly, it's been that way since early days,
in case ptrace were used to put a breakpoint into a MAP_SHARED
mapping of an executable: to prevent that modification from
reaching the file, if the file happened to be opened O_RDWR.
Usually it's not open for writing, and mapped MAP_PRIVATE anyway.

That is still something worth protecting against, I presume;
but I'd much rather do it by failing the awkward case,
than by perverting the VM to break its own rules.

If I'm not mistaken, Konstantin (who happens to be already on this
Cc list) had a patch (that I hated) to complicate things, to fix up
some of the inconsistencies arising from this very odd and overlooked
corner-case.  I think he'd prefer this simplification to his patch too.

I'll look into it further, but not in haste.

Hugh

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Linus Torvalds wrote:
> On Tue, Mar 18, 2014 at 5:38 PM, Hugh Dickins <hughd@google.com> wrote:
> >
> > And yes, it is possible (though very unusual) to find an anon page or
> > swap entry in a VM_SHARED nonlinear mapping: coming from that horrid
> > get_user_pages(write, force) case which COWs even in a shared mapping.
> 
> Hmm. Maybe we could just disallow that forced case.
> 
> It *used* to be a trivial "we can just do a COW", but that was back
> when the VM was much simpler and we had no rmap's etc. So "that horrid
> case" used to be a simple hack that wasn't painful. But I suspect we
> could very easily just fail it instead of forcing a COW, if that would
> make it simpler for the VM code.

I'd love that, if we can get away with it now: depends very
much on whether we then turn out to break userspace or not.

If I remember correctly, it's been that way since early days,
in case ptrace were used to put a breakpoint into a MAP_SHARED
mapping of an executable: to prevent that modification from
reaching the file, if the file happened to be opened O_RDWR.
Usually it's not open for writing, and mapped MAP_PRIVATE anyway.

That is still something worth protecting against, I presume;
but I'd much rather do it by failing the awkward case,
than by perverting the VM to break its own rules.

If I'm not mistaken, Konstantin (who happens to be already on this
Cc list) had a patch (that I hated) to complicate things, to fix up
some of the inconsistencies arising from this very odd and overlooked
corner-case.  I think he'd prefer this simplification to his patch too.

I'll look into it further, but not in haste.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Linus Torvalds]

On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins <hughd@google.com> wrote:
>
> I'd love that, if we can get away with it now: depends very
> much on whether we then turn out to break userspace or not.

Right. I suspect we can, though, but it's one of those "we can try it
and see". Remind me early in the 3.15 merge window, and we can just
turn the "force" case into an error case and see if anybody hollers.

> If I remember correctly, it's been that way since early days,
> in case ptrace were used to put a breakpoint into a MAP_SHARED
> mapping of an executable: to prevent that modification from
> reaching the file, if the file happened to be opened O_RDWR.
> Usually it's not open for writing, and mapped MAP_PRIVATE anyway.

Yes, it's been that way since the very beginning, I think it goes back
pretty much as far as MAP_SHARED does.

We used to play lots of games wrt MAP_SHARED - in fact I think we used
to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for
the longest time there was no "true" writable MAP_SHARED at all, but
we did have a coherent MAP_PRIVATE and something like the indexer for
nntpd wanted a read-only shared mapping of the nntp spool or something
like that. I forget the details, it's a _loong_ time ago.

So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used
to make a lot more sense in that kind of situation, when MAP_SHARED vs
MAP_PRIVATE was much less of a black-and-white thing.

I really suspect nobody cares wrt ptrace, especially since presumably
other systems haven't had those kinds of games (although who knows -
HP-UX in particular had some of the shittiest mmap() implementations
on the planet - it made even the original Linux mmap hacks look like a
thing of pure beauty in comparison).

              Linus

Re: bad rss-counter message in 3.14rc5

[Linus Torvalds]

On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins <hughd@google.com> wrote:
>
> I'd love that, if we can get away with it now: depends very
> much on whether we then turn out to break userspace or not.

Right. I suspect we can, though, but it's one of those "we can try it
and see". Remind me early in the 3.15 merge window, and we can just
turn the "force" case into an error case and see if anybody hollers.

> If I remember correctly, it's been that way since early days,
> in case ptrace were used to put a breakpoint into a MAP_SHARED
> mapping of an executable: to prevent that modification from
> reaching the file, if the file happened to be opened O_RDWR.
> Usually it's not open for writing, and mapped MAP_PRIVATE anyway.

Yes, it's been that way since the very beginning, I think it goes back
pretty much as far as MAP_SHARED does.

We used to play lots of games wrt MAP_SHARED - in fact I think we used
to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for
the longest time there was no "true" writable MAP_SHARED at all, but
we did have a coherent MAP_PRIVATE and something like the indexer for
nntpd wanted a read-only shared mapping of the nntp spool or something
like that. I forget the details, it's a _loong_ time ago.

So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used
to make a lot more sense in that kind of situation, when MAP_SHARED vs
MAP_PRIVATE was much less of a black-and-white thing.

I really suspect nobody cares wrt ptrace, especially since presumably
other systems haven't had those kinds of games (although who knows -
HP-UX in particular had some of the shittiest mmap() implementations
on the planet - it made even the original Linux mmap hacks look like a
thing of pure beauty in comparison).

              Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Linus Torvalds wrote:
> On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins <hughd@google.com> wrote:
> >
> > I'd love that, if we can get away with it now: depends very
> > much on whether we then turn out to break userspace or not.
> 
> Right. I suspect we can, though, but it's one of those "we can try it
> and see". Remind me early in the 3.15 merge window, and we can just
> turn the "force" case into an error case and see if anybody hollers.

Super, I'll do that, thanks.

For 3.15, and probably 3.16 too, we should keep in place whatever
partial accommodations we have for the case (such as allowing for
anon and swap in fremap's zap_pte), in case we do need to revert;
but clean those away later on.  (Not many, I think: it was mainly
a guilty secret that VM accounting didn't really hold together.)

> 
> > If I remember correctly, it's been that way since early days,
> > in case ptrace were used to put a breakpoint into a MAP_SHARED
> > mapping of an executable: to prevent that modification from
> > reaching the file, if the file happened to be opened O_RDWR.
> > Usually it's not open for writing, and mapped MAP_PRIVATE anyway.
> 
> Yes, it's been that way since the very beginning, I think it goes back
> pretty much as far as MAP_SHARED does.
> 
> We used to play lots of games wrt MAP_SHARED - in fact I think we used
> to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for
> the longest time there was no "true" writable MAP_SHARED at all, but
> we did have a coherent MAP_PRIVATE and something like the indexer for
> nntpd wanted a read-only shared mapping of the nntp spool or something
> like that. I forget the details, it's a _loong_ time ago.
> 
> So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used
> to make a lot more sense in that kind of situation, when MAP_SHARED vs
> MAP_PRIVATE was much less of a black-and-white thing.
> 
> I really suspect nobody cares wrt ptrace, especially since presumably
> other systems haven't had those kinds of games (although who knows -
> HP-UX in particular had some of the shittiest mmap() implementations
> on the planet - it made even the original Linux mmap hacks look like a
> thing of pure beauty in comparison).

:)  That fits with what I heard of HP-UX mmap,
but I never had the pleasure of dealing with it.

Hugh

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Linus Torvalds wrote:
> On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins <hughd@google.com> wrote:
> >
> > I'd love that, if we can get away with it now: depends very
> > much on whether we then turn out to break userspace or not.
> 
> Right. I suspect we can, though, but it's one of those "we can try it
> and see". Remind me early in the 3.15 merge window, and we can just
> turn the "force" case into an error case and see if anybody hollers.

Super, I'll do that, thanks.

For 3.15, and probably 3.16 too, we should keep in place whatever
partial accommodations we have for the case (such as allowing for
anon and swap in fremap's zap_pte), in case we do need to revert;
but clean those away later on.  (Not many, I think: it was mainly
a guilty secret that VM accounting didn't really hold together.)

> 
> > If I remember correctly, it's been that way since early days,
> > in case ptrace were used to put a breakpoint into a MAP_SHARED
> > mapping of an executable: to prevent that modification from
> > reaching the file, if the file happened to be opened O_RDWR.
> > Usually it's not open for writing, and mapped MAP_PRIVATE anyway.
> 
> Yes, it's been that way since the very beginning, I think it goes back
> pretty much as far as MAP_SHARED does.
> 
> We used to play lots of games wrt MAP_SHARED - in fact I think we used
> to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for
> the longest time there was no "true" writable MAP_SHARED at all, but
> we did have a coherent MAP_PRIVATE and something like the indexer for
> nntpd wanted a read-only shared mapping of the nntp spool or something
> like that. I forget the details, it's a _loong_ time ago.
> 
> So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used
> to make a lot more sense in that kind of situation, when MAP_SHARED vs
> MAP_PRIVATE was much less of a black-and-white thing.
> 
> I really suspect nobody cares wrt ptrace, especially since presumably
> other systems haven't had those kinds of games (although who knows -
> HP-UX in particular had some of the shittiest mmap() implementations
> on the planet - it made even the original Linux mmap hacks look like a
> thing of pure beauty in comparison).

:)  That fits with what I heard of HP-UX mmap,
but I never had the pleasure of dealing with it.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Linus Torvalds]

On Tue, Mar 18, 2014 at 7:37 PM, Hugh Dickins <hughd@google.com> wrote:
>
> For 3.15, and probably 3.16 too, we should keep in place whatever
> partial accommodations we have for the case (such as allowing for
> anon and swap in fremap's zap_pte), in case we do need to revert;
> but clean those away later on.  (Not many, I think: it was mainly
> a guilty secret that VM accounting didn't really hold together.)

Absolutely. See if it works to just stop doing that special COW, and
then later on, if we have decided "nobody even noticed", we can remove
the hacks we have to support the fact that shared mappings sometimes
have anon pages in them.

> :)  That fits with what I heard of HP-UX mmap,
> but I never had the pleasure of dealing with it.

They had purely virtually indexed caches, making coherency
"interesting". Together with a VM based on some really old BSD VM code
that everybody else had thrown out, and that didn't allow you to unmap
things partially etc. So HPUX mmap really didn't work, not even for
non-shared mmap's.

I think they fixed the interfaces in HP-UX 11. But not being coherent
meant that the shared mappings tended to still have trouble. nntp
largely died, but was replaced with the cyrus imapd that played
similar games.

At least out mmap was always coherent. Even in MAP_PRIVATE, and with
regards to both write() system calls and other mmap PROT_WRITE users.

Except when we had bugs. Shared mmap really isn't very simple to get right.

                 Linus

Re: bad rss-counter message in 3.14rc5

[Linus Torvalds]

On Tue, Mar 18, 2014 at 7:37 PM, Hugh Dickins <hughd@google.com> wrote:
>
> For 3.15, and probably 3.16 too, we should keep in place whatever
> partial accommodations we have for the case (such as allowing for
> anon and swap in fremap's zap_pte), in case we do need to revert;
> but clean those away later on.  (Not many, I think: it was mainly
> a guilty secret that VM accounting didn't really hold together.)

Absolutely. See if it works to just stop doing that special COW, and
then later on, if we have decided "nobody even noticed", we can remove
the hacks we have to support the fact that shared mappings sometimes
have anon pages in them.

> :)  That fits with what I heard of HP-UX mmap,
> but I never had the pleasure of dealing with it.

They had purely virtually indexed caches, making coherency
"interesting". Together with a VM based on some really old BSD VM code
that everybody else had thrown out, and that didn't allow you to unmap
things partially etc. So HPUX mmap really didn't work, not even for
non-shared mmap's.

I think they fixed the interfaces in HP-UX 11. But not being coherent
meant that the shared mappings tended to still have trouble. nntp
largely died, but was replaced with the cyrus imapd that played
similar games.

At least out mmap was always coherent. Even in MAP_PRIVATE, and with
regards to both write() system calls and other mmap PROT_WRITE users.

Except when we had bugs. Shared mmap really isn't very simple to get right.

                 Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Jan Kara]

On Tue 18-03-14 19:37:01, Hugh Dickins wrote:
> On Tue, 18 Mar 2014, Linus Torvalds wrote:
> > On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins <hughd@google.com> wrote:
> > >
> > > I'd love that, if we can get away with it now: depends very
> > > much on whether we then turn out to break userspace or not.
> > 
> > Right. I suspect we can, though, but it's one of those "we can try it
> > and see". Remind me early in the 3.15 merge window, and we can just
> > turn the "force" case into an error case and see if anybody hollers.
> 
> Super, I'll do that, thanks.
> 
> For 3.15, and probably 3.16 too, we should keep in place whatever
> partial accommodations we have for the case (such as allowing for
> anon and swap in fremap's zap_pte), in case we do need to revert;
> but clean those away later on.  (Not many, I think: it was mainly
> a guilty secret that VM accounting didn't really hold together.)
  Different drivers actually use the 'force' argument of get_user_pages() a
lot on userspace provided buffers (AFAIU because they want to tell the
kernel HW is going to write to that memory so they want to prepare for it).
It is hard to imagine someone will use this for MAP_SHARED pages (or what
that would be supposed to achieve) but sometimes userspace is surprisingly
inventive... Just something to be aware of...

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: bad rss-counter message in 3.14rc5

[Jan Kara]

On Tue 18-03-14 19:37:01, Hugh Dickins wrote:
> On Tue, 18 Mar 2014, Linus Torvalds wrote:
> > On Tue, Mar 18, 2014 at 7:06 PM, Hugh Dickins <hughd@google.com> wrote:
> > >
> > > I'd love that, if we can get away with it now: depends very
> > > much on whether we then turn out to break userspace or not.
> > 
> > Right. I suspect we can, though, but it's one of those "we can try it
> > and see". Remind me early in the 3.15 merge window, and we can just
> > turn the "force" case into an error case and see if anybody hollers.
> 
> Super, I'll do that, thanks.
> 
> For 3.15, and probably 3.16 too, we should keep in place whatever
> partial accommodations we have for the case (such as allowing for
> anon and swap in fremap's zap_pte), in case we do need to revert;
> but clean those away later on.  (Not many, I think: it was mainly
> a guilty secret that VM accounting didn't really hold together.)
  Different drivers actually use the 'force' argument of get_user_pages() a
lot on userspace provided buffers (AFAIU because they want to tell the
kernel HW is going to write to that memory so they want to prepare for it).
It is hard to imagine someone will use this for MAP_SHARED pages (or what
that would be supposed to achieve) but sometimes userspace is surprisingly
inventive... Just something to be aware of...

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/18/2014 08:38 PM, Hugh Dickins wrote:
> On Tue, 11 Mar 2014, Dave Jones wrote:
>> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
>>   > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>>   > >  >
>>   > >  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
>>   > >  > was passed, right? Share it too please.
>>   > >
>>   > > Hm, I may have been mistaken, and the damage was done by a previous run.
>>   > > I went from being able to reproduce it almost instantly to now not being able
>>   > > to reproduce it at all.  Will keep trying.
>>   >
>>   > Sasha already gave a link to the syscalls sequence, so no rush.
>>
>> It'd be nice to get a more concise reproducer, his list had a little of everything in there.
>
> I've so far failed to find any explanation for your swapops.h BUG;
> but believe I have identified one cause for "Bad rss-counter"s.
>
> My hunch is that the swapops.h BUG is "nearby", but I just cannot
> fit it together (the swapops.h BUG comes when rmap cannot find all
> all the migration entries it inserted earlier: it's a very useful
> BUG for validating rmap).
>
> Untested patch below: I can't quite say Reported-by, because it may
> not even be one that you and Sasha have been seeing; but I'm hopeful,
> remap_file_pages is in the list.
>
> Please give this a try, preferably on 3.14-rc or earlier: I've never
> seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
> a lot more than most of us); but have seen them on mmotm/next, so
> some other trigger is coming up there, I'll worry about that once
> it reaches 3.15-rc.

The patch fixed the "Bad rss-counter" errors I've been seeing both in
3.14-rc7 and -next.


Thanks,
Sasha

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/18/2014 08:38 PM, Hugh Dickins wrote:
> On Tue, 11 Mar 2014, Dave Jones wrote:
>> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
>>   > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>>   > >  >
>>   > >  > Dave, iirc trinity can write log file pointing which exactly syscall sequence
>>   > >  > was passed, right? Share it too please.
>>   > >
>>   > > Hm, I may have been mistaken, and the damage was done by a previous run.
>>   > > I went from being able to reproduce it almost instantly to now not being able
>>   > > to reproduce it at all.  Will keep trying.
>>   >
>>   > Sasha already gave a link to the syscalls sequence, so no rush.
>>
>> It'd be nice to get a more concise reproducer, his list had a little of everything in there.
>
> I've so far failed to find any explanation for your swapops.h BUG;
> but believe I have identified one cause for "Bad rss-counter"s.
>
> My hunch is that the swapops.h BUG is "nearby", but I just cannot
> fit it together (the swapops.h BUG comes when rmap cannot find all
> all the migration entries it inserted earlier: it's a very useful
> BUG for validating rmap).
>
> Untested patch below: I can't quite say Reported-by, because it may
> not even be one that you and Sasha have been seeing; but I'm hopeful,
> remap_file_pages is in the list.
>
> Please give this a try, preferably on 3.14-rc or earlier: I've never
> seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
> a lot more than most of us); but have seen them on mmotm/next, so
> some other trigger is coming up there, I'll worry about that once
> it reaches 3.15-rc.

The patch fixed the "Bad rss-counter" errors I've been seeing both in
3.14-rc7 and -next.


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote:
 
 > > Untested patch below: I can't quite say Reported-by, because it may
 > > not even be one that you and Sasha have been seeing; but I'm hopeful,
 > > remap_file_pages is in the list.
 > >
 > > Please give this a try, preferably on 3.14-rc or earlier: I've never
 > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
 > > a lot more than most of us); but have seen them on mmotm/next, so
 > > some other trigger is coming up there, I'll worry about that once
 > > it reaches 3.15-rc.
 > 
 > The patch fixed the "Bad rss-counter" errors I've been seeing both in
 > 3.14-rc7 and -next.
 
It's looking good here too so far. I'll leave it running overnight to be sure.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote:
 
 > > Untested patch below: I can't quite say Reported-by, because it may
 > > not even be one that you and Sasha have been seeing; but I'm hopeful,
 > > remap_file_pages is in the list.
 > >
 > > Please give this a try, preferably on 3.14-rc or earlier: I've never
 > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
 > > a lot more than most of us); but have seen them on mmotm/next, so
 > > some other trigger is coming up there, I'll worry about that once
 > > it reaches 3.15-rc.
 > 
 > The patch fixed the "Bad rss-counter" errors I've been seeing both in
 > 3.14-rc7 and -next.
 
It's looking good here too so far. I'll leave it running overnight to be sure.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote:
 > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote:
 >  
 >  > > Untested patch below: I can't quite say Reported-by, because it may
 >  > > not even be one that you and Sasha have been seeing; but I'm hopeful,
 >  > > remap_file_pages is in the list.
 >  > >
 >  > > Please give this a try, preferably on 3.14-rc or earlier: I've never
 >  > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
 >  > > a lot more than most of us); but have seen them on mmotm/next, so
 >  > > some other trigger is coming up there, I'll worry about that once
 >  > > it reaches 3.15-rc.
 >  > 
 >  > The patch fixed the "Bad rss-counter" errors I've been seeing both in
 >  > 3.14-rc7 and -next.
 >  
 > It's looking good here too so far. I'll leave it running overnight to be sure.

Of course, that isn't going to happen. Immediately after posting this, I hit the
swapops bug.  Patch does seem to have cured the bad rss counters though.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote:
 > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote:
 >  
 >  > > Untested patch below: I can't quite say Reported-by, because it may
 >  > > not even be one that you and Sasha have been seeing; but I'm hopeful,
 >  > > remap_file_pages is in the list.
 >  > >
 >  > > Please give this a try, preferably on 3.14-rc or earlier: I've never
 >  > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
 >  > > a lot more than most of us); but have seen them on mmotm/next, so
 >  > > some other trigger is coming up there, I'll worry about that once
 >  > > it reaches 3.15-rc.
 >  > 
 >  > The patch fixed the "Bad rss-counter" errors I've been seeing both in
 >  > 3.14-rc7 and -next.
 >  
 > It's looking good here too so far. I'll leave it running overnight to be sure.

Of course, that isn't going to happen. Immediately after posting this, I hit the
swapops bug.  Patch does seem to have cured the bad rss counters though.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Dave Jones wrote:
> On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote:
>  > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote:
>  >  
>  >  > > Untested patch below: I can't quite say Reported-by, because it may
>  >  > > not even be one that you and Sasha have been seeing; but I'm hopeful,
>  >  > > remap_file_pages is in the list.
>  >  > >
>  >  > > Please give this a try, preferably on 3.14-rc or earlier: I've never
>  >  > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
>  >  > > a lot more than most of us); but have seen them on mmotm/next, so
>  >  > > some other trigger is coming up there, I'll worry about that once
>  >  > > it reaches 3.15-rc.
>  >  > 
>  >  > The patch fixed the "Bad rss-counter" errors I've been seeing both in
>  >  > 3.14-rc7 and -next.
>  >  
>  > It's looking good here too so far. I'll leave it running overnight to be sure.
> 
> Of course, that isn't going to happen. Immediately after posting this, I hit the
> swapops bug.  Patch does seem to have cured the bad rss counters though.

Another positive on the rss counters, great, thanks Dave.
That encourages me to think again on the swapops BUG, but no promises.

Hugh

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Dave Jones wrote:
> On Tue, Mar 18, 2014 at 10:06:02PM -0400, Dave Jones wrote:
>  > On Tue, Mar 18, 2014 at 09:32:36PM -0400, Sasha Levin wrote:
>  >  
>  >  > > Untested patch below: I can't quite say Reported-by, because it may
>  >  > > not even be one that you and Sasha have been seeing; but I'm hopeful,
>  >  > > remap_file_pages is in the list.
>  >  > >
>  >  > > Please give this a try, preferably on 3.14-rc or earlier: I've never
>  >  > > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
>  >  > > a lot more than most of us); but have seen them on mmotm/next, so
>  >  > > some other trigger is coming up there, I'll worry about that once
>  >  > > it reaches 3.15-rc.
>  >  > 
>  >  > The patch fixed the "Bad rss-counter" errors I've been seeing both in
>  >  > 3.14-rc7 and -next.
>  >  
>  > It's looking good here too so far. I'll leave it running overnight to be sure.
> 
> Of course, that isn't going to happen. Immediately after posting this, I hit the
> swapops bug.  Patch does seem to have cured the bad rss counters though.

Another positive on the rss counters, great, thanks Dave.
That encourages me to think again on the swapops BUG, but no promises.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote:

 > Another positive on the rss counters, great, thanks Dave.
 > That encourages me to think again on the swapops BUG, but no promises.

So while I slept I ran a test kernel with that swapops BUG replaced with a printk.
I'm not sure of the validity of this, given the state of the kernel afterwards
is somewhat suspect, but I did see in the logs this morning..

[18728.075153] migration_entry_to_page BUG hit
[18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted)
[18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted)

This might be collateral damage from the swapops thing, I guess we won't know until
that gets fixed, but I thought I'd mention that we might still have a problem here.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote:

 > Another positive on the rss counters, great, thanks Dave.
 > That encourages me to think again on the swapops BUG, but no promises.

So while I slept I ran a test kernel with that swapops BUG replaced with a printk.
I'm not sure of the validity of this, given the state of the kernel afterwards
is somewhat suspect, but I did see in the logs this morning..

[18728.075153] migration_entry_to_page BUG hit
[18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted)
[18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted)

This might be collateral damage from the swapops thing, I guess we won't know until
that gets fixed, but I thought I'd mention that we might still have a problem here.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Wed, 19 Mar 2014, Dave Jones wrote:
> On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote:
> 
>  > Another positive on the rss counters, great, thanks Dave.
>  > That encourages me to think again on the swapops BUG, but no promises.
> 
> So while I slept I ran a test kernel with that swapops BUG replaced with a printk.
> I'm not sure of the validity of this, given the state of the kernel afterwards
> is somewhat suspect, but I did see in the logs this morning..
> 
> [18728.075153] migration_entry_to_page BUG hit
> [18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted)
> [18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted)
> 
> This might be collateral damage from the swapops thing, I guess we won't know until
> that gets fixed, but I thought I'd mention that we might still have a problem here.

Yes, those Bad rss-counters could well be collateral damage from the
swapops BUG.  To which I believe I now have the answer: again untested,
but please give this a try...

(It's worth saying, by the way, that these bugs are not a consequence
of recent changes at all, they've been there for ages; but trinity has
just got better at taunting remap_file_pages and the rest of mm...)


[PATCH] mm: fix swapops.h:131 bug if remap_file_pages raced migration

Add remove_linear_migration_ptes_from_nonlinear(), to fix an interesting
little include/linux/swapops.h:131 BUG_ON(!PageLocked) found by trinity:
indicating that remove_migration_ptes() failed to find one of the
migration entries that was temporarily inserted.

The problem comes from remap_file_pages()'s switch from vma_interval_tree
(good for inserting the migration entry) to i_mmap_nonlinear list (no good
for locating it again); but can only be a problem if the remap_file_pages()
range does not cover the whole of the vma (zap_pte() clears the range).

remove_migration_ptes() needs a file_nonlinear method to go down the
i_mmap_nonlinear list, applying linear location to look for migration
entries in those vmas too, just in case there was this race.

The file_nonlinear method does need rmap_walk_control.arg to do this;
but it never needed vma passed in - vma comes from its own iteration.

Signed-off-by: Hugh Dickins <hughd@google.com>
---

 include/linux/rmap.h |    3 +--
 mm/migrate.c         |   32 ++++++++++++++++++++++++++++++++
 mm/rmap.c            |    5 +++--
 3 files changed, 36 insertions(+), 4 deletions(-)

--- 3.14-rc7/include/linux/rmap.h	2014-02-02 18:49:07.429302104 -0800
+++ linux/include/linux/rmap.h	2014-03-19 20:12:27.056451541 -0700
@@ -250,8 +250,7 @@ struct rmap_walk_control {
 	int (*rmap_one)(struct page *page, struct vm_area_struct *vma,
 					unsigned long addr, void *arg);
 	int (*done)(struct page *page);
-	int (*file_nonlinear)(struct page *, struct address_space *,
-					struct vm_area_struct *vma);
+	int (*file_nonlinear)(struct page *, struct address_space *, void *arg);
 	struct anon_vma *(*anon_lock)(struct page *page);
 	bool (*invalid_vma)(struct vm_area_struct *vma, void *arg);
 };
--- 3.14-rc7/mm/migrate.c	2014-03-16 19:24:19.635512576 -0700
+++ linux/mm/migrate.c	2014-03-19 21:06:02.704527965 -0700
@@ -178,6 +178,37 @@ out:
 }
 
 /*
+ * Congratulations to trinity for discovering this bug.
+ * mm/fremap.c's remap_file_pages() accepts any range within a single vma to
+ * convert that vma to VM_NONLINEAR; and generic_file_remap_pages() will then
+ * replace the specified range by file ptes throughout (maybe populated after).
+ * If page migration finds a page within that range, while it's still located
+ * by vma_interval_tree rather than lost to i_mmap_nonlinear list, no problem:
+ * zap_pte() clears the temporary migration entry before mmap_sem is dropped.
+ * But if the migrating page is in a part of the vma outside the range to be
+ * remapped, then it will not be cleared, and remove_migration_ptes() needs to
+ * deal with it.  Fortunately, this part of the vma is of course still linear,
+ * so we just need to use linear location on the nonlinear list.
+ */
+static int remove_linear_migration_ptes_from_nonlinear(struct page *page,
+		struct address_space *mapping, void *arg)
+{
+	struct vm_area_struct *vma;
+	/* hugetlbfs does not support remap_pages, so no huge pgoff worries */
+	pgoff_t pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT);
+	unsigned long addr;
+
+	list_for_each_entry(vma,
+		&mapping->i_mmap_nonlinear, shared.nonlinear) {
+
+		addr = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT);
+		if (addr >= vma->vm_start && addr < vma->vm_end)
+			remove_migration_pte(page, vma, addr, arg);
+	}
+	return SWAP_AGAIN;
+}
+
+/*
  * Get rid of all migration entries and replace them by
  * references to the indicated page.
  */
@@ -186,6 +217,7 @@ static void remove_migration_ptes(struct
 	struct rmap_walk_control rwc = {
 		.rmap_one = remove_migration_pte,
 		.arg = old,
+		.file_nonlinear = remove_linear_migration_ptes_from_nonlinear,
 	};
 
 	rmap_walk(new, &rwc);
--- 3.14-rc7/mm/rmap.c	2014-02-02 18:49:07.929302115 -0800
+++ linux/mm/rmap.c	2014-03-19 20:16:03.552456686 -0700
@@ -1360,8 +1360,9 @@ static int try_to_unmap_cluster(unsigned
 }
 
 static int try_to_unmap_nonlinear(struct page *page,
-		struct address_space *mapping, struct vm_area_struct *vma)
+		struct address_space *mapping, void *arg)
 {
+	struct vm_area_struct *vma;
 	int ret = SWAP_AGAIN;
 	unsigned long cursor;
 	unsigned long max_nl_cursor = 0;
@@ -1663,7 +1664,7 @@ static int rmap_walk_file(struct page *p
 	if (list_empty(&mapping->i_mmap_nonlinear))
 		goto done;
 
-	ret = rwc->file_nonlinear(page, mapping, vma);
+	ret = rwc->file_nonlinear(page, mapping, rwc->arg);
 
 done:
 	mutex_unlock(&mapping->i_mmap_mutex);

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Wed, 19 Mar 2014, Dave Jones wrote:
> On Tue, Mar 18, 2014 at 07:19:09PM -0700, Hugh Dickins wrote:
> 
>  > Another positive on the rss counters, great, thanks Dave.
>  > That encourages me to think again on the swapops BUG, but no promises.
> 
> So while I slept I ran a test kernel with that swapops BUG replaced with a printk.
> I'm not sure of the validity of this, given the state of the kernel afterwards
> is somewhat suspect, but I did see in the logs this morning..
> 
> [18728.075153] migration_entry_to_page BUG hit
> [18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted)
> [18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted)
> 
> This might be collateral damage from the swapops thing, I guess we won't know until
> that gets fixed, but I thought I'd mention that we might still have a problem here.

Yes, those Bad rss-counters could well be collateral damage from the
swapops BUG.  To which I believe I now have the answer: again untested,
but please give this a try...

(It's worth saying, by the way, that these bugs are not a consequence
of recent changes at all, they've been there for ages; but trinity has
just got better at taunting remap_file_pages and the rest of mm...)


[PATCH] mm: fix swapops.h:131 bug if remap_file_pages raced migration

Add remove_linear_migration_ptes_from_nonlinear(), to fix an interesting
little include/linux/swapops.h:131 BUG_ON(!PageLocked) found by trinity:
indicating that remove_migration_ptes() failed to find one of the
migration entries that was temporarily inserted.

The problem comes from remap_file_pages()'s switch from vma_interval_tree
(good for inserting the migration entry) to i_mmap_nonlinear list (no good
for locating it again); but can only be a problem if the remap_file_pages()
range does not cover the whole of the vma (zap_pte() clears the range).

remove_migration_ptes() needs a file_nonlinear method to go down the
i_mmap_nonlinear list, applying linear location to look for migration
entries in those vmas too, just in case there was this race.

The file_nonlinear method does need rmap_walk_control.arg to do this;
but it never needed vma passed in - vma comes from its own iteration.

Signed-off-by: Hugh Dickins <hughd@google.com>
---

 include/linux/rmap.h |    3 +--
 mm/migrate.c         |   32 ++++++++++++++++++++++++++++++++
 mm/rmap.c            |    5 +++--
 3 files changed, 36 insertions(+), 4 deletions(-)

--- 3.14-rc7/include/linux/rmap.h	2014-02-02 18:49:07.429302104 -0800
+++ linux/include/linux/rmap.h	2014-03-19 20:12:27.056451541 -0700
@@ -250,8 +250,7 @@ struct rmap_walk_control {
 	int (*rmap_one)(struct page *page, struct vm_area_struct *vma,
 					unsigned long addr, void *arg);
 	int (*done)(struct page *page);
-	int (*file_nonlinear)(struct page *, struct address_space *,
-					struct vm_area_struct *vma);
+	int (*file_nonlinear)(struct page *, struct address_space *, void *arg);
 	struct anon_vma *(*anon_lock)(struct page *page);
 	bool (*invalid_vma)(struct vm_area_struct *vma, void *arg);
 };
--- 3.14-rc7/mm/migrate.c	2014-03-16 19:24:19.635512576 -0700
+++ linux/mm/migrate.c	2014-03-19 21:06:02.704527965 -0700
@@ -178,6 +178,37 @@ out:
 }
 
 /*
+ * Congratulations to trinity for discovering this bug.
+ * mm/fremap.c's remap_file_pages() accepts any range within a single vma to
+ * convert that vma to VM_NONLINEAR; and generic_file_remap_pages() will then
+ * replace the specified range by file ptes throughout (maybe populated after).
+ * If page migration finds a page within that range, while it's still located
+ * by vma_interval_tree rather than lost to i_mmap_nonlinear list, no problem:
+ * zap_pte() clears the temporary migration entry before mmap_sem is dropped.
+ * But if the migrating page is in a part of the vma outside the range to be
+ * remapped, then it will not be cleared, and remove_migration_ptes() needs to
+ * deal with it.  Fortunately, this part of the vma is of course still linear,
+ * so we just need to use linear location on the nonlinear list.
+ */
+static int remove_linear_migration_ptes_from_nonlinear(struct page *page,
+		struct address_space *mapping, void *arg)
+{
+	struct vm_area_struct *vma;
+	/* hugetlbfs does not support remap_pages, so no huge pgoff worries */
+	pgoff_t pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT);
+	unsigned long addr;
+
+	list_for_each_entry(vma,
+		&mapping->i_mmap_nonlinear, shared.nonlinear) {
+
+		addr = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT);
+		if (addr >= vma->vm_start && addr < vma->vm_end)
+			remove_migration_pte(page, vma, addr, arg);
+	}
+	return SWAP_AGAIN;
+}
+
+/*
  * Get rid of all migration entries and replace them by
  * references to the indicated page.
  */
@@ -186,6 +217,7 @@ static void remove_migration_ptes(struct
 	struct rmap_walk_control rwc = {
 		.rmap_one = remove_migration_pte,
 		.arg = old,
+		.file_nonlinear = remove_linear_migration_ptes_from_nonlinear,
 	};
 
 	rmap_walk(new, &rwc);
--- 3.14-rc7/mm/rmap.c	2014-02-02 18:49:07.929302115 -0800
+++ linux/mm/rmap.c	2014-03-19 20:16:03.552456686 -0700
@@ -1360,8 +1360,9 @@ static int try_to_unmap_cluster(unsigned
 }
 
 static int try_to_unmap_nonlinear(struct page *page,
-		struct address_space *mapping, struct vm_area_struct *vma)
+		struct address_space *mapping, void *arg)
 {
+	struct vm_area_struct *vma;
 	int ret = SWAP_AGAIN;
 	unsigned long cursor;
 	unsigned long max_nl_cursor = 0;
@@ -1663,7 +1664,7 @@ static int rmap_walk_file(struct page *p
 	if (list_empty(&mapping->i_mmap_nonlinear))
 		goto done;
 
-	ret = rwc->file_nonlinear(page, mapping, vma);
+	ret = rwc->file_nonlinear(page, mapping, rwc->arg);
 
 done:
 	mutex_unlock(&mapping->i_mmap_mutex);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote:

 > > This might be collateral damage from the swapops thing, I guess we won't know until
 > > that gets fixed, but I thought I'd mention that we might still have a problem here.
 > 
 > Yes, those Bad rss-counters could well be collateral damage from the
 > swapops BUG.  To which I believe I now have the answer: again untested,
 > but please give this a try...

This survived an overnight run. No swapops bug, and no bad RSS. Good job :)

 > (It's worth saying, by the way, that these bugs are not a consequence
 > of recent changes at all, they've been there for ages; but trinity has
 > just got better at taunting remap_file_pages and the rest of mm...)

Indeed. I hope to lift the covers on more stuff like this (and hopefully
get it done in a more reproducable manner).  A lot of the stuff trinity
is doing with VM syscalls is still very naive.

	Dave

Re: bad rss-counter message in 3.14rc5

[Dave Jones]

On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote:

 > > This might be collateral damage from the swapops thing, I guess we won't know until
 > > that gets fixed, but I thought I'd mention that we might still have a problem here.
 > 
 > Yes, those Bad rss-counters could well be collateral damage from the
 > swapops BUG.  To which I believe I now have the answer: again untested,
 > but please give this a try...

This survived an overnight run. No swapops bug, and no bad RSS. Good job :)

 > (It's worth saying, by the way, that these bugs are not a consequence
 > of recent changes at all, they've been there for ages; but trinity has
 > just got better at taunting remap_file_pages and the rest of mm...)

Indeed. I hope to lift the covers on more stuff like this (and hopefully
get it done in a more reproducable manner).  A lot of the stuff trinity
is doing with VM syscalls is still very naive.

	Dave

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/20/2014 09:51 AM, Dave Jones wrote:
> On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote:
>
>   > > This might be collateral damage from the swapops thing, I guess we won't know until
>   > > that gets fixed, but I thought I'd mention that we might still have a problem here.
>   >
>   > Yes, those Bad rss-counters could well be collateral damage from the
>   > swapops BUG.  To which I believe I now have the answer: again untested,
>   > but please give this a try...
>
> This survived an overnight run. No swapops bug, and no bad RSS. Good job:)

Same here, swapops bug is gone!


Thanks,
Sasha

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/20/2014 09:51 AM, Dave Jones wrote:
> On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote:
>
>   > > This might be collateral damage from the swapops thing, I guess we won't know until
>   > > that gets fixed, but I thought I'd mention that we might still have a problem here.
>   >
>   > Yes, those Bad rss-counters could well be collateral damage from the
>   > swapops BUG.  To which I believe I now have the answer: again untested,
>   > but please give this a try...
>
> This survived an overnight run. No swapops bug, and no bad RSS. Good job:)

Same here, swapops bug is gone!


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Thu, 20 Mar 2014, Sasha Levin wrote:
> On 03/20/2014 09:51 AM, Dave Jones wrote:
> > On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote:
> > 
> >   > > This might be collateral damage from the swapops thing, I guess we
> > won't know until
> >   > > that gets fixed, but I thought I'd mention that we might still have a
> > problem here.
> >   >
> >   > Yes, those Bad rss-counters could well be collateral damage from the
> >   > swapops BUG.  To which I believe I now have the answer: again untested,
> >   > but please give this a try...
> > 
> > This survived an overnight run. No swapops bug, and no bad RSS. Good job:)
> 
> Same here, swapops bug is gone!

That was welcome news, thanks guys.  I notice it has not (yet) magically
appeared in Linus's public tree like the rss one did: so to be on the
safe side, I'll just repost it now, with your Reported-and-tested-bys,
otherwise unchanged.

Hugh

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Thu, 20 Mar 2014, Sasha Levin wrote:
> On 03/20/2014 09:51 AM, Dave Jones wrote:
> > On Wed, Mar 19, 2014 at 10:00:29PM -0700, Hugh Dickins wrote:
> > 
> >   > > This might be collateral damage from the swapops thing, I guess we
> > won't know until
> >   > > that gets fixed, but I thought I'd mention that we might still have a
> > problem here.
> >   >
> >   > Yes, those Bad rss-counters could well be collateral damage from the
> >   > swapops BUG.  To which I believe I now have the answer: again untested,
> >   > but please give this a try...
> > 
> > This survived an overnight run. No swapops bug, and no bad RSS. Good job:)
> 
> Same here, swapops bug is gone!

That was welcome news, thanks guys.  I notice it has not (yet) magically
appeared in Linus's public tree like the rss one did: so to be on the
safe side, I'll just repost it now, with your Reported-and-tested-bys,
otherwise unchanged.

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Sasha Levin wrote:
> On 03/18/2014 08:38 PM, Hugh Dickins wrote:
> > On Tue, 11 Mar 2014, Dave Jones wrote:
> > > On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
> > >   > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
> > >   > >  >
> > >   > >  > Dave, iirc trinity can write log file pointing which exactly
> > > syscall sequence
> > >   > >  > was passed, right? Share it too please.
> > >   > >
> > >   > > Hm, I may have been mistaken, and the damage was done by a previous
> > > run.
> > >   > > I went from being able to reproduce it almost instantly to now not
> > > being able
> > >   > > to reproduce it at all.  Will keep trying.
> > >   >
> > >   > Sasha already gave a link to the syscalls sequence, so no rush.
> > > 
> > > It'd be nice to get a more concise reproducer, his list had a little of
> > > everything in there.
> > 
> > I've so far failed to find any explanation for your swapops.h BUG;
> > but believe I have identified one cause for "Bad rss-counter"s.
> > 
> > My hunch is that the swapops.h BUG is "nearby", but I just cannot
> > fit it together (the swapops.h BUG comes when rmap cannot find all
> > all the migration entries it inserted earlier: it's a very useful
> > BUG for validating rmap).
> > 
> > Untested patch below: I can't quite say Reported-by, because it may
> > not even be one that you and Sasha have been seeing; but I'm hopeful,
> > remap_file_pages is in the list.
> > 
> > Please give this a try, preferably on 3.14-rc or earlier: I've never
> > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
> > a lot more than most of us); but have seen them on mmotm/next, so
> > some other trigger is coming up there, I'll worry about that once
> > it reaches 3.15-rc.
> 
> The patch fixed the "Bad rss-counter" errors I've been seeing both in
> 3.14-rc7 and -next.

Great, thanks a lot, Sasha.  I was afraid that you'd hit those swapops
BUGs, which seemed perhaps to be paired with these; but glad to hear
a positive.  Let's see how Dave fares.  (I've not forgotten shmem
fallocate, by the way, but those probably aren't as high on my agenda
as you'd like.)

Hugh

Re: bad rss-counter message in 3.14rc5

[Hugh Dickins]

On Tue, 18 Mar 2014, Sasha Levin wrote:
> On 03/18/2014 08:38 PM, Hugh Dickins wrote:
> > On Tue, 11 Mar 2014, Dave Jones wrote:
> > > On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
> > >   > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
> > >   > >  >
> > >   > >  > Dave, iirc trinity can write log file pointing which exactly
> > > syscall sequence
> > >   > >  > was passed, right? Share it too please.
> > >   > >
> > >   > > Hm, I may have been mistaken, and the damage was done by a previous
> > > run.
> > >   > > I went from being able to reproduce it almost instantly to now not
> > > being able
> > >   > > to reproduce it at all.  Will keep trying.
> > >   >
> > >   > Sasha already gave a link to the syscalls sequence, so no rush.
> > > 
> > > It'd be nice to get a more concise reproducer, his list had a little of
> > > everything in there.
> > 
> > I've so far failed to find any explanation for your swapops.h BUG;
> > but believe I have identified one cause for "Bad rss-counter"s.
> > 
> > My hunch is that the swapops.h BUG is "nearby", but I just cannot
> > fit it together (the swapops.h BUG comes when rmap cannot find all
> > all the migration entries it inserted earlier: it's a very useful
> > BUG for validating rmap).
> > 
> > Untested patch below: I can't quite say Reported-by, because it may
> > not even be one that you and Sasha have been seeing; but I'm hopeful,
> > remap_file_pages is in the list.
> > 
> > Please give this a try, preferably on 3.14-rc or earlier: I've never
> > seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
> > a lot more than most of us); but have seen them on mmotm/next, so
> > some other trigger is coming up there, I'll worry about that once
> > it reaches 3.15-rc.
> 
> The patch fixed the "Bad rss-counter" errors I've been seeing both in
> 3.14-rc7 and -next.

Great, thanks a lot, Sasha.  I was afraid that you'd hit those swapops
BUGs, which seemed perhaps to be paired with these; but glad to hear
a positive.  Let's see how Dave fares.  (I've not forgotten shmem
fallocate, by the way, but those probably aren't as high on my agenda
as you'd like.)

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/18/2014 10:12 PM, Hugh Dickins wrote:
> On Tue, 18 Mar 2014, Sasha Levin wrote:
>> On 03/18/2014 08:38 PM, Hugh Dickins wrote:
>>> On Tue, 11 Mar 2014, Dave Jones wrote:
>>>> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
>>>>    > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>>>>    > >  >
>>>>    > >  > Dave, iirc trinity can write log file pointing which exactly
>>>> syscall sequence
>>>>    > >  > was passed, right? Share it too please.
>>>>    > >
>>>>    > > Hm, I may have been mistaken, and the damage was done by a previous
>>>> run.
>>>>    > > I went from being able to reproduce it almost instantly to now not
>>>> being able
>>>>    > > to reproduce it at all.  Will keep trying.
>>>>    >
>>>>    > Sasha already gave a link to the syscalls sequence, so no rush.
>>>>
>>>> It'd be nice to get a more concise reproducer, his list had a little of
>>>> everything in there.
>>>
>>> I've so far failed to find any explanation for your swapops.h BUG;
>>> but believe I have identified one cause for "Bad rss-counter"s.
>>>
>>> My hunch is that the swapops.h BUG is "nearby", but I just cannot
>>> fit it together (the swapops.h BUG comes when rmap cannot find all
>>> all the migration entries it inserted earlier: it's a very useful
>>> BUG for validating rmap).
>>>
>>> Untested patch below: I can't quite say Reported-by, because it may
>>> not even be one that you and Sasha have been seeing; but I'm hopeful,
>>> remap_file_pages is in the list.
>>>
>>> Please give this a try, preferably on 3.14-rc or earlier: I've never
>>> seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
>>> a lot more than most of us); but have seen them on mmotm/next, so
>>> some other trigger is coming up there, I'll worry about that once
>>> it reaches 3.15-rc.
>>
>> The patch fixed the "Bad rss-counter" errors I've been seeing both in
>> 3.14-rc7 and -next.
>
> Great, thanks a lot, Sasha.  I was afraid that you'd hit those swapops
> BUGs, which seemed perhaps to be paired with these; but glad to hear
> a positive.  Let's see how Dave fares.  (I've not forgotten shmem
> fallocate, by the way, but those probably aren't as high on my agenda
> as you'd like.)

I do hit the swapops issue a lot, I didn't think that your patch was
supposed to fix that so I didn't mention it.

Thanks for keeping shmem in mind, I've removed shmem from testing for now
but I agree, it's not one of the more important issues to be taken care of.


Thanks,
Sasha

Re: bad rss-counter message in 3.14rc5

[Sasha Levin]

On 03/18/2014 10:12 PM, Hugh Dickins wrote:
> On Tue, 18 Mar 2014, Sasha Levin wrote:
>> On 03/18/2014 08:38 PM, Hugh Dickins wrote:
>>> On Tue, 11 Mar 2014, Dave Jones wrote:
>>>> On Tue, Mar 11, 2014 at 09:36:03PM +0400, Cyrill Gorcunov wrote:
>>>>    > On Tue, Mar 11, 2014 at 01:10:45PM -0400, Dave Jones wrote:
>>>>    > >  >
>>>>    > >  > Dave, iirc trinity can write log file pointing which exactly
>>>> syscall sequence
>>>>    > >  > was passed, right? Share it too please.
>>>>    > >
>>>>    > > Hm, I may have been mistaken, and the damage was done by a previous
>>>> run.
>>>>    > > I went from being able to reproduce it almost instantly to now not
>>>> being able
>>>>    > > to reproduce it at all.  Will keep trying.
>>>>    >
>>>>    > Sasha already gave a link to the syscalls sequence, so no rush.
>>>>
>>>> It'd be nice to get a more concise reproducer, his list had a little of
>>>> everything in there.
>>>
>>> I've so far failed to find any explanation for your swapops.h BUG;
>>> but believe I have identified one cause for "Bad rss-counter"s.
>>>
>>> My hunch is that the swapops.h BUG is "nearby", but I just cannot
>>> fit it together (the swapops.h BUG comes when rmap cannot find all
>>> all the migration entries it inserted earlier: it's a very useful
>>> BUG for validating rmap).
>>>
>>> Untested patch below: I can't quite say Reported-by, because it may
>>> not even be one that you and Sasha have been seeing; but I'm hopeful,
>>> remap_file_pages is in the list.
>>>
>>> Please give this a try, preferably on 3.14-rc or earlier: I've never
>>> seen "Bad rss-counter"s there myself (trinity uses remap_file_pages
>>> a lot more than most of us); but have seen them on mmotm/next, so
>>> some other trigger is coming up there, I'll worry about that once
>>> it reaches 3.15-rc.
>>
>> The patch fixed the "Bad rss-counter" errors I've been seeing both in
>> 3.14-rc7 and -next.
>
> Great, thanks a lot, Sasha.  I was afraid that you'd hit those swapops
> BUGs, which seemed perhaps to be paired with these; but glad to hear
> a positive.  Let's see how Dave fares.  (I've not forgotten shmem
> fallocate, by the way, but those probably aren't as high on my agenda
> as you'd like.)

I do hit the swapops issue a lot, I didn't think that your patch was
supposed to fix that so I didn't mention it.

Thanks for keeping shmem in mind, I've removed shmem from testing for now
but I agree, it's not one of the more important issues to be taken care of.


Thanks,
Sasha

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 18, 2014 at 05:38:38PM -0700, Hugh Dickins wrote:
> 
> (Cyrill, entirely unrelated, but in preparing this patch I noticed
> your soft_dirty work in install_file_pte(): which looked good at
> first, until I realized that it's propagating the soft_dirty of a
> pte it's about to zap completely, to the unrelated entry it's about
> to insert in its place.  Which seems very odd to me.)
> 

Thanks a lot Hugh for pointing! I'll revisit all file-softdirty cases.
(btw, I've grabbed Dave's config to run trinity and somehow help in
 testing and attempt to figure out what causes it but didn't yet
 find hardware node to run, hopefully i'll get a spare machine for
 testing in a couple of days).

Re: bad rss-counter message in 3.14rc5

[Cyrill Gorcunov]

On Tue, Mar 18, 2014 at 05:38:38PM -0700, Hugh Dickins wrote:
> 
> (Cyrill, entirely unrelated, but in preparing this patch I noticed
> your soft_dirty work in install_file_pte(): which looked good at
> first, until I realized that it's propagating the soft_dirty of a
> pte it's about to zap completely, to the unrelated entry it's about
> to insert in its place.  Which seems very odd to me.)
> 

Thanks a lot Hugh for pointing! I'll revisit all file-softdirty cases.
(btw, I've grabbed Dave's config to run trinity and somehow help in
 testing and attempt to figure out what causes it but didn't yet
 find hardware node to run, hopefully i'll get a spare machine for
 testing in a couple of days).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>