* hv_balloon issues??
@ 2021-01-25 20:19 Jon Stanley
2021-01-25 20:51 ` [EXTERNAL] " KY Srinivasan
0 siblings, 1 reply; 4+ messages in thread
From: Jon Stanley @ 2021-01-25 20:19 UTC (permalink / raw)
To: kys, haiyangz, sthemmin, wei.liu, linux-hyperv
I'm working to make a method to install bare-metal machines with
Packer images, and in testing (this isn't going to wind up in
production on Hyper-V) I think I've found an issue in hv_balloon, but
I'm not sure.
Starting from a RHEL 8 live CD, I make a tmpfs filesystem and download
a disk image to it. Despite having plenty of memory to do this (I was
downloading a 5GB image onto a VM with 16GB of RAM), I got paid a
visit by the OOM killer.
If I turn off dynamic memory, then things work as expected. This isn't
100% reproducible, I tried immediately after boot and it worked,
unmounted the tmpfs filesystem and waited for a kernel message that
said the balloon floor was reached and tried again, and BOOM!
The actual process that is filling the filesystem (curl) doesn't get
killed (which makes sense I guess since *it* isn't taking a ton of
memory), and also never completes presumably due to it's I/O becoming
blocked. Does this have to do with a sudden, enormous demand for
memory perhaps that the hypervisor is having difficulty fulfilling?
The host has plenty of memory available (63GB right now)
On another note, is there a way that I'm not seeing to tell the
current status of the balloon driver - i.e. current/max allocations? A
quick look through /proc and /sys wasn't revealing.
Also, sorry to be using a distro kernel instead of upstream.
-Jon
Jan 25 14:58:43 dhcp-132.rmrf.net kernel: hv_balloon: Balloon request
will be partially fulfilled. Balloon floor reached.
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: tuned invoked oom-killer:
gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: CPU: 0 PID: 1165 Comm: tuned
Not tainted 4.18.0-240.10.1.el8_3.x86_64 #1
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Hardware name: Microsoft
Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release
v4.0 11/01/2019
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Call Trace:
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: dump_stack+0x5c/0x80
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: dump_header+0x51/0x308
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: oom_kill_process.cold.28+0xb/0x10
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: out_of_memory+0x1c1/0x4b0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: __alloc_pages_slowpath+0xc24/0xd40
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: __alloc_pages_nodemask+0x245/0x280
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: filemap_fault+0x3b8/0x840
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ? hrtimer_cancel+0x11/0x20
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ? futex_wait+0x19a/0x210
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ? xas_load+0x8/0x80
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ? xas_find+0x173/0x1b0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ? filemap_map_pages+0x1a3/0x380
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ext4_filemap_fault+0x2c/0x40 [ext4]
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: __do_fault+0x38/0xc0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: do_fault+0x191/0x3c0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: __handle_mm_fault+0x3e6/0x7c0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: handle_mm_fault+0xc2/0x1d0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: __do_page_fault+0x21b/0x4d0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: do_page_fault+0x32/0x110
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: ? page_fault+0x8/0x30
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: page_fault+0x1e/0x30
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RIP: 0033:0x7faf2f8c5df2
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Code: Bad RIP value.
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RSP: 002b:00007faf242629a0
EFLAGS: 00010246
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RAX: ffffffffffffff92 RBX:
00007faf24262a40 RCX: 00007faf2f8c5df2
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RDX: 0000000000000000 RSI:
0000000000000189 RDI: 00007faf1c002490
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RBP: 00007faf1c002490 R08:
0000000000000000 R09: 00000000ffffffff
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: R10: 00007faf24262a40 R11:
0000000000000246 R12: 0000000000000000
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: R13: 0000000000000000 R14:
00007faf24262a40 R15: 000000003b9aca00
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Mem-Info:
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: active_anon:18180
inactive_anon:738744 isolated_anon:0
active_file:18
inactive_file:337 isolated_file:32
unevictable:132114 dirty:0
writeback:0 unstable:0
slab_reclaimable:6250
slab_unreclaimable:5966
mapped:1626 shmem:738916
pagetables:1396 bounce:0
free:31759 free_pcp:30 free_cma:0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 active_anon:72720kB
inactive_anon:2954976kB active_file:72kB inactive_file:1348kB
unevictable:528456kB isolated(anon):0kB i>
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 DMA free:15908kB
min:64kB low:80kB high:96kB active_anon:0kB inactive_anon:0kB
active_file:0kB inactive_file:0kB unevictabl>
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: lowmem_reserve[]: 0 3845
15960 15960 15960
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 DMA32 free:64676kB
min:16264kB low:20328kB high:24392kB active_anon:1424kB
inactive_anon:2489752kB active_file:28kB inactiv>
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: lowmem_reserve[]: 0 0 12114
12114 12114
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 Normal free:46452kB
min:51248kB low:64060kB high:76872kB active_anon:71296kB
inactive_anon:465224kB active_file:4kB inactiv>
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: lowmem_reserve[]: 0 0 0 0 0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 DMA: 1*4kB (U) 0*8kB
0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB
(U) 1*2048kB (M) 3*4096kB (M) = >
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 DMA32: 29*4kB (UE)
36*8kB (UE) 33*16kB (UME) 6*32kB (UE) 3*64kB (UME) 1*128kB (U) 3*256kB
(UME) 2*512kB (UM) 2*1024kB (U) 3>
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 Normal: 833*4kB (UME)
712*8kB (UME) 305*16kB (UME) 152*32kB (UME) 52*64kB (E) 28*128kB (UME)
15*256kB (UME) 11*512kB (UME) >
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 hugepages_total=0
hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 hugepages_total=0
hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 871413 total pagecache pages
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 0 pages in swap cache
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Swap cache stats: add 0,
delete 0, find 0/0
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Free swap = 0kB
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Total swap = 0kB
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 4194027 pages RAM
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 0 pages HighMem/MovableOnly
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 91830 pages reserved
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 0 pages hwpoisoned
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ pid ] uid tgid total_vm
rss pgtables_bytes swapents oom_score_adj name
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 762] 0 762 27626
1788 290816 0 0 systemd-journal
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 816] 0 816 25338
353 212992 0 -1000 systemd-udevd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 819] 0 819 15287
152 135168 0 -1000 auditd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 860] 81 860 14087
213 155648 0 -900 dbus-daemon
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 875] 995 875 29968
111 147456 0 0 chronyd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 907] 0 907 48443
510 405504 0 0 sssd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 908] 997 908 404961
1915 331776 0 0 polkitd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 913] 0 913 1085
16 53248 0 0 hypervvssd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 914] 994 914 40028
204 208896 0 0 rngd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 921] 0 921 50484
659 421888 0 0 sssd_be
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 922] 0 922 53956
395 462848 0 0 sssd_nss
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 925] 0 925 74573
5478 466944 0 0 firewalld
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 926] 0 926 24290
252 204800 0 0 systemd-logind
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 940] 0 940 116867
614 389120 0 0 NetworkManager
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 958] 0 958 23072
224 212992 0 -1000 sshd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 968] 0 968 1778
30 61440 0 0 hypervkvpd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 969] 0 969 106589
3721 450560 0 0 tuned
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 972] 0 972 9232
221 106496 0 0 crond
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 973] 0 973 10449
135 114688 0 0 rhsmcertd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1189] 0 1189 56455
509 192512 0 0 rsyslogd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1201] 0 1201 30749
215 266240 0 0 login
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1206] 0 1206 23443
331 225280 0 0 systemd
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1210] 0 1210 37531
648 299008 0 0 (sd-pam)
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1216] 0 1216 6554
154 86016 0 0 bash
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1285] 0 1285 20229
245 196608 0 0 curl
Jan 25 14:59:30 dhcp-132.rmrf.net kernel:
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/firewalld.service,>
Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Out of memory: Killed
process 925 (firewalld) total-vm:298292kB, anon-rss:21912kB,
file-rss:0kB, shmem-rss:0kB, UID:0
Jan 25 14:59:34 dhcp-132.rmrf.net systemd[1]: firewalld.service: Main
process exited, code=killed, status=9/KILL
Jan 25 14:59:47 dhcp-132.rmrf.net systemd[1]: firewalld.service:
Failed with result 'signal'.
^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: [EXTERNAL] hv_balloon issues??
2021-01-25 20:19 hv_balloon issues?? Jon Stanley
@ 2021-01-25 20:51 ` KY Srinivasan
2021-01-25 21:07 ` Jon Stanley
0 siblings, 1 reply; 4+ messages in thread
From: KY Srinivasan @ 2021-01-25 20:51 UTC (permalink / raw)
To: Jon Stanley, Haiyang Zhang, Stephen Hemminger, wei.liu, linux-hyperv
I take it that this is on Windows Server machine. What are the Dynamic memory settings for the VM under question.
K. Y
> -----Original Message-----
> From: Jon Stanley <jonstanley@gmail.com>
> Sent: Monday, January 25, 2021 12:20 PM
> To: KY Srinivasan <kys@microsoft.com>; Haiyang Zhang
> <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; wei.liu@kernel.org; linux-
> hyperv@vger.kernel.org
> Subject: [EXTERNAL] hv_balloon issues??
>
> I'm working to make a method to install bare-metal machines with Packer
> images, and in testing (this isn't going to wind up in production on Hyper-V) I
> think I've found an issue in hv_balloon, but I'm not sure.
>
> Starting from a RHEL 8 live CD, I make a tmpfs filesystem and download a disk
> image to it. Despite having plenty of memory to do this (I was downloading a
> 5GB image onto a VM with 16GB of RAM), I got paid a visit by the OOM killer.
>
> If I turn off dynamic memory, then things work as expected. This isn't 100%
> reproducible, I tried immediately after boot and it worked, unmounted the
> tmpfs filesystem and waited for a kernel message that said the balloon floor
> was reached and tried again, and BOOM!
>
> The actual process that is filling the filesystem (curl) doesn't get killed (which
> makes sense I guess since *it* isn't taking a ton of memory), and also never
> completes presumably due to it's I/O becoming blocked. Does this have to do
> with a sudden, enormous demand for memory perhaps that the hypervisor is
> having difficulty fulfilling?
> The host has plenty of memory available (63GB right now)
>
> On another note, is there a way that I'm not seeing to tell the current status of
> the balloon driver - i.e. current/max allocations? A quick look through /proc
> and /sys wasn't revealing.
>
> Also, sorry to be using a distro kernel instead of upstream.
>
> -Jon
>
> Jan 25 14:58:43 dhcp-132.rmrf.net kernel: hv_balloon: Balloon request will be
> partially fulfilled. Balloon floor reached.
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: tuned invoked oom-killer:
> gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: CPU: 0 PID: 1165 Comm: tuned Not
> tainted 4.18.0-240.10.1.el8_3.x86_64 #1 Jan 25 14:59:30 dhcp-132.rmrf.net
> kernel: Hardware name: Microsoft Corporation Virtual Machine/Virtual
> Machine, BIOS Hyper-V UEFI Release
> v4.0 11/01/2019
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Call Trace:
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: dump_stack+0x5c/0x80 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: dump_header+0x51/0x308 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: oom_kill_process.cold.28+0xb/0x10 Jan
> 25 14:59:30 dhcp-132.rmrf.net kernel: out_of_memory+0x1c1/0x4b0 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: __alloc_pages_slowpath+0xc24/0xd40
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel:
> __alloc_pages_nodemask+0x245/0x280
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: filemap_fault+0x3b8/0x840 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: ? hrtimer_cancel+0x11/0x20 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: ? futex_wait+0x19a/0x210 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: ? xas_load+0x8/0x80 Jan 25 14:59:30
> dhcp-132.rmrf.net kernel: ? xas_find+0x173/0x1b0 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: ? filemap_map_pages+0x1a3/0x380 Jan 25 14:59:30
> dhcp-132.rmrf.net kernel: ext4_filemap_fault+0x2c/0x40 [ext4] Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: __do_fault+0x38/0xc0 Jan 25 14:59:30
> dhcp-132.rmrf.net kernel: do_fault+0x191/0x3c0 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: __handle_mm_fault+0x3e6/0x7c0 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: handle_mm_fault+0xc2/0x1d0 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: __do_page_fault+0x21b/0x4d0 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: do_page_fault+0x32/0x110 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: ? page_fault+0x8/0x30 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: page_fault+0x1e/0x30 Jan 25 14:59:30 dhcp-132.rmrf.net
> kernel: RIP: 0033:0x7faf2f8c5df2 Jan 25 14:59:30 dhcp-132.rmrf.net kernel:
> Code: Bad RIP value.
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RSP: 002b:00007faf242629a0
> EFLAGS: 00010246
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RAX: ffffffffffffff92 RBX:
> 00007faf24262a40 RCX: 00007faf2f8c5df2
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RDX: 0000000000000000 RSI:
> 0000000000000189 RDI: 00007faf1c002490
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: RBP: 00007faf1c002490 R08:
> 0000000000000000 R09: 00000000ffffffff
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: R10: 00007faf24262a40 R11:
> 0000000000000246 R12: 0000000000000000
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: R13: 0000000000000000 R14:
> 00007faf24262a40 R15: 000000003b9aca00
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Mem-Info:
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: active_anon:18180
> inactive_anon:738744 isolated_anon:0
> active_file:18
> inactive_file:337 isolated_file:32
> unevictable:132114 dirty:0
> writeback:0 unstable:0
> slab_reclaimable:6250
> slab_unreclaimable:5966
> mapped:1626 shmem:738916
> pagetables:1396 bounce:0
> free:31759 free_pcp:30 free_cma:0 Jan 25 14:59:30
> dhcp-132.rmrf.net kernel: Node 0 active_anon:72720kB
> inactive_anon:2954976kB active_file:72kB inactive_file:1348kB
> unevictable:528456kB isolated(anon):0kB i> Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: Node 0 DMA free:15908kB min:64kB low:80kB high:96kB
> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
> unevictabl> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: lowmem_reserve[]: 0
> 3845
> 15960 15960 15960
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 DMA32 free:64676kB
> min:16264kB low:20328kB high:24392kB active_anon:1424kB
> inactive_anon:2489752kB active_file:28kB inactiv> Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: lowmem_reserve[]: 0 0 12114
> 12114 12114
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 Normal free:46452kB
> min:51248kB low:64060kB high:76872kB active_anon:71296kB
> inactive_anon:465224kB active_file:4kB inactiv> Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: lowmem_reserve[]: 0 0 0 0 0 Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: Node 0 DMA: 1*4kB (U) 0*8kB 0*16kB 1*32kB (U) 2*64kB
> (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB
> (U) 1*2048kB (M) 3*4096kB (M) = >
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 DMA32: 29*4kB (UE)
> 36*8kB (UE) 33*16kB (UME) 6*32kB (UE) 3*64kB (UME) 1*128kB (U) 3*256kB
> (UME) 2*512kB (UM) 2*1024kB (U) 3>
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Node 0 Normal: 833*4kB (UME)
> 712*8kB (UME) 305*16kB (UME) 152*32kB (UME) 52*64kB (E) 28*128kB
> (UME) 15*256kB (UME) 11*512kB (UME) > Jan 25 14:59:30 dhcp-132.rmrf.net
> kernel: Node 0 hugepages_total=0
> hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: Node 0 hugepages_total=0
> hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: 871413 total pagecache pages Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: 0 pages in swap cache Jan 25 14:59:30
> dhcp-132.rmrf.net kernel: Swap cache stats: add 0, delete 0, find 0/0 Jan 25
> 14:59:30 dhcp-132.rmrf.net kernel: Free swap = 0kB Jan 25 14:59:30 dhcp-
> 132.rmrf.net kernel: Total swap = 0kB Jan 25 14:59:30 dhcp-132.rmrf.net
> kernel: 4194027 pages RAM Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 0 pages
> HighMem/MovableOnly Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 91830
> pages reserved Jan 25 14:59:30 dhcp-132.rmrf.net kernel: 0 pages hwpoisoned
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ pid ] uid tgid total_vm
> rss pgtables_bytes swapents oom_score_adj name
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 762] 0 762 27626
> 1788 290816 0 0 systemd-journal
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 816] 0 816 25338
> 353 212992 0 -1000 systemd-udevd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 819] 0 819 15287
> 152 135168 0 -1000 auditd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 860] 81 860 14087
> 213 155648 0 -900 dbus-daemon
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 875] 995 875 29968
> 111 147456 0 0 chronyd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 907] 0 907 48443
> 510 405504 0 0 sssd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 908] 997 908 404961
> 1915 331776 0 0 polkitd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 913] 0 913 1085
> 16 53248 0 0 hypervvssd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 914] 994 914 40028
> 204 208896 0 0 rngd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 921] 0 921 50484
> 659 421888 0 0 sssd_be
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 922] 0 922 53956
> 395 462848 0 0 sssd_nss
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 925] 0 925 74573
> 5478 466944 0 0 firewalld
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 926] 0 926 24290
> 252 204800 0 0 systemd-logind
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 940] 0 940 116867
> 614 389120 0 0 NetworkManager
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 958] 0 958 23072
> 224 212992 0 -1000 sshd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 968] 0 968 1778
> 30 61440 0 0 hypervkvpd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 969] 0 969 106589
> 3721 450560 0 0 tuned
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 972] 0 972 9232
> 221 106496 0 0 crond
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 973] 0 973 10449
> 135 114688 0 0 rhsmcertd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1189] 0 1189 56455
> 509 192512 0 0 rsyslogd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1201] 0 1201 30749
> 215 266240 0 0 login
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1206] 0 1206 23443
> 331 225280 0 0 systemd
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1210] 0 1210 37531
> 648 299008 0 0 (sd-pam)
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1216] 0 1216 6554
> 154 86016 0 0 bash
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: [ 1285] 0 1285 20229
> 245 196608 0 0 curl
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel:
> oom-
> kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=
> 0,global_oom,task_memcg=/system.slice/firewalld.service,>
> Jan 25 14:59:30 dhcp-132.rmrf.net kernel: Out of memory: Killed process 925
> (firewalld) total-vm:298292kB, anon-rss:21912kB, file-rss:0kB, shmem-rss:0kB,
> UID:0 Jan 25 14:59:34 dhcp-132.rmrf.net systemd[1]: firewalld.service: Main
> process exited, code=killed, status=9/KILL Jan 25 14:59:47 dhcp-132.rmrf.net
> systemd[1]: firewalld.service:
> Failed with result 'signal'.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [EXTERNAL] hv_balloon issues??
2021-01-25 20:51 ` [EXTERNAL] " KY Srinivasan
@ 2021-01-25 21:07 ` Jon Stanley
2021-01-25 21:45 ` KY Srinivasan
0 siblings, 1 reply; 4+ messages in thread
From: Jon Stanley @ 2021-01-25 21:07 UTC (permalink / raw)
To: KY Srinivasan; +Cc: Haiyang Zhang, Stephen Hemminger, wei.liu, linux-hyperv
On Mon, Jan 25, 2021 at 3:51 PM KY Srinivasan <kys@microsoft.com> wrote:
>
> I take it that this is on Windows Server machine. What are the Dynamic memory settings for the VM under question.
No, sorry for leaving that out. This is on Windows 10 Pro for
Workstations, on a Xeon W-2155 machine with 192GB RAM. Apologies but
I'm more of a Linux guy than Windows :). If I've gotten the wrong
thing let me know,
PS C:\Users\jonst> Get-VM -VMName build-test2 |Format-List *
<snip>
CPUUsage : 0
MemoryAssigned : 6036652032
MemoryDemand : 5130682368
MemoryStatus : OK
NumaAligned : False
<snip>
DynamicMemoryEnabled : True
MemoryMaximum : 1099511627776
MemoryMinimum : 536870912
^ permalink raw reply [flat|nested] 4+ messages in thread
* RE: [EXTERNAL] hv_balloon issues??
2021-01-25 21:07 ` Jon Stanley
@ 2021-01-25 21:45 ` KY Srinivasan
0 siblings, 0 replies; 4+ messages in thread
From: KY Srinivasan @ 2021-01-25 21:45 UTC (permalink / raw)
To: Jon Stanley; +Cc: Haiyang Zhang, Stephen Hemminger, wei.liu, linux-hyperv
The DM feature provides a feedback loop for managing the guest memory dynamically between the min memory and the max memory. If the memory pressure is low, the host will drain the memory out of the guest - the floor on what will be left in the guest is the min memory. It is likely that the host was actively reducing the memory assigned to the guest and there was sudden burst of pressure. Perhaps increasing the min memory would help you case.
K. Y
> -----Original Message-----
> From: Jon Stanley <jonstanley@gmail.com>
> Sent: Monday, January 25, 2021 1:07 PM
> To: KY Srinivasan <kys@microsoft.com>
> Cc: Haiyang Zhang <haiyangz@microsoft.com>; Stephen Hemminger
> <sthemmin@microsoft.com>; wei.liu@kernel.org; linux-
> hyperv@vger.kernel.org
> Subject: Re: [EXTERNAL] hv_balloon issues??
>
> On Mon, Jan 25, 2021 at 3:51 PM KY Srinivasan <kys@microsoft.com> wrote:
> >
> > I take it that this is on Windows Server machine. What are the Dynamic
> memory settings for the VM under question.
>
> No, sorry for leaving that out. This is on Windows 10 Pro for Workstations, on a
> Xeon W-2155 machine with 192GB RAM. Apologies but I'm more of a Linux guy
> than Windows :). If I've gotten the wrong thing let me know,
>
> PS C:\Users\jonst> Get-VM -VMName build-test2 |Format-List * <snip>
> CPUUsage : 0
> MemoryAssigned : 6036652032
> MemoryDemand : 5130682368
> MemoryStatus : OK
> NumaAligned : False
> <snip>
> DynamicMemoryEnabled : True
> MemoryMaximum : 1099511627776
> MemoryMinimum : 536870912
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-01-25 21:47 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-25 20:19 hv_balloon issues?? Jon Stanley
2021-01-25 20:51 ` [EXTERNAL] " KY Srinivasan
2021-01-25 21:07 ` Jon Stanley
2021-01-25 21:45 ` KY Srinivasan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.