Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[PGNet Dev]

I've upgraded a Xen server's

	xen-4.9.0_08-517.2.x86_64
	xen-libs-4.9.0_08-517.2.x86_64

kernel from 4.12x to 4.13x

	uname -rm
		4.13.0-rc2-2.gb545b87-default x86_64

After upgrading, I see in my Dom0-attached serial console, a steady stream of,

	...
	(XEN) [2017-07-27 18:00:25] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2017-07-27 18:00:58] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2017-07-27 18:01:31] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2017-07-27 18:02:04] d1v0 Over-allocation for domain 1: 524545 > 524544
	(XEN) [2017-07-27 18:02:36] d1v0 Over-allocation for domain 1: 524545 > 524544
	...

every 32-33 seconds.

I've seen these b4, they went away afaict after mv to 4.12 kernel, and now have reappeared.

I didn't know 'back then' what use, or harm, these were -- and now am not clear if there's a new/returned problem, or just resurfaced log noise.

#irc, I was told

	Q: Any insight on what these are telling me?
	A: dom1 can't count, and is trying to grab more memory from Xen than it is permitted to have

This seemed possibly relevant https://patchwork.kernel.org/patch/9506769/

	"...
	I should say that this is all a gross hack, and is in desperate need of a proper API to make rings entirely outside of the gfn space, but this hack should work for now.
	..."

I rec'd a 'simpler' suggestion, to set mem != max for dom0,

-	dom0_mem=4G,max:4G
+	dom0_mem=4016M,max:4096M

No difference.

Still getting the errors.

@ irc, suggested

	sounds like there is a new regression in the ballooning driver
	probably best email xen-devel@ and see if anyone has any ideas

Is this a real/returned problem -- or just noise?



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[Juergen Gross]

On 27/07/17 20:20, PGNet Dev wrote:
> I've upgraded a Xen server's
> 
> 	xen-4.9.0_08-517.2.x86_64
> 	xen-libs-4.9.0_08-517.2.x86_64
> 
> kernel from 4.12x to 4.13x
> 
> 	uname -rm
> 		4.13.0-rc2-2.gb545b87-default x86_64
> 
> After upgrading, I see in my Dom0-attached serial console, a steady stream of,
> 
> 	...
> 	(XEN) [2017-07-27 18:00:25] d1v0 Over-allocation for domain 1: 524545 > 524544
> 	(XEN) [2017-07-27 18:00:58] d1v0 Over-allocation for domain 1: 524545 > 524544
> 	(XEN) [2017-07-27 18:01:31] d1v0 Over-allocation for domain 1: 524545 > 524544
> 	(XEN) [2017-07-27 18:02:04] d1v0 Over-allocation for domain 1: 524545 > 524544
> 	(XEN) [2017-07-27 18:02:36] d1v0 Over-allocation for domain 1: 524545 > 524544
> 	...
> 
> every 32-33 seconds.

Can you please post the domain's config file used to create the domain
and the kernel config?


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[PGNet Dev]

On 7/27/17 11:23 PM, Juergen Gross wrote:
> Can you please post the domain's config file used to create the domain
> and the kernel config?

Sure.

   https://pastebin.com/M6cr2pX7


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[PGNet Dev]

On 7/28/17 9:02 AM, PGNet Dev wrote:
> On 7/27/17 11:23 PM, Juergen Gross wrote:
>> Can you please post the domain's config file used to create the domain
>> and the kernel config?
> 
> Sure.
> 
>    https://pastebin.com/M6cr2pX7
> 

Any add'l info needed?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[Juergen Gross]

On 01/08/17 16:28, PGNet Dev wrote:
> On 7/28/17 9:02 AM, PGNet Dev wrote:
>> On 7/27/17 11:23 PM, Juergen Gross wrote:
>>> Can you please post the domain's config file used to create the domain
>>> and the kernel config?
>>
>> Sure.
>>
>>    https://pastebin.com/M6cr2pX7
>>
> 
> Any add'l info needed?

No, I don't think so.

IMO the problem is related to the fact that the balloon driver tries to
use then kernel's view of how much memory it is owning and setting this
number in relation to Xen's view how much memory it should try to have.

Maybe before adding memory from Xen the kernel should ask the hypervisor
how much memory it has already from Xen's point of view and how much it
is allowed to have. This will avoid the messages you have seen as long
as there are no interfering actions from Xen (e.g. lowering the maximum
reservation) while the kernel is trying to balloon up.


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[Boris Ostrovsky]

On 08/02/2017 08:01 AM, Juergen Gross wrote:
> On 01/08/17 16:28, PGNet Dev wrote:
>> On 7/28/17 9:02 AM, PGNet Dev wrote:
>>> On 7/27/17 11:23 PM, Juergen Gross wrote:
>>>> Can you please post the domain's config file used to create the domain
>>>> and the kernel config?
>>> Sure.
>>>
>>>    https://pastebin.com/M6cr2pX7
>>>
>> Any add'l info needed?
> No, I don't think so.
>
> IMO the problem is related to the fact that the balloon driver tries to
> use then kernel's view of how much memory it is owning and setting this
> number in relation to Xen's view how much memory it should try to have.
>
> Maybe before adding memory from Xen the kernel should ask the hypervisor
> how much memory it has already from Xen's point of view and how much it
> is allowed to have. This will avoid the messages you have seen as long
> as there are no interfering actions from Xen (e.g. lowering the maximum
> reservation) while the kernel is trying to balloon up.

Could this be caused by your recent ballooning patch
(96edd61dcf44362d3ef0bed1a5361e0ac7886a63)?

-boris

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Xen 4.9 + kernel 4.13rc2 -- ballooning regression? reappearance of "Over-allocation for domain 1" errors

[Juergen Gross]

On 03/08/17 00:43, Boris Ostrovsky wrote:
> On 08/02/2017 08:01 AM, Juergen Gross wrote:
>> On 01/08/17 16:28, PGNet Dev wrote:
>>> On 7/28/17 9:02 AM, PGNet Dev wrote:
>>>> On 7/27/17 11:23 PM, Juergen Gross wrote:
>>>>> Can you please post the domain's config file used to create the domain
>>>>> and the kernel config?
>>>> Sure.
>>>>
>>>>    https://pastebin.com/M6cr2pX7
>>>>
>>> Any add'l info needed?
>> No, I don't think so.
>>
>> IMO the problem is related to the fact that the balloon driver tries to
>> use then kernel's view of how much memory it is owning and setting this
>> number in relation to Xen's view how much memory it should try to have.
>>
>> Maybe before adding memory from Xen the kernel should ask the hypervisor
>> how much memory it has already from Xen's point of view and how much it
>> is allowed to have. This will avoid the messages you have seen as long
>> as there are no interfering actions from Xen (e.g. lowering the maximum
>> reservation) while the kernel is trying to balloon up.
> 
> Could this be caused by your recent ballooning patch
> (96edd61dcf44362d3ef0bed1a5361e0ac7886a63)?

Only by trying to reduce the domain's memory via "xl mem-set" before the
balloon driver was initialized and then adding memory after the balloon
driver has been started.

Other than that I have no idea how my patch would result in such a
behavior.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel