Re: [PATCH v3] tools/xenconsoled: Increase file descriptor limit

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 19/02/15 16:30, Ian Jackson wrote:
> Wei Liu writes ("Re: [PATCH v3] tools/xenconsoled: Increase file descriptor limit"):
>> On Tue, Feb 17, 2015 at 05:55:52PM +0000, Andrew Cooper wrote:
>>> XenServer's VM density testing uncovered a regression when moving from
>>> sysvinit to systemd where the file descriptor limit dropped from 4096 to
>>> 1024. (XenServer had previously inserted a ulimit statement into its
>>> initscripts.)
> I think that putting something like this in the initscripts is a good
> idea.
>
> I don't like the idea of this kind of resource limit mangling (quite
> vigorous, too!) being buried in the C code.
>
>>> One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the
>>> lost ulimit, but that is only a stopgap solution.
> Why is this only a stopgap solution ?

It is yet another place with an arbitrary limit, which is one more
moving part to go wrong.

>
>>> As Xenconsoled genuinely needs a large number of file descriptors
>>> if a large number of domains are running, and is well behaved with
>>> its descriptors, attempt to up the limit to the system maximum.
> Perhaps you mean it should be "unlimited" but that doesn't seem right
> either.

Where do I perhaps mean "unlimited"?

Attempting to set NOFILE with RLIM_INFINITY is an unconditional failure,
and there is unfortunately no RLIM_SYSTEM_MAX.

> The value should be some multiple of the maximum number of
> domains.

The maximum number of domains (limited by the domid abi) is 32751

However, a brief grep through the code shows that the fd situation is
far worse than I first thought.  There are 4 FDs held open per
domain[1], and some overhead for hypervisor logging, gntdev and privcmd.

That would put the number of fds needed at 131004 + overhead (20
perhaps?).  This is substantially larger than the default on Linux, but
within the default system max of 2^20.

~Andrew

[1] along with a todo note suggesting that opening evtchn for each
domain is inefficient.