Re: Race between SIGIO and epoll from SMP host

From: Anton Ivanov <anton.ivanov@kot-begemot.co.uk>
To: linux-um@lists.infradead.org
Subject: Re: Race between SIGIO and epoll from SMP host
Date: Wed, 21 Apr 2021 15:21:21 +0100	[thread overview]
Message-ID: <4992839c-447f-f28e-fa76-12263f7b0494@kot-begemot.co.uk> (raw)
In-Reply-To: <CABqSeAQ-UdKoBGvWC-n=020opHDkSKV42c98cb=KvgxHn7gCfw@mail.gmail.com>

On 21/04/2021 14:35, YiFei Zhu wrote:
> On Wed, Apr 21, 2021 at 7:32 AM Anton Ivanov
> <anton.ivanov@kot-begemot.co.uk> wrote:
>>> Considering that this is a race on the host, what would be the best
>>> way to fix this?
>>
>> Interesting one. I need to think.
>>
>> One option would be to wait for epoll events with a timeout which is larger than zero - f.e. HZ.
> 
> I was about to say I could reproduce it even with a timeout of 1ms,
> then I realized that code I pasted above already used 1ms timeout.
> Assertion failures using 1ms timeout seems much rarer than 0 timeout
> however.
> 
> For reference my CONFIG_HZ on the host is 1000. I also use
> CONFIG_NO_HZ_IDLE if that's relevant (I'm not too familiar with how
> the kernel ticking works).

Reading the code around the kernel the race always applies. What changes is the likelihood.

The usual pattern of doing things is:

1) wake up something - means differ depending on the actual place where this is. This may or may not end up on a different core.

2) send SIGIOs.

The only place which does things the other way around is actually ttys. I had a look there in a couple of places and there it first sends SIGIO, then wakes anyone pending for notifications.

That is why you are seeing it on ttys and that is why we are likely to see this more often on ttys compared to other IO. It will happen on other IO as well as there is no guarantee that the notification handling will be performed on the same core as UML.

As far as using time-out on the poll, there is a number of fds which have lots of spurious SIGIOs and it will end up waiting on them. The worst offender is random, but it is not alone on this one.

This will not be easy - we use epoll with a EPOLLET pattern so if we have missed an event after a SIGIO we will not see it until there is a rerun.

I need to think on this one.

A.

 >
 >
> 
>> If we have received a SIGIO there is an epoll event on the way. The fact that it is not in the queue right now means that we are due to process it shortly.
>>
>> A.
> 
> YiFei Zhu
> 
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
> 

-- 
Anton R. Ivanov
https://www.kot-begemot.co.uk/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um