Re: [PATCH v3 4/5] evtchn: convert domain event lock to an r/w one

[Julien Grall <julien@xen.org>]

Hi Jan,

On 14/12/2020 09:40, Jan Beulich wrote:
> On 11.12.2020 11:57, Julien Grall wrote:
>> On 11/12/2020 10:32, Jan Beulich wrote:
>>> On 09.12.2020 12:54, Julien Grall wrote:
>>>> On 23/11/2020 13:29, Jan Beulich wrote:
>>>>> @@ -620,7 +620,7 @@ int evtchn_close(struct domain *d1, int
>>>>>         long           rc = 0;
>>>>>     
>>>>>      again:
>>>>> -    spin_lock(&d1->event_lock);
>>>>> +    write_lock(&d1->event_lock);
>>>>>     
>>>>>         if ( !port_is_valid(d1, port1) )
>>>>>         {
>>>>> @@ -690,13 +690,11 @@ int evtchn_close(struct domain *d1, int
>>>>>                     BUG();
>>>>>     
>>>>>                 if ( d1 < d2 )
>>>>> -            {
>>>>> -                spin_lock(&d2->event_lock);
>>>>> -            }
>>>>> +                read_lock(&d2->event_lock);
>>>>
>>>> This change made me realized that I don't quite understand how the
>>>> rwlock is meant to work for event_lock. I was actually expecting this to
>>>> be a write_lock() given there are state changed in the d2 events.
>>>
>>> Well, the protection needs to be against racing changes, i.e.
>>> parallel invocations of this same function, or evtchn_close().
>>> It is debatable whether evtchn_status() and
>>> domain_dump_evtchn_info() would better also be locked out
>>> (other read_lock() uses aren't applicable to interdomain
>>> channels).
>>>
>>>> Could you outline how a developper can find out whether he/she should
>>>> use read_lock or write_lock?
>>>
>>> I could try to, but it would again be a port type dependent
>>> model, just like for the per-channel locks.
>>
>> It is quite important to have clear locking strategy (in particular
>> rwlock) so we can make correct decision when to use read_lock or write_lock.
>>
>>> So I'd like it to
>>> be clarified first whether you aren't instead indirectly
>>> asking for these to become write_lock()
>>
>> Well, I don't understand why this is a read_lock() (even with your
>> previous explanation). I am not suggesting to switch to a write_lock(),
>> but instead asking for the reasoning behind the decision.
> 
> So if what I've said in my previous reply isn't enough (including the
> argument towards using two write_lock() here), I'm struggling to
> figure what else to say. The primary goal is to exclude changes to
> the same ports. For this it is sufficient to hold just one of the two
> locks in writer mode, as the other (racing) one will acquire that
> same lock for at least reading. The question whether both need to use
> writer mode can only be decided when looking at the sites acquiring
> just one of the locks in reader mode (hence the reference to
> evtchn_status() and domain_dump_evtchn_info()) - if races with them
> are deemed to be a problem, switching to both-writers will be needed.

I had another look at the code based on your explanation. I don't think 
it is fine to allow evtchn_status() to be concurrently called with 
evtchn_close().

evtchn_close() contains the following code:

   chn2->state = ECS_UNBOUND;
   chn2->u.unbound.remote_domid = d1->domain_id;

Where chn2 is a event channel of the remote domain (d2). Your patch will 
only held the read lock for d2.

However evtchn_status() expects the event channel state to not change 
behind its back. This assumption doesn't hold for d2, and you could 
possibly end up to see the new value of chn2->state after the new 
chn2->u.unbound.remote_domid.

Thanksfully, it doesn't look like chn2->u.interdomain.remote_domain 
would be overwritten. Otherwise, this would be a straight dereference of 
an invalid pointer.

So I think, we need to held the write event lock for both domain.

Cheers,

-- 
Julien Grall