Re: [PATCH v2 0/2] Refactor snapshot vs nocow writers locking

[Nikolay Borisov <nborisov@suse.com>]

On 30.07.19 г. 14:03 ч., Valentin Schneider wrote:
> On 29/07/2019 17:32, Valentin Schneider wrote:
>> On 29/07/2019 16:33, Catalin Marinas wrote:
> [...]
>>> I'd say that's one of the pitfalls of PlusCal. The above is executed
>>> atomically, so you'd have the lock_state read and updated in the same
>>> action. Looking at the C patches, there is an
>>> atomic_read(&lock->readers) followed by a
>>> percpu_counter_inc(&lock->writers). Between these two, you can have
>>> "readers" becoming non-zero via a different CPU.
>>>
>>> My suggestion would be to use procedures with labels to express the
>>> non-atomicity of such sequences.
>>>
>>
> 
> FYI, with a very simple and stupid modification of the spec:
> 
> ----->8-----
> macro ReadUnlock()
> {
>     reader_count := reader_count - 1;
>     \* Condition variable signal is "implicit" here
> }
> 
> macro WriteUnlock()
> {
>     writer_count := writer_count - 1;
>     \* Ditto on the cond var
> }
> 
> procedure ReadLock()
> {
> add:
>     reader_count := reader_count + 1;
> lock:
>     await writer_count = 0;
>     return;
> }
> 
> procedure WriteLock()
> {
> add:
>     writer_count := writer_count + 1;
> lock:
>     await reader_count = 0;
>     return;
> };
> -----8<-----
> 
> it's quite easy to trigger the case Paul pointed out in [1]:

Yes, however, there was a bug in the original posting, in that
btrfs_drw_try_write_lock should have called btrfs_drw_write_unlock
instead of btrfs_drw_read_unlock if it sees that readers incremented
while it has already incremented its percpu counter.

Additionally the implementation doesn't await with the respective
variable incremented. Is there a way to express something along the
lines of :


> procedure WriteLock()
> {
> add:
>     writer_count := writer_count + 1;
> lock:
>     await reader_count = 0;

If we are about to wait then also decrement writer_count?  I guess the
correct way to specify it would be:

procedure WriteLock()
 {

     writer_count := writer_count + 1;
     await reader_count = 0;
     return;
 };


Because the implementation (by using barriers and percpu counters
ensures all of this happens as one atomic step?) E.g. before going to
sleep we decrement the write unlock.

>     return;
> };

> 
> ----->8-----
> Error: Deadlock reached.
> Error: The behavior up to this point is:
> State 1: <Initial predicate>
> /\ stack = (<<reader, 1>> :> <<>> @@ <<writer, 1>> :> <<>>)
> /\ pc = (<<reader, 1>> :> "loop" @@ <<writer, 1>> :> "loop_")
> /\ writer_count = 0
> /\ reader_count = 0
> /\ lock_state = "idle"
> 
> State 2: <loop_ line 159, col 16 to line 164, col 72 of module specs>
> /\ stack = ( <<reader, 1>> :> <<>> @@
>   <<writer, 1>> :> <<[pc |-> "write_cs", procedure |-> "WriteLock"]>> )
> /\ pc = (<<reader, 1>> :> "loop" @@ <<writer, 1>> :> "add")
> /\ writer_count = 0
> /\ reader_count = 0
> /\ lock_state = "idle"
> 
> State 3: <add line 146, col 14 to line 149, col 63 of module specs>
> /\ stack = ( <<reader, 1>> :> <<>> @@
>   <<writer, 1>> :> <<[pc |-> "write_cs", procedure |-> "WriteLock"]>> )
> /\ pc = (<<reader, 1>> :> "loop" @@ <<writer, 1>> :> "lock")
> /\ writer_count = 1
> /\ reader_count = 0
> /\ lock_state = "idle"
> 
> State 4: <loop line 179, col 15 to line 184, col 71 of module specs>
> /\ stack = ( <<reader, 1>> :> <<[pc |-> "read_cs", procedure |-> "ReadLock"]>> @@
>   <<writer, 1>> :> <<[pc |-> "write_cs", procedure |-> "WriteLock"]>> )
> /\ pc = (<<reader, 1>> :> "add_" @@ <<writer, 1>> :> "lock")
> /\ writer_count = 1
> /\ reader_count = 0
> /\ lock_state = "idle"
> 
> State 5: <add_ line 133, col 15 to line 136, col 64 of module specs>
> /\ stack = ( <<reader, 1>> :> <<[pc |-> "read_cs", procedure |-> "ReadLock"]>> @@
>   <<writer, 1>> :> <<[pc |-> "write_cs", procedure |-> "WriteLock"]>> )
> /\ pc = (<<reader, 1>> :> "lock_" @@ <<writer, 1>> :> "lock")
> /\ writer_count = 1
> /\ reader_count = 1
> /\ lock_state = "idle"
> -----8<-----
> 
> Which I think is pretty cool considering the effort that was required
> (read: not much).
> 
> [1]: https://lore.kernel.org/lkml/20190607105251.GB28207@linux.ibm.com/
>