On Wed, Jan 16, 2019 at 03:34:55PM +1100, Dave Chinner wrote:
> On Tue, Jan 15, 2019 at 09:23:12PM -0500, Jerome Glisse wrote:
> > On Tue, Jan 15, 2019 at 06:01:09PM -0800, Dan Williams wrote:
> > > On Tue, Jan 15, 2019 at 5:56 PM Jerome Glisse <jglisse@redhat.com> wrote:
> > > > On Tue, Jan 15, 2019 at 04:44:41PM -0800, John Hubbard wrote:
> > > [..]
> > > > To make it clear.
> > > >
> > > > Lock code:
> > > >     GUP()
> > > >         ...
> > > >         lock_page(page);
> > > >         if (PageWriteback(page)) {
> > > >             unlock_page(page);
> > > >             wait_stable_page(page);
> > > >             goto retry;
> > > >         }
> > > >         atomic_add(page->refcount, PAGE_PIN_BIAS);
> > > >         unlock_page(page);
> > > >
> > > >     test_set_page_writeback()
> > > >         bool pinned = false;
> > > >         ...
> > > >         pinned = page_is_pin(page); // could be after TestSetPageWriteback
> > > >         TestSetPageWriteback(page);
> > > >         ...
> > > >         return pinned;
> > > >
> > > > Memory barrier:
> > > >     GUP()
> > > >         ...
> > > >         atomic_add(page->refcount, PAGE_PIN_BIAS);
> > > >         smp_mb();
> > > >         if (PageWriteback(page)) {
> > > >             atomic_add(page->refcount, -PAGE_PIN_BIAS);
> > > >             wait_stable_page(page);
> > > >             goto retry;
> > > >         }
> > > >
> > > >     test_set_page_writeback()
> > > >         bool pinned = false;
> > > >         ...
> > > >         TestSetPageWriteback(page);
> > > >         smp_wmb();
> > > >         pinned = page_is_pin(page);
> > > >         ...
> > > >         return pinned;
> > > >
> > > >
> > > > One is not more complex than the other. One can contend, the other
> > > > will _never_ contend.
> > > 
> > > The complexity is in the validation of lockless algorithms. It's
> > > easier to reason about locks than barriers for the long term
> > > maintainability of this code. I'm with Jan and John on wanting to
> > > explore lock_page() before a barrier-based scheme.
> > 
> > How is the above hard to validate ?
> 
> Well, if you think it's so easy, then please write the test cases so
> we can add them to fstests and make sure that we don't break it in
> future.
> 
> If you can't write filesystem test cases that exercise these race
> conditions reliably, then the answer to your question is "it is
> extremely hard to validate" and the correct thing to do is to start
> with the simple lock_page() based algorithm.
> 
> Premature optimisation in code this complex is something we really,
> really need to avoid.

Litmus test shows that this never happens, i am attaching 2 litmus
test one with barrier and one without. Without barrier we can see
the double negative !PageWriteback in GUP and !page_pinned() in
test_set_page_writeback() (0:EAX = 0; 1:EAX = 0; below)


    ~/local/bin/litmus7 -r 100 gup.litmus

    ...

    Histogram (3 states)
    2     *>0:EAX=0; 1:EAX=0; x=1; y=1;
    4999999:>0:EAX=1; 1:EAX=0; x=1; y=1;
    4999999:>0:EAX=0; 1:EAX=1; x=1; y=1;
    Ok

    Witnesses
    Positive: 2, Negative: 9999998
    Condition exists (0:EAX=0 /\ 1:EAX=0) is validated
    Hash=2d53e83cd627ba17ab11c875525e078b
    Observation SB Sometimes 2 9999998
    Time SB 3.24



With the barrier this never happens:
    ~/local/bin/litmus7 -r 10000 gup-mb.litmus

    ...

    Histogram (3 states)
    499579828:>0:EAX=1; 1:EAX=0; x=1; y=1;
    499540152:>0:EAX=0; 1:EAX=1; x=1; y=1;
    880020:>0:EAX=1; 1:EAX=1; x=1; y=1;
    No

    Witnesses
    Positive: 0, Negative: 1000000000
    Condition exists (0:EAX=0 /\ 1:EAX=0) is NOT validated
    Hash=0dd48258687c8f737921f907c093c316
    Observation SB Never 0 1000000000


I do not know any better test than litmus for this kind of thing.

Cheers,
Jérôme