I'd like to see enable / disable cookie work from atomic context

I'd like to see enable / disable cookie work from atomic context

[Milosz Tanski]

David,

I'd like to continue our conversation from IRC about making enable /
disable cookie from from atomic context. Maybe this diagram and the
description will detail it better.

remote1         MDS         thread 1            thread2     thread N
                            file_open(READ)
                            enable_cache()
file_open(WR)
                revoke_read()
                            spinlock()
                            lose_cache_cap()
                            spinunlock()
                                                start_read()
                            disable_cookie()
                            wait_on_ops()                   start_read()

The issue here is what happens in my scenario in thread2 where is
manages to schedule a possibly stale read in thread 2 is worrying.

What's worse is that disabling of the cookie waits on operations count
to drop to 0. But it doesn't seam to me like anything is stopping more
operations from getting queued.

Ideally what I'd like to see is an ability to enqueue a disable
barrier in the operation queue. This way we avoid both scenarios
(thread2, other threads) in the diagram above.  Hopefully that can be
done in atomic context and the rest of the disable work can be
deferred till later.

-- 
Milosz Tanski
CTO
10 East 53rd Street, 37th floor
New York, NY 10022

p: 646-253-9055
e: milosz@adfin.com