kmap_local semantics

kmap_local semantics

[Christoph Hellwig]

So with the new kmap_local interface is it possible / advisable to
use local kmaps over code that might schedule(), e.g. to wait for I/O?

Re: kmap_local semantics

[Ira Weiny]

On Fri, Mar 12, 2021 at 07:54:13AM +0100, Christoph Hellwig wrote:
> So with the new kmap_local interface is it possible / advisable to
> use local kmaps over code that might schedule(), e.g. to wait for I/O?

It is possible yes.  "Advisable" I think so.  Thomas knows better than I what
performance would be.

FWIW I have been working on converting kmaps to kmap_local.  Most of the
instances don't schedule AFAICT.

What I really don't want to see is any kmap'ings handed to another thread.  I
am working hard to eliminate the use of kmap for that use.  Is that going to be
a problem?

Ira

Re: kmap_local semantics

[Thomas Gleixner]

On Fri, Mar 12 2021 at 07:36, Ira Weiny wrote:
> On Fri, Mar 12, 2021 at 07:54:13AM +0100, Christoph Hellwig wrote:
>> So with the new kmap_local interface is it possible / advisable to
>> use local kmaps over code that might schedule(), e.g. to wait for I/O?
>
> It is possible yes.  "Advisable" I think so.  Thomas knows better than I what
> performance would be.

The performance impact is only relevant for 32bit machines with HIGHMEM
enabled. On 64bit kmap_local is a NOOP except when you enable
CONFIG_DEBUG_KMAP_LOCAL there which then actually does the mapping (on
x86 only for now) with guard maps between and that will do the 'drop
map' on schedule out and restore on schedule in dance. But for regular
64bit there is absolutely _NO_ overhead.

> FWIW I have been working on converting kmaps to kmap_local.  Most of the
> instances don't schedule AFAICT.

The whole point of kmap_local is to be preemptible and does not have the
nasty side effects of kmap_atomic, except for the 1 page per map and map
nesting ordering requirements.

> What I really don't want to see is any kmap'ings handed to another thread.  I
> am working hard to eliminate the use of kmap for that use.  Is that going to be
> a problem?

kmap != kmap_local

kmap_local mappings are thread local and can't be handed anywhere. It's
documented.

Thanks,

        tglx

Re: kmap_local semantics

[Ira Weiny]

On Wed, Mar 17, 2021 at 02:59:01PM +0100, Thomas Gleixner wrote:
> On Fri, Mar 12 2021 at 07:36, Ira Weiny wrote:
> > On Fri, Mar 12, 2021 at 07:54:13AM +0100, Christoph Hellwig wrote:
> >> So with the new kmap_local interface is it possible / advisable to
> >> use local kmaps over code that might schedule(), e.g. to wait for I/O?
> >
> > It is possible yes.  "Advisable" I think so.  Thomas knows better than I what
> > performance would be.
> 
> The performance impact is only relevant for 32bit machines with HIGHMEM
> enabled. On 64bit kmap_local is a NOOP except when you enable
> CONFIG_DEBUG_KMAP_LOCAL there which then actually does the mapping (on
> x86 only for now) with guard maps between and that will do the 'drop
> map' on schedule out and restore on schedule in dance. But for regular
> 64bit there is absolutely _NO_ overhead.

Thanks for clarification.

> 
> > FWIW I have been working on converting kmaps to kmap_local.  Most of the
> > instances don't schedule AFAICT.
> 
> The whole point of kmap_local is to be preemptible and does not have the
> nasty side effects of kmap_atomic, except for the 1 page per map and map
> nesting ordering requirements.
> 
> > What I really don't want to see is any kmap'ings handed to another thread.  I
> > am working hard to eliminate the use of kmap for that use.  Is that going to be
> > a problem?
> 
> kmap != kmap_local
> 
> kmap_local mappings are thread local and can't be handed anywhere. It's
> documented.

Agreed.  I'm Sorry, I did not word the above clearly enough.  Let me rephrase
that.

Christoph, do you anticipate additional need to call kmap and hand the mappings
to other threads?  If not then kmap_local is what you should use.  If so, I'd
like to know why.

Ira

> 
> Thanks,
> 
>         tglx

Re: kmap_local semantics

[Christoph Hellwig]

On Wed, Mar 17, 2021 at 07:44:20PM -0700, Ira Weiny wrote:
> Agreed.  I'm Sorry, I did not word the above clearly enough.  Let me rephrase
> that.
> 
> Christoph, do you anticipate additional need to call kmap and hand the mappings
> to other threads?  If not then kmap_local is what you should use.  If so, I'd
> like to know why.

No.  Just looking into callers that want a mapping and then synchronously
wait for I/O.