TLB flush question (x86_64)

TLB flush question (x86_64)

[Arkaprava Basu]

Hello,

     I am using 2.6.31 kernel on x86_64 (with SMP configuration). I
have a confusion on a TLB flush function.
     I found that the method tlb_flush_mmu (include/asm-generic/tlb.h)
, which is called by many system calls like munmap,madvise,mprotect
etc; ignores the virtual address range (start and end parameters) for
which it is called. It internally calls tlb_flush method, which on x86
gets converted to flush_tlb_mm. It seems that flush_tlb_mm flushes out
all the mappings for a given mm_struct (i.e., the whole user address
space for a process) from the tlb. Is there any particular reason why
the entire address space is flushed from the tlb while only a range of
virtual address space was supposedly required to be flushed? For
example why the method flush_tlb_range , which would have flushed
mappings belonging to a given virtual address range only could not
have been used instead of flush_tlb_mm? Is there any performance or
some other correctness related concern for not doing that?

Thanks
Arka

Re: TLB flush question (x86_64)

[Andreas Mohr]

Hi,

decided to reply since there hasn't been any activity yet.

> Is there any particular reason why
> the entire address space is flushed from the tlb while only a range of
> virtual address space was supposedly required to be flushed?

While this particular function has actually changed in less medieval
kernel versions, it would be nice if someone knew an answer to that,
especially since TLB activity may have grave performance implications.

Andreas Mohr

Re: TLB flush question (x86_64)

[Brian Gerst]

On Sat, Sep 17, 2011 at 3:12 PM, Andreas Mohr <andi@lisas.de> wrote:
> Hi,
>
> decided to reply since there hasn't been any activity yet.
>
>> Is there any particular reason why
>> the entire address space is flushed from the tlb while only a range of
>> virtual address space was supposedly required to be flushed?
>
> While this particular function has actually changed in less medieval
> kernel versions, it would be nice if someone knew an answer to that,
> especially since TLB activity may have grave performance implications.
>

x86 hardware can only flush single pages or the whole TLB.  It would
be possible to loop over a small set of pages and invalidate each one,
but the threshold for where that becomes more efficient than flushing
everything is difficult to determine.

--
Brian Gerst

Re: TLB flush question (x86_64)

[Borislav Petkov]

On Sat, Sep 17, 2011 at 07:37:59PM -0400, Brian Gerst wrote:
> On Sat, Sep 17, 2011 at 3:12 PM, Andreas Mohr <andi@lisas.de> wrote:
> > Hi,
> >
> > decided to reply since there hasn't been any activity yet.
> >
> >> Is there any particular reason why
> >> the entire address space is flushed from the tlb while only a range of
> >> virtual address space was supposedly required to be flushed?
> >
> > While this particular function has actually changed in less medieval
> > kernel versions, it would be nice if someone knew an answer to that,
> > especially since TLB activity may have grave performance implications.
> >
> 
> x86 hardware can only flush single pages or the whole TLB.  It would
> be possible to loop over a small set of pages and invalidate each one,
> but the threshold for where that becomes more efficient than flushing
> everything is difficult to determine.

... also, TLBs in current CPUs are highly optimized beasts so that the
cost of a full rewalk is paid only very seldomly by caching even partial
translations in lower level TLBs (L2), etc.

And also, flush_tlb_mm ends up modifying CR3 which doesn't flush the
global mappings.

-- 
Regards/Gruss,
    Boris.