l1tf: Kernel suggests I throw away third of my memory. I'd rather not

l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

Hi!

6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not
going to be pleasant.

l1tf.html says:

# The Linux kernel contains a mitigation for this attack vector, PTE
# inversion, which is permanently enabled and has no performance
# impact.

I don't believe it has "no" performance impact, but I guess it is lost
in the noise.

#  The kernel ensures that the address bits of PTEs, which are
# not marked present, never point to cacheable physical memory space.

# A system with an up to date kernel is protected against attacks from
# malicious user space applications.

These are not true.

cat /sys/devices/system/cpu/vulnerabilities/l1tf
Vulnerable
uname -a
Linux amd 4.19.0-rc8-next-20181017autobisect1539371050 #189 SMP Wed
Oct 17 12:04:23 CEST 2018 i686 GNU/Linux

Now question is... can we do better? Kernel stores information about
swapped-out pages there, right? That sounds like a cool hack, but
maybe it is time to get rid of that hack?

As a workaround, can I simply do swapoff -a to be safe for now?

Thanks,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Michal Hocko]

On Wed 17-10-18 12:56:10, Pavel Machek wrote:
> Hi!
> 
> 6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not
> going to be pleasant.
> 
> l1tf.html says:
> 
> # The Linux kernel contains a mitigation for this attack vector, PTE
> # inversion, which is permanently enabled and has no performance
> # impact.
> 
> I don't believe it has "no" performance impact, but I guess it is lost
> in the noise.

Please prove otherwise. I would be more than surprised if inverting pfn
part of the pte is noticeable. But I can be wrong of course.

> #  The kernel ensures that the address bits of PTEs, which are
> # not marked present, never point to cacheable physical memory space.
> 
> # A system with an up to date kernel is protected against attacks from
> # malicious user space applications.
> 
> These are not true.
> 
> cat /sys/devices/system/cpu/vulnerabilities/l1tf
> Vulnerable
> uname -a
> Linux amd 4.19.0-rc8-next-20181017autobisect1539371050 #189 SMP Wed
> Oct 17 12:04:23 CEST 2018 i686 GNU/Linux

This is a result of you having memory above MAX_PFN/2 right?

> Now question is... can we do better? Kernel stores information about
> swapped-out pages there, right? That sounds like a cool hack, but
> maybe it is time to get rid of that hack?

Patches are welcome.
 
> As a workaround, can I simply do swapoff -a to be safe for now?

Well, that depends. Do you care about PROT_NONE attacks as well? If not
then no-swap would help you. But even then no-swap is rather theoretical
attack on a physical host unless you allow an arbitrary swapout to a
malicious user (e.g. allow a user controlled memcg hard limit that would
cause excessive local swapouts).

-- 
Michal Hocko
SUSE Labs

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2418 bytes --]

Hi!

> > 6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not
> > going to be pleasant.
> > 
> > l1tf.html says:
> > 
> > # The Linux kernel contains a mitigation for this attack vector, PTE
> > # inversion, which is permanently enabled and has no performance
> > # impact.
> > 
> > I don't believe it has "no" performance impact, but I guess it is lost
> > in the noise.
> 
> Please prove otherwise. I would be more than surprised if inverting pfn
> part of the pte is noticeable. But I can be wrong of course.

I'm not saying its noticeable. I'm saying that inversion takes few
clock cycles (including a branch?) and that neither caches nor RAM is
free.

"no noticeable performance impact" I'd agree with :).

> > #  The kernel ensures that the address bits of PTEs, which are
> > # not marked present, never point to cacheable physical memory space.
> > 
> > # A system with an up to date kernel is protected against attacks from
> > # malicious user space applications.
> > 
> > These are not true.
> > 
> > cat /sys/devices/system/cpu/vulnerabilities/l1tf
> > Vulnerable
> > uname -a
> > Linux amd 4.19.0-rc8-next-20181017autobisect1539371050 #189 SMP Wed
> > Oct 17 12:04:23 CEST 2018 i686 GNU/Linux
> 
> This is a result of you having memory above MAX_PFN/2 right?

Yes.

> > Now question is... can we do better? Kernel stores information about
> > swapped-out pages there, right? That sounds like a cool hack, but
> > maybe it is time to get rid of that hack?
> 
> Patches are welcome.

Cooperation will be needed if you want to see patches. As
in... answering the questions above.

> > As a workaround, can I simply do swapoff -a to be safe for now?
> 
> Well, that depends. Do you care about PROT_NONE attacks as well? If not
> then no-swap would help you. But even then no-swap is rather theoretical
> attack on a physical host unless you allow an arbitrary swapout to a
> malicious user (e.g. allow a user controlled memcg hard limit that would
> cause excessive local swapouts).

PROT_NONE attack.. aha, so kernel stores not only information about
swapped-out pages but also about file-backed pages that are currently
not present? Hmm. That makes it more complex :-(. 

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Michal Hocko]

On Wed 17-10-18 13:32:26, Pavel Machek wrote:
[...]
> > > Now question is... can we do better? Kernel stores information about
> > > swapped-out pages there, right? That sounds like a cool hack, but
> > > maybe it is time to get rid of that hack?
> > 
> > Patches are welcome.
> 
> Cooperation will be needed if you want to see patches. As
> in... answering the questions above.

The question is whether that is really worth it. L1TF is mostly about
virtual environments. If you are running in a native HW then I wouldn't
lose much sleep over it. a) pfns stored in PROT_NONE entries are
controlled by the OS b) swap based attacks with something interesting in
L1$ colliding with the swap entry is theoretical at best.
-- 
Michal Hocko
SUSE Labs

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Andi Kleen]

On Wed, Oct 17, 2018 at 12:56:10PM +0200, Pavel Machek wrote:
> Hi!
> 
> 6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not
> going to be pleasant.

Just rebuild your kernel with PAE? I assume your CPU supports it.

This will also give you NX, which if you're really worried
about security is far more important than L1TF.

If you don't worry about security just ignore.

-Andi

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Vlastimil Babka]

On 10/17/18 4:08 PM, Andi Kleen wrote:
> On Wed, Oct 17, 2018 at 12:56:10PM +0200, Pavel Machek wrote:
>> Hi!
>>
>> 6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not
>> going to be pleasant.
> 
> Just rebuild your kernel with PAE? I assume your CPU supports it.

I think it is built with PAE or this would kick in:

#if CONFIG_PGTABLE_LEVELS == 2
        pr_warn("Kernel not compiled for PAE. No mitigation for L1TF\n");
        return;
#endif

I.e. no MAX_PA/2 messages.

> This will also give you NX, which if you're really worried
> about security is far more important than L1TF.
> 
> If you don't worry about security just ignore.
> 
> -Andi
> 

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Dave Hansen]

[-- Attachment #1.1: Type: text/plain, Size: 833 bytes --]

On 10/17/2018 04:32 AM, Pavel Machek wrote:
>> Well, that depends. Do you care about PROT_NONE attacks as well? If not
>> then no-swap would help you. But even then no-swap is rather theoretical
>> attack on a physical host unless you allow an arbitrary swapout to a
>> malicious user (e.g. allow a user controlled memcg hard limit that would
>> cause excessive local swapouts).
> PROT_NONE attack.. aha, so kernel stores not only information about
> swapped-out pages but also about file-backed pages that are currently
> not present? Hmm. That makes it more complex :-(. 

There are also migration PTE entries that are "swap-like".  They can
exist even if you swapoff -a.

Can we do better?  Sure.  I think we'd all be happy to review patches
that improve the situation if folks have simple ideas for improvement.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: l1tf: Kernel suggests I throw away third of my memory. I'd rather not

[Vlastimil Babka]

On 10/18/18 12:21 AM, Dave Hansen wrote:
> On 10/17/2018 04:32 AM, Pavel Machek wrote:
>>> Well, that depends. Do you care about PROT_NONE attacks as well? If not
>>> then no-swap would help you. But even then no-swap is rather theoretical
>>> attack on a physical host unless you allow an arbitrary swapout to a
>>> malicious user (e.g. allow a user controlled memcg hard limit that would
>>> cause excessive local swapouts).
>> PROT_NONE attack.. aha, so kernel stores not only information about
>> swapped-out pages but also about file-backed pages that are currently
>> not present? Hmm. That makes it more complex :-(. 
> 
> There are also migration PTE entries that are "swap-like".  They can
> exist even if you swapoff -a.
> 
> Can we do better?  Sure.  I think we'd all be happy to review patches
> that improve the situation if folks have simple ideas for improvement.

I think if would be rather unfortunate changing this fundamentally due
to a HW bug that should hopefully eventually go away with new CPUs, and
with MAX_PA/2 being limited only on very old CPUs. I don't think using
page table entries like this is a hack. The pte must IMHO always carry
some information that identifies is as swap entry or prot_none or
whatever. If we had to look up the swap offset (or pfn associated with
PROT_NONE pte elsewhere), there would be additional overhead both for
storing this mapping and for looking it up.
PTEs seems just like a natural place for this IMHO, and it's defined
that the bits can be used by the kernel however it likes for !present
PTEs. Why rewrite it all just because some CPUs implementation is not as
defined...