linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Andy Lutomirski <luto@kernel.org>
To: Len Brown <lenb@kernel.org>
Cc: Andy Lutomirski <luto@kernel.org>, Willy Tarreau <w@1wt.eu>,
	Florian Weimer <fweimer@redhat.com>,
	"Bae, Chang Seok" <chang.seok.bae@intel.com>,
	Dave Hansen <dave.hansen@intel.com>, X86 ML <x86@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-abi@vger.kernel.org,
	"libc-alpha@sourceware.org" <libc-alpha@sourceware.org>,
	Rich Felker <dalias@libc.org>, Kyle Huey <me@kylehuey.com>,
	Keno Fischer <keno@juliacomputing.com>
Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features
Date: Thu, 15 Apr 2021 09:24:35 -0700	[thread overview]
Message-ID: <CALCETrUxc1Y=4XmNthSSCeQfJ_SKNS762Nt3B-CynE-rhRSBGw@mail.gmail.com> (raw)
In-Reply-To: <CAJvTdKmppOcyy=akSo3+dn0KN5GmOKLK5cp3eZ_ymujE-=ExwQ@mail.gmail.com>

On Wed, Apr 14, 2021 at 2:48 PM Len Brown <lenb@kernel.org> wrote:
>

>
> > Then I take the transition penalty into and out of AMX code (I'll
> > believe there is no penalty when I see it -- we've had a penalty with
> > VEX and with AVX-512) and my program runs *slower*.
>
> If you have a clear definition of what "transition penalty" is, please share it.

Given the generally awful state of Intel's documentation about these
issues, it's quite hard to tell for real.  But here are some examples.

VEX: Figures 11-1 ("AVX-SSE Transitions in the Broadwell, and Prior
Generation Microarchitectures") and 11-2 ("AVX-SSE Transitions in the
Skylake Microarchitecture").  We *still* have a performance regression
in the upstream kernel because, despite all common sense, the CPUs
consider LDMXCSR to be an SSE instruction and VLDMXCSR to be an AVX
instruction despite the fact that neither one of them touch the XMM or
YMM state at all.

AVX-512:

https://lore.kernel.org/linux-crypto/CALCETrU06cuvUF5NDSm8--dy3dOkxYQ88cGWaakOQUE4Vkz88w@mail.gmail.com/

https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html





>
> Lacking one, I'll assume you are referring to the
> impact on turbo frequency of using AMX hardware?
>
> Again...
>
> On the hardware that supports AMX, there is zero impact on frequency
> due to the presence of AMX state, whether modified or unmodified.
>
> We resolved on another thread that Linux will never allow entry
> into idle with modified AMX state, and so AMX will have zero impact
> on the ability of the process to enter deep power-saving C-states.
>
> It is true that AMX activity is considered when determining max turbo.
> (as it must be)
> However, the *release* of the turbo credits consumed by AMX is
> "several orders of magnitude" faster on this generation
> than it was for AVX-512 on pre-AMX hardware.

What is the actual impact of a trivial function that initializes the
tile config, does one tiny math op, and then does TILERELEASE?

> Yes, the proposal, and the working patch set on the list, context
> switches XFD -- which is exactly what that hardware was designed to do.
> If the old and new tasks have the same value of XFD, the MSR write is skipped.
>
> I'm not aware of any serious proposal to context-switch XCR0,
> as it would break the current programming model, where XCR0
> advertises what the OS supports.  It would also impact performance,
> as every write to XCR0 necessarily provokes a VMEXIT.

You're arguing against a nonsensical straw man.

In the patches, *as submitted*, if you trip the XFD #NM *once* and you
are the only thread on the system to do so, you will eat the cost of a
WRMSR on every subsequent context switch.  This is not free.  If we
use XCR0 (I'm not saying we will -- I'm just mentioning at a
possibility), then the penalty is presumably worse due to the VMX
issue.

  reply	other threads:[~2021-04-15 16:24 UTC|newest]

Thread overview: 130+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-26 23:12 Candidate Linux ABI for Intel AMX and hypothetical new related features Andy Lutomirski
2021-03-26 23:18 ` Andy Lutomirski
2021-03-27  3:39   ` Len Brown
2021-03-27  9:14     ` Borislav Petkov
2021-03-27  9:58     ` Greg KH
2021-03-29 15:47       ` Len Brown
2021-03-29 16:38         ` Len Brown
2021-03-29 16:48           ` Florian Weimer
2021-03-29 18:14           ` Andy Lutomirski
2021-03-29 18:16         ` Andy Lutomirski
2021-03-29 22:38           ` Len Brown
2021-03-30  5:08             ` Andy Lutomirski
2021-03-30  5:50               ` Noah Goldstein
2021-03-30 17:01               ` Len Brown
2021-03-30 17:05                 ` Andy Lutomirski
2021-03-30 17:56                   ` Len Brown
2021-03-30 19:12                     ` Dave Hansen
2021-03-30 20:20                       ` Andy Lutomirski
2021-03-30 20:42                         ` Len Brown
2021-03-30 22:01                           ` David Laight
2021-03-31 16:31                             ` Len Brown
2021-03-31 16:53                               ` Andy Lutomirski
2021-03-31 21:42                                 ` Robert O'Callahan
2021-03-31 22:11                                   ` Len Brown
2021-03-31 22:28                                 ` Len Brown
2021-03-31 22:45                                   ` Andy Lutomirski
2021-04-09 20:52                                     ` Len Brown
2021-04-09 21:44                                       ` Andy Lutomirski
2021-04-11 19:07                                         ` Len Brown
2021-04-12  7:59                                           ` David Laight
2021-04-12 12:19                                           ` Borislav Petkov
2021-04-12 17:14                                           ` Sean Christopherson
2021-03-31 22:52                                   ` Borislav Petkov
2021-04-09 20:55                                     ` Len Brown
2021-03-28  0:53   ` Thomas Gleixner
2021-03-29  7:27     ` Peter Zijlstra
2021-03-29 15:06     ` Dave Hansen
2021-03-31  8:24 ` Borislav Petkov
     [not found] ` <87lf9nk2ku.fsf@oldenburg.str.redhat.com>
2021-04-12 14:31   ` Borislav Petkov
2021-04-12 14:38     ` Florian Weimer
2021-04-12 15:08       ` Borislav Petkov
2021-04-12 15:10       ` Andy Lutomirski
2021-04-12 15:21   ` Andy Lutomirski
2021-04-12 23:46     ` Len Brown
2021-04-13  0:17       ` Thomas Gleixner
2021-04-13  1:25         ` Len Brown
2021-04-13  3:43       ` Willy Tarreau
2021-04-13 19:51         ` Len Brown
2021-04-14  9:58           ` Borislav Petkov
2021-04-14 10:06             ` Willy Tarreau
2021-04-14 10:08               ` Borislav Petkov
2021-04-14 21:57             ` Len Brown
2021-04-15  4:43               ` Borislav Petkov
2021-04-15  5:29                 ` Willy Tarreau
2021-04-15  5:47                   ` Borislav Petkov
2021-04-16 22:05                     ` Len Brown
2021-04-19 14:14                       ` Borislav Petkov
2021-04-19 18:18                         ` Len Brown
2021-04-19 19:15                           ` Borislav Petkov
2021-04-19 21:33                             ` Len Brown
2021-04-19 21:58                               ` Borislav Petkov
2021-04-23 19:35                                 ` Len Brown
2021-04-23 19:57                                   ` Borislav Petkov
2021-05-02 15:27                                     ` Len Brown
2021-05-03  5:18                                       ` Florian Weimer
2021-05-03 13:43                                         ` Dave Hansen
2021-05-03 13:47                                           ` Florian Weimer
2021-05-03 14:14                                             ` Dave Hansen
2021-05-07 18:44                                           ` Thomas Gleixner
2021-05-07 18:50                                             ` Andy Lutomirski
2021-05-07 19:22                                               ` Thomas Gleixner
2021-05-08  9:45                                       ` Thomas Gleixner
2021-05-18 20:39                                         ` Len Brown
2021-05-19 23:29                                           ` Andy Lutomirski
2021-05-20 19:16                                             ` Len Brown
2021-05-17  9:45                                       ` Thomas Gleixner
2021-05-17  9:56                                         ` Florian Weimer
2021-05-17 10:18                                           ` Thomas Gleixner
2021-05-21 16:29                                           ` Len Brown
2021-05-17 13:49                                         ` Arjan van de Ven
2021-05-20 15:35                                         ` Len Brown
2021-05-20 20:54                                           ` Thomas Gleixner
2021-05-20 21:13                                             ` Dave Hansen
2021-05-20 21:41                                               ` Len Brown
2021-05-20 22:53                                                 ` Dave Hansen
2021-05-21  9:41                                                   ` Thomas Gleixner
2021-05-21 14:44                                                   ` Florian Weimer
2021-05-21 14:49                                                     ` Peter Zijlstra
2021-06-23 15:06                                                       ` Florian Weimer
2021-06-23 23:11                                                         ` Len Brown
2021-06-28 10:14                                                           ` Enrico Weigelt, metux IT consult
2021-06-28 12:49                                                             ` Florian Weimer
2021-06-30 12:22                                                               ` Enrico Weigelt, metux IT consult
2021-06-30 12:41                                                                 ` Willy Tarreau
2021-06-30 13:55                                                                 ` Arjan van de Ven
2021-06-30 15:20                                                                   ` Len Brown
2021-06-30 15:25                                                                   ` Enrico Weigelt, metux IT consult
2021-05-21 16:14                                                     ` Dave Hansen
2021-05-21 16:19                                                       ` Florian Weimer
2021-05-21 16:26                                                         ` Len Brown
2021-05-21 16:28                                                         ` Dave Hansen
2021-05-21 16:31                                                         ` Andy Lutomirski
2021-05-21 19:10                                                           ` Thomas Gleixner
2021-05-21 20:07                                                             ` Andy Lutomirski
2021-05-21 21:43                                                               ` Thomas Gleixner
2021-05-21 22:07                                                             ` Len Brown
2021-05-21 22:46                                                               ` Thomas Gleixner
2021-05-21 23:31                                                                 ` Len Brown
2021-05-22  7:16                                                                   ` Florian Weimer
2021-05-22 23:55                                                                     ` Andy Lutomirski
2021-05-21 23:06                                                               ` Dave Hansen
2021-05-21 23:08                                                                 ` Len Brown
2021-05-21 19:05                                                         ` Thomas Gleixner
2021-05-20 21:22                                             ` Len Brown
2021-05-20 21:41                                               ` Thomas Gleixner
2021-05-20 21:49                                                 ` Len Brown
2021-05-21  9:26                                                   ` Thomas Gleixner
2021-04-19 23:52                               ` Paul Eggert
2021-04-13 20:16       ` Andy Lutomirski
2021-04-13 22:47         ` Len Brown
2021-04-13 22:58           ` Andy Lutomirski
2021-04-14 21:48             ` Len Brown
2021-04-15 16:24               ` Andy Lutomirski [this message]
2021-04-15 17:00                 ` Dave Hansen
2021-04-15 17:38                   ` Andy Lutomirski
2021-04-16 21:54                 ` Len Brown
2021-04-16 22:03                   ` Andy Lutomirski
2021-04-16 22:10                     ` Len Brown
2021-04-16 22:14                       ` Andy Lutomirski
2021-04-17  1:57                         ` Len Brown

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALCETrUxc1Y=4XmNthSSCeQfJ_SKNS762Nt3B-CynE-rhRSBGw@mail.gmail.com' \
    --to=luto@kernel.org \
    --cc=chang.seok.bae@intel.com \
    --cc=dalias@libc.org \
    --cc=dave.hansen@intel.com \
    --cc=fweimer@redhat.com \
    --cc=keno@juliacomputing.com \
    --cc=lenb@kernel.org \
    --cc=libc-alpha@sourceware.org \
    --cc=linux-abi@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=me@kylehuey.com \
    --cc=w@1wt.eu \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).