Re: Candidate Linux ABI for Intel AMX and hypothetical new related features

From: Andy Lutomirski <luto@kernel.org>
To: Len Brown <lenb@kernel.org>, Willy Tarreau <w@1wt.eu>
Cc: Andy Lutomirski <luto@kernel.org>,
	Florian Weimer <fweimer@redhat.com>,
	"Bae, Chang Seok" <chang.seok.bae@intel.com>,
	Dave Hansen <dave.hansen@intel.com>, X86 ML <x86@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-abi@vger.kernel.org,
	"libc-alpha@sourceware.org" <libc-alpha@sourceware.org>,
	Rich Felker <dalias@libc.org>, Kyle Huey <me@kylehuey.com>,
	Keno Fischer <keno@juliacomputing.com>
Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features
Date: Tue, 13 Apr 2021 13:16:27 -0700	[thread overview]
Message-ID: <CALCETrVvapzL79BQNEvOupMHHzriR+n97955tRA+TPE6rgRC4Q@mail.gmail.com> (raw)
In-Reply-To: <CAJvTdKkAzEeAKrEYMU-gBWXoNGyJ09ZGw1gsU0b3uCuo8vrX0A@mail.gmail.com>

On Mon, Apr 12, 2021 at 4:46 PM Len Brown <lenb@kernel.org> wrote:
>
> On Mon, Apr 12, 2021 at 11:21 AM Andy Lutomirski <luto@kernel.org> wrote:
>
> > AMX: Multiplying a 4x4 matrix probably looks *great* in a
> > microbenchmark.  Do it once and you permanently allocate 8kB (is that
> > even a constant?  can it grow in newer parts?), potentially hurts all
> > future context switches, and does who-knows-what to Turbo licenses and
> > such.
>
> Intel expects that AMX will be extremely valuable to key workloads.
> It is true that you may never run that kind of workload on the machine
> in front of you,
> and so you have every right to be doubtful about the value of AMX.

I fully believe that AMX will be amazing when used for the right
workload.  The problem is that a library may have no way to tell
whether a workload is the type of computationally intensive workload
for which it makes sense.  Imagine you have a little function:

int matrix_times_vector(int dim, float *out, const float *matrix,
const float *vector);

A clever library might use AMX for this.  If dim == 4 and the caller
is planning to call it in a long, tight loop, maybe this even makes
sense.  If dim == 4 and it's being called once, AMX is probably a
losing proposition.  With previous technologies, at least the impact
was limited to the function itself and maybe once per call to the
caller.  But now, with AMX, the program that invoked this takes a
performance and memory hit *forever* if it uses AMX once.

Beyond that, we have the signal handling issue.  One solution, going
off of what WIlly mentioned, is:

bool amx_begin(void *signal_save_buffer);
void amx_end();

In the amx_begin() region, if you get a signal, the AMX state is saved
in the buffer.  Outside the region, if you get a signal and AMX is in
use, the kernel will either unceremoniously kill the task or will
deliver SIGYOUBLEWIT. [0]

I'm really hoping some userspace people can chime in.

[0] We really ought to have a SIGSIGNALFAILURE or something for the
case where normal signal delivery fails.  This is the userspace
equivalent of #DF.  SIGYOUBLEWIT could be folded in.  There would be a
flag in the signal frame saying "don't even try to sigreturn".