From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A28E7C2B9F2 for ; Sat, 22 May 2021 23:56:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 860CA61177 for ; Sat, 22 May 2021 23:56:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231478AbhEVX5Y (ORCPT ); Sat, 22 May 2021 19:57:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:42482 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231440AbhEVX5X (ORCPT ); Sat, 22 May 2021 19:57:23 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 2198B61182 for ; Sat, 22 May 2021 23:55:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1621727758; bh=6dXbeile7lzsbpeK/x/mWQ+Zr2cMohG3GFCCGvZUMNM=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=t2dtZYHN4jh0HJtjGL9PAqNUzbTubKtHgUnjKOMeNKvll0rGjq+uk2A0wZbdmyj9y WqvTgKrk4NdcBHxcDIBhCvSarLpFmGF7RzbXCzugFR9Kx8Acgy+lNd10nsb48I6vSQ IzQwSmcLXwu1AqyAk27BtDGeWtrEsFicsPp+vWObkdVLtqQQkrXwvYWPd94VOjQkfi EIQls3REdhYS3+NRof0yu4Lc4XO8M6CLAwnQefbysKWpD51ecrNKoOe2SM28jKFKb5 IYnn571R5veecTo1tC2uSpVOUTLwmLfKDEw/8eL1ntnFmorFUswG6wLpWCVTg1A+SM nfr3XqBQoZHhA== Received: by mail-ej1-f46.google.com with SMTP id lz27so36013383ejb.11 for ; Sat, 22 May 2021 16:55:58 -0700 (PDT) X-Gm-Message-State: AOAM532UVnKfag8GYXJnAbH9fC0BG2B33b2XFMmJGCOGF7g/6UCOeG1p XK657ex1+r1XaKtTEwCUxXC/I07ZC4Vrf01bGhvZIA== X-Google-Smtp-Source: ABdhPJxJkfTVZX1D7O/qi5DCkclvDa1gfvKJMw3NcfhZGA89F9ozJOuPevIQdleeWUC5lmn1dO2QP7wG/n5rEjVIIns= X-Received: by 2002:a17:906:c010:: with SMTP id e16mr16823348ejz.214.1621727756732; Sat, 22 May 2021 16:55:56 -0700 (PDT) MIME-Version: 1.0 References: <20210415044258.GA6318@zn.tnic> <20210419191539.GH9093@zn.tnic> <20210419215809.GJ9093@zn.tnic> <874kf11yoz.ffs@nanos.tec.linutronix.de> <87k0ntazyn.ffs@nanos.tec.linutronix.de> <37833625-3e6b-5d93-cc4d-26164d06a0c6@intel.com> <9c8138eb-3956-e897-ed4e-426bf6663c11@intel.com> <87pmxk87th.fsf@oldenburg.str.redhat.com> <939ec057-3851-d8fb-7b45-993fa07c4cb5@intel.com> <87r1i06ow2.fsf@oldenburg.str.redhat.com> <263a58a9-26d5-4e55-b3e1-3718baf1b81d@www.fastmail.com> <87k0nraonu.ffs@nanos.tec.linutronix.de> <878s47aeni.ffs@nanos.tec.linutronix.de> <877djr5jc3.fsf@oldenburg.str.redhat.com> In-Reply-To: <877djr5jc3.fsf@oldenburg.str.redhat.com> From: Andy Lutomirski Date: Sat, 22 May 2021 16:55:45 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features To: Florian Weimer Cc: Len Brown , Thomas Gleixner , Andy Lutomirski , Dave Hansen , Dave Hansen via Libc-alpha , Rich Felker , Linux API , "Bae, Chang Seok" , "the arch/x86 maintainers" , Linux Kernel Mailing List , Kyle Huey , Borislav Petkov , Keno Fischer , Arjan van de Ven , Willy Tarreau Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On May 22, 2021, at 12:17 AM, Florian Weimer wrote: > > =EF=BB=BF* Len Brown: > >> A. per-task. If we do it this way, then we will likely wind up >> mandating a GET at the start of every routine in every library that >> touches AMX, and potentially also a PUT. This is because the library >> has no idea what thread called it. The plus is that this will address >> the "used once and sits on a buffer for the rest of the process >> lifetime' scenario. The minus is that high performance users will be >> executing thousands of unnecessary system calls that have zero value. > > We could revive the KTLS proposal (userspace donates memory for use by > the kernel & vDSO), and the thread could reserve (on-stack) buffer space > for kernel use for the duration of the AMX computation. There would be > a pointer to that space in the KTLS area, set upon entry of the AMX > region, and cleared upon exit. It's not extremely cheap (unbounded > alloca has a stack probing loop nowadays). But no system call is > required. > Making this work well would be very nasty. The memory *must* be available at context switch out time, which means it would need to be pinned at context switch in time, which is not great. But also Intel, in its infinite wisdom, decided to mix =E2=80=9Csupervisor= =E2=80=9D states in which the state that user space is permitted to directly access. Putting the supervisor state on the stack would be problematic.