From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E59FC433E0 for ; Wed, 3 Mar 2021 19:20:12 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F1D264EF4 for ; Wed, 3 Mar 2021 19:20:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F1D264EF4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Hb+Ap0FuPQGM1s/F3TTwysOGa4ocTORJ+tecMDCJbjE=; b=Mp++R1ubCVwRs3weAv2HLLo73 li5xyqRU8DfoNMw4zRBvCN4yklArvCgcaNgt1gPrywRRobPh9yQti1bUtqGmNcB5/IrYvrbBSPhzM o6Cz05BM6SMc7V5fTw59e4W9CEVLZEyxU4dW056yHOpzvYUZGpzHmKp836yI+HWnLZf0+0ctLawdm +lVQsjWBYe5CNcu0NPFQHPXIpEQh8gLVRtsYmpA5mW9DIU8rA7wn1Xej0C0lxsli+xoE8jjVjLeiF Dmzyi68QdsE/DfGrWtUlrcIQGzzZ4FNm129rVTEHR5HM2TvI1Z6E1Y1cR/5L6766+RO86BYHFRHvH 9HhHUmjeg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHWzz-006Awn-11; Wed, 03 Mar 2021 19:17:32 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT2r-005Hrn-3n for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:04:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:List-Id:To:Subject: Message-ID:Date:From:In-Reply-To:References:MIME-Version:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=1FsybmuyP5cOjkfNe0e0f6tZbBSYUS5F5BusqJHA1Oo=; b=ndXlO0hohnCOkmCSDty3xuj7+I vsZ+i38pmiFNK2iMMi+k1Ah6PT9KqXFEZEvAKVfrpF4lYRc4ok3AhgX33BtKEbakn9WEevZTkyHGw U8Gwzj7Esgqd643qjLU9rzhpiq8YHekDgbIaA/bduuks0d6o4IXKGCAoOMxxj6+djrJlTdIz65kuI syTXDKMsyKCA+/JJ01DsGCqo7nYmzZjNcFZNX3V9HPv+aSp1GnNfOTFtu9mwONbCbqiRkviuMiAcr Oh91Eb/pecEbQwytrHWL+uEDFQE1TYEeAFY9+9iUYdZc+MnDltGlvwp72wBzBQkklihFhzomgT3QE YtSTtDcA==; Received: from mail.kernel.org ([198.145.29.99]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lH30w-00GzrA-Pu for linux-arm-kernel@lists.infradead.org; Tue, 02 Mar 2021 11:16:32 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id A12AE64F07 for ; Tue, 2 Mar 2021 11:07:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1614683235; bh=JXsntDomzvvp35KqPtyck4SYOcKp8C4CKAHwX45dSbk=; h=References:In-Reply-To:From:Date:Subject:To:List-Id:Cc:From; b=KtChTbLobWxE790ocN6tY6pzK8us82tmSNZzHL0pjxAF1zvqb2b4lE/TH2q3Rtz5p tWRAk/mT0xENLL2LzwfWbH6pSyi+Aiw2pqAeDc8v4ecyikzgY/aCl8c6PGP27BsG3J wqpfga9299ZfCFQgLl/VZFxLjUjG+SFJqDQth1SYd0M+oEjoJMFTMsQ2N0gzbiMfOn 0IkDiQKvyWKNAetPBYrwb1qwLbMrfJAloknLJkKF6KLkNoLjoQmxF4J3NOz3JN5BjV EnaETFLFMCbYyBzRH10h8dlIkYlUPcSX0ENWHEnOiPw2H1RoxQcSJE3fdQkNSAVq06 POo9NPkAg6g7g== Received: by mail-ot1-f51.google.com with SMTP id f33so19553438otf.11 for ; Tue, 02 Mar 2021 03:07:15 -0800 (PST) X-Gm-Message-State: AOAM5329vzbJAMWDzxfghKv7l59CSQAYR5qQKLFtidOcydexUX1KW9NX bz/i4xFzPF07MMlEwBB8vT5J4kBnwyqmIiIFPbQ= X-Google-Smtp-Source: ABdhPJxhU9eCujXvv4W27bbuQrN4TqJbps/AVuDpVM+qdE75c882pb9q50Qcmq/ml0VnXFoQJVtbrz+VtQpfjeZra24= X-Received: by 2002:a9d:7f11:: with SMTP id j17mr18038402otq.251.1614683234751; Tue, 02 Mar 2021 03:07:14 -0800 (PST) MIME-Version: 1.0 References: <20210108105241.1757799-1-misono.tomohiro@jp.fujitsu.com> <20210108125410.GA84941@C02TD0UTHF1T.local> In-Reply-To: From: Arnd Bergmann Date: Tue, 2 Mar 2021 12:06:57 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC PATCH 00/10] Add Fujitsu A64FX soc entry/hardware barrier driver To: "misono.tomohiro@fujitsu.com" List-Id: Cc: Mark Rutland , Catalin Marinas , SoC Team , Olof Johansson , Will Deacon , Linux ARM X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210302_111632_453843_804E0CD7 X-CRM114-Status: GOOD ( 35.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Feb 18, 2021 at 10:49 AM misono.tomohiro@fujitsu.com wrote: > > > > > > Also, It is common usage that each running thread is bound to one PE in > > > > > multi-threaded HPC applications. > > > > > > > > I think the expectation that all threads are bound to a physical CPU > > > > makes sense for using this feature, but I think it would be necessary > > > > to enforce that, e.g. by allowing only threads to enable it after they > > > > are isolated to a non-shared CPU, and automatically disabling it > > > > if the CPU isolation is changed. > > > > > > > > For the user space interface, something based on process IDs > > > > seems to make more sense to me than something based on CPU > > > > numbers. All of the above does require some level of integration > > > > with the core kernel of course. > > > > > > > > I think the next step would be to try to come up with a high-level > > > > user interface design that has a chance to get merged, rather than > > > > addressing the review comments for the current implementation. > > Hello, > > Sorry for late response but while thinking new approaches, I come up with > some different idea and want to hear your opinions. How about offload > all control to user space while the driver just offers read/write access > to the needed registers? Let me explain in detail. > > Although I searched similar functions in other products, I could not find > it. Also, this hardware barrier performs intra-numa synchronization and > it is hard to be used for general inter-process barrier. So I think > generalizing this feature in kernel does not go well. Ok, thank you for taking a look. > As I said this is mainly for HPC application. In the usual situations, the > user has full control of the PC nodes when running HPC application and > thus the user has full responsibility of running processes on the machine. > Offloading all controls to these registers to the user is acceptable in that > case (i.e. the driver just offers access to the registers and does not control it). > This is the safe for the kernel operation as manipulating barrier related > registers just affects user application. > > In this approach we could remove ioctls or control logic in the driver but > we need some way to access the needed registers. I firstly think if I can > use x86's MSR driver like approach but I know the idea is rejected > recently for security concerns: > https://lore.kernel.org/linux-arm-kernel/20201130174833.41315-1-rongwei.wang@linux.alibaba.com/ > > Based on these observations, I have two ideas currently: > 1) make the driver to only expose sysfs interface for reading/writing > A64FX's barrier registers > or > 2) generalizing (1) in some way; To make some mechanism to expose > CPU defined registers which can be safely accessed from user space > > Are these idea acceptable ways to explore to get merged in upstream? > I'd appreciate any criticism/comments. I'm also running out of ideas here. I don't think a sysfs interface would be any different to your earlier ioctl interface or the the /dev/msr approach, they all share the same problem that they expose low-level access to platform specific registers in a way that is neither portable nor safe to use for general-purpose applications outside the very narrow scope of running highly optimized HPC applications. You can of course continue using the module you have as an external module that gets built outside of the kernel itself, and shipped along with the application or library using it, rather than with the kernel. Obviously this is not something I would generally recommend either, but this may be the last resort to fall back to when everything else fails. Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel