linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Oded Gabbay <ogabbay@kernel.org>
To: Greg KH <gregkh@linuxfoundation.org>
Cc: Jiho Chu <jiho.chu@samsung.com>, Arnd Bergmann <arnd@arndb.de>,
	"Linux-Kernel@Vger. Kernel. Org" <linux-kernel@vger.kernel.org>,
	yelini.jeong@samsung.com, myungjoo.ham@samsung.com
Subject: Re: [PATCH 0/9] Samsung Trinity NPU device driver
Date: Mon, 25 Jul 2022 12:10:07 +0300	[thread overview]
Message-ID: <CAFCwf13JA+5vuAKqvBSs3MkcF-gbE_8vd9nSvStQga55vW80VA@mail.gmail.com> (raw)
In-Reply-To: <Yt5cFBgiTLwGXv17@kroah.com>

On Mon, Jul 25, 2022 at 12:02 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Mon, Jul 25, 2022 at 03:52:59PM +0900, Jiho Chu wrote:
> > Hello,
> >
> > My name is Jiho Chu, and working for device driver and system daemon for
> > several years at Samsung Electronics.
> >
> > Trinity Neural Processing Unit (NPU) series are hardware accelerators
> > for neural network processing in embedded systems, which are integrated
> > into application processors or SoCs. Trinity NPU is compatible with AMBA
> > bus architecture and first launched in 2018 with its first version for
> > vision processing, Trinity Version1 (TRIV1). Its second version, TRIV2,
> > is released in Dec, 2021. Another Trinity NPU for audio processing is
> > referred as TRIA.
> >
> > TRIV2 is shipped for many models of 2022 Samsung TVs, providing
> > acceleration for various AI-based applications, which include image
> > recognition and picture quality improvements for streaming video, which
> > can be accessed via GStreamer and its neural network plugins,
> > NNStreamer.
> >
> > In this patch set, it includes Trinity Vision 2 kernel device driver.
> > Trinity Vision 2 supports accelerating image inference process for
> > Convolution Neural Network (CNN). The CNN workload is executed by Deep
> > Learning Accelerator (DLA), and general Neural Network Layers are
> > executed by Digital Signal Processor (DSP). And there is a Control
> > Processor (CP) which can control DLA and DSP. These three IPs (DLA, DSP,
> > CP) are composing Trinity Vision 2 NPU, and the device driver mainly
> > supervise the CP to manage entire NPU.
> >
> > Controlling DLA and DSP operations is performed with internal command
> > instructions. and the instructions for the Trinity is similar with
> > general processor's ISA, but it is specialized for Neural Processing
> > operations. The virtual ISA (vISA) is designed for calculating multiple
> > data with single operation, like modern SIMD processor. The device
> > driver loads a program to CP at start up, and the program can decode a
> > binary which is built with the vISA. We calls this decoding program as a
> > Instruction Decoding Unit (IDU) program. While running the NPU, the CP
> > executes IDU program to fetch and decode instructions which made up of
> > vISA, by the scheduling policy of the device driver.
> >
> > These DLA, DSP and CP are loosely coupled using ARM's AMBA, so the
> > Trinity can easily communicate with most ARM processors. Each IPs
> > designed to have memory-mapped registers which can be used to control
> > the IP, and the CP provides Wait-For-Event (WFE) operation to subscribe
> > interrupt signals from the DLA and DSP. Also, embedded Direct Memory
> > Access Controller (DMAC) manages data communications between internal
> > SRAM and outer main memory, IOMMU module supports unified memory space.
> >
> > A user can control the Trinity NPU with IOCTLs provided by driver. These
> > controls includes memory management operations to transfer model data
> > (HWMEM_ALLOC/HWMEM_DEALLOC), NPU workload control operations to submit
> > workload (RUN/STOP), and statistics operations to check current NPU
> > status. (STAT)
> >
> > The device driver also implemented features for developers. It provides
> > sysfs control attributes like stop, suspend, sched_test, and profile.
> > Also, it provides status attributes like app status, a number of total
> > requests, a number of active requests and memory usages. For the tracing
> > operations, several ftrace events are defined and embedded for several
> > important points.
>
> If you have created sysfs files, you need to document them in
> Documentation/ABI/ which I do not see in your diffstat.  Perhaps add
> that for your next respin?
>
> Also, please remove the "tracing" logic you have in the code, use
> ftrace, don't abuse dev_info() everywhere, that's not needed at all.
>
> thanks,
>
> greg k-h

Hi,
Why isn't this submitted to soc/ subsystem ?
Don't you think that would be more appropriate, given that this IP is
integrated into application processors ?

Thanks,
Oded

  reply	other threads:[~2022-07-25  9:10 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20220725065308epcas1p2f6de3d74792854bb312cca4b310badac@epcas1p2.samsung.com>
2022-07-25  6:52 ` [PATCH 0/9] Samsung Trinity NPU device driver Jiho Chu
     [not found]   ` <CGME20220725065309epcas1p4565e7bb0fea1aaf3e5e300de00774c2a@epcas1p4.samsung.com>
2022-07-25  6:53     ` [PATCH 1/9] trinity: Add base driver Jiho Chu
2022-07-27 11:54       ` Krzysztof Kozlowski
2022-07-27 13:22       ` Greg KH
2022-09-01 19:04         ` Dafna Hirschfeld
2022-09-02  5:39           ` Greg KH
     [not found]       ` <CGME20220725065309epcas1p4565e7bb0fea1aaf3e5e300de00774c2a@epcms1p2>
2022-07-28  2:05         ` MyungJoo Ham
2022-09-01 18:36       ` Mark Brown
2022-09-02  8:23         ` Jiho Chu
     [not found]   ` <CGME20220725065309epcas1p3c691bbc54c84775524b97c4b717c7ce7@epcas1p3.samsung.com>
2022-07-25  6:53     ` [PATCH 2/9] tirnity: Add dma memory module Jiho Chu
2022-07-27 13:15       ` Greg KH
     [not found]   ` <CGME20220725065309epcas1p20c847655e7332c818fc0fd2c50fb0e27@epcas1p2.samsung.com>
2022-07-25  6:53     ` [PATCH 3/9] trinity: Add load/unload IDU files Jiho Chu
2022-07-27 13:14       ` Greg KH
2022-09-17  7:39         ` Jiho Chu
     [not found]   ` <CGME20220725065309epcas1p42ba84c5241d69192ea73904ed6af17d7@epcas1p4.samsung.com>
2022-07-25  6:53     ` [PATCH 4/9] trinity: Add schduler module Jiho Chu
2022-07-27 13:09       ` Greg KH
     [not found]   ` <CGME20220725065309epcas1p413498a418cbf58570f8009ae7fd91015@epcas1p4.samsung.com>
2022-07-25  6:53     ` [PATCH 5/9] trinity: Add sysfs debugfs module Jiho Chu
2022-07-27 13:25       ` Greg KH
     [not found]   ` <CGME20220725065310epcas1p3688f336bfd5c732145575524f3365a0f@epcas1p3.samsung.com>
2022-07-25  6:53     ` [PATCH 6/9] trinity: Add pm and ioctl feature Jiho Chu
     [not found]   ` <CGME20220725065310epcas1p2ae58294d9cf44e622ed8cc7e5a8b988c@epcas1p2.samsung.com>
2022-07-25  6:53     ` [PATCH 7/9] trinity: Add profile module Jiho Chu
     [not found]   ` <CGME20220725065310epcas1p1841fde6ae768a98543418c81790c0832@epcas1p1.samsung.com>
2022-07-25  6:53     ` [PATCH 8/9] trinity: Add trace module Jiho Chu
     [not found]   ` <CGME20220725065310epcas1p2735e463512b0db489f2af532f15dae6e@epcas1p2.samsung.com>
2022-07-25  6:53     ` [PATCH 9/9] MAINTAINERS: add TRINITY driver Jiho Chu
2022-07-25  9:02   ` [PATCH 0/9] Samsung Trinity NPU device driver Greg KH
2022-07-25  9:10     ` Oded Gabbay [this message]
     [not found]     ` <CGME20220725065308epcas1p2f6de3d74792854bb312cca4b310badac@epcms1p5>
2022-07-26  2:09       ` MyungJoo Ham
2022-07-26  6:59         ` Krzysztof Kozlowski
2022-07-26  7:51           ` Arnd Bergmann
2022-07-26 11:24             ` Oded Gabbay
2022-07-29 17:50             ` Pavel Machek
     [not found]       ` <20220726050305epcms1p5ef19a54322263c768ea71d59da7e2616@epcms1p5>
2022-07-26 14:35         ` 추지호/Robot Intelligence팀(SR)/Staff Engineer/삼성전자
2022-07-26  6:57   ` Krzysztof Kozlowski
2022-07-27 11:51     ` Jiho Chu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAFCwf13JA+5vuAKqvBSs3MkcF-gbE_8vd9nSvStQga55vW80VA@mail.gmail.com \
    --to=ogabbay@kernel.org \
    --cc=arnd@arndb.de \
    --cc=gregkh@linuxfoundation.org \
    --cc=jiho.chu@samsung.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=myungjoo.ham@samsung.com \
    --cc=yelini.jeong@samsung.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).