From: Jason Gunthorpe <jgg@ziepe.ca>
To: Oded Gabbay <oded.gabbay@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
izur@habana.ai, Gal Pressman <galpress@amazon.com>,
Jakub Kicinski <kuba@kernel.org>,
"Linux-Kernel@Vger. Kernel. Org" <linux-kernel@vger.kernel.org>,
netdev@vger.kernel.org, SW_Drivers <SW_Drivers@habana.ai>,
"David S. Miller" <davem@davemloft.net>,
Andrew Lunn <andrew@lunn.ch>,
Florian Fainelli <f.fainelli@gmail.com>,
linux-rdma@vger.kernel.org, Olof Johansson <olof@lixom.net>
Subject: Re: [PATCH v3 00/14] Adding GAUDI NIC code to habanalabs driver
Date: Fri, 18 Sep 2020 11:19:09 -0300 [thread overview]
Message-ID: <20200918141909.GU8409@ziepe.ca> (raw)
In-Reply-To: <CAFCwf13rJgb4=as7yW-2ZHvSnUd2NK1GP0UKKjyMfkB3vsnE5w@mail.gmail.com>
On Fri, Sep 18, 2020 at 05:12:04PM +0300, Oded Gabbay wrote:
> On Fri, Sep 18, 2020 at 4:59 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >
> > On Fri, Sep 18, 2020 at 04:49:25PM +0300, Oded Gabbay wrote:
> > > On Fri, Sep 18, 2020 at 4:26 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > > >
> > > > On Fri, Sep 18, 2020 at 04:02:24PM +0300, Oded Gabbay wrote:
> > > >
> > > > > The problem with MR is that the API doesn't let us return a new VA. It
> > > > > forces us to use the original VA that the Host OS allocated.
> > > >
> > > > If using the common MR API you'd have to assign a unique linear range
> > > > in the single device address map and record both the IOVA and the MMU
> > > > VA in the kernel struct.
> > > >
> > > > Then when submitting work using that MR lkey the kernel will adjust
> > > > the work VA using the equation (WORK_VA - IOVA) + MMU_VA before
> > > > forwarding to HW.
> > > >
> > > We can't do that. That will kill the performance. If for every
> > > submission I need to modify the packet's contents, the throughput will
> > > go downhill.
> >
> > You clearly didn't read where I explained there is a fast path and
> > slow path expectation.
> >
> > > Also, submissions to our RDMA qmans are coupled with submissions to
> > > our DMA/Compute QMANs. We can't separate those to different API calls.
> > > That will also kill performance and in addition, will prevent us from
> > > synchronizing all the engines.
> >
> > Not sure I see why this is a problem. I already explained the fast
> > device specific path.
> >
> > As long as the kernel maintains proper security when it processes
> > submissions the driver can allow objects to cross between the two
> > domains.
> Can you please explain what you mean by "two domains" ?
> You mean the RDMA and compute domains ? Or something else ?
Yes
> What I was trying to say is that I don't want the application to split
> its submissions to different system calls.
If you can manage the security then you can cross them. Eg since The
RDMA PD would be created on top of the /dev/misc char dev then it is
fine for the /dev/misc char dev to access the RDMA objects as a 'dv
fast path'.
But now that you say everything is interconnected, I'm wondering,
without HW security how do you keep netdev isolated from userspace?
Can I issue commands to /dev/misc and write to kernel memory (does the
kernel put any pages into the single MMU?) or corrupt the netdev
driver operations in any way?
Jason
next prev parent reply other threads:[~2020-09-18 14:19 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-15 17:10 [PATCH v3 00/14] Adding GAUDI NIC code to habanalabs driver Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 02/14] habanalabs/gaudi: add NIC firmware-related definitions Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 03/14] habanalabs/gaudi: add NIC security configuration Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 04/14] habanalabs/gaudi: add support for NIC QMANs Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 05/14] habanalabs/gaudi: add NIC Ethernet support Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 06/14] habanalabs/gaudi: add NIC PHY code Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 07/14] habanalabs/gaudi: allow user to get MAC addresses in INFO IOCTL Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 08/14] habanalabs/gaudi: add a new IOCTL for NIC control operations Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 09/14] habanalabs/gaudi: add CQ " Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 10/14] habanalabs/gaudi: add WQ " Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 11/14] habanalabs/gaudi: add QP error handling Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 12/14] habanalabs/gaudi: Add ethtool support using coresight Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 13/14] habanalabs/gaudi: support DCB protocol Oded Gabbay
2020-09-15 17:10 ` [PATCH v3 14/14] habanalabs/gaudi: add NIC init/fini calls from common code Oded Gabbay
2020-09-15 20:35 ` [PATCH v3 00/14] Adding GAUDI NIC code to habanalabs driver Jakub Kicinski
2020-09-15 20:46 ` Oded Gabbay
2020-09-15 21:04 ` Jakub Kicinski
2020-09-15 21:20 ` Oded Gabbay
2020-09-15 21:37 ` Andrew Lunn
2020-09-15 21:43 ` Oded Gabbay
2020-09-15 22:35 ` David Miller
2020-09-15 22:36 ` David Miller
2020-09-15 22:34 ` David Miller
2020-09-16 4:26 ` Oded Gabbay
2020-09-17 17:18 ` Jason Gunthorpe
2020-09-18 11:36 ` Gal Pressman
2020-09-18 11:52 ` Leon Romanovsky
2020-09-18 11:56 ` Oded Gabbay
2020-09-18 12:03 ` Leon Romanovsky
2020-09-18 12:07 ` Oded Gabbay
2020-09-18 12:19 ` Leon Romanovsky
2020-09-18 12:31 ` Oded Gabbay
2020-09-18 13:09 ` Leon Romanovsky
2020-09-19 6:40 ` Greg Kroah-Hartman
2020-09-19 8:20 ` Leon Romanovsky
2020-09-19 8:30 ` Greg Kroah-Hartman
2020-09-19 8:58 ` Leon Romanovsky
2020-09-19 16:43 ` Oded Gabbay
2020-09-19 17:27 ` Greg Kroah-Hartman
2020-09-19 19:22 ` Jason Gunthorpe
2020-09-20 8:47 ` Greg Kroah-Hartman
2020-09-20 19:05 ` Oded Gabbay
2020-09-21 10:39 ` Leon Romanovsky
2020-09-21 11:52 ` Jason Gunthorpe
2020-09-21 21:20 ` Jakub Kicinski
2020-09-22 11:49 ` Jason Gunthorpe
2020-09-19 18:49 ` Andrew Lunn
2020-09-18 11:56 ` Jason Gunthorpe
2020-09-18 11:59 ` Oded Gabbay
2020-09-18 12:16 ` Jason Gunthorpe
2020-09-18 12:34 ` Oded Gabbay
2020-09-18 12:50 ` Jason Gunthorpe
2020-09-18 13:02 ` Oded Gabbay
2020-09-18 13:26 ` Jason Gunthorpe
2020-09-18 13:49 ` Oded Gabbay
2020-09-18 13:59 ` Jason Gunthorpe
2020-09-18 14:12 ` Oded Gabbay
2020-09-18 14:19 ` Jason Gunthorpe [this message]
2020-09-18 14:45 ` Oded Gabbay
2020-09-18 15:07 ` Jason Gunthorpe
2020-09-18 15:15 ` Oded Gabbay
2020-09-18 15:28 ` Jason Gunthorpe
2020-09-21 11:22 ` Gal Pressman
2020-09-21 11:49 ` Leon Romanovsky
2020-09-22 11:41 ` Jason Gunthorpe
2020-09-22 12:46 ` Gal Pressman
2020-09-22 16:14 ` Jason Gunthorpe
2020-09-22 16:30 ` Gal Pressman
2020-09-22 16:52 ` Jason Gunthorpe
2020-09-18 12:10 ` Oded Gabbay
2020-09-15 20:42 ` David Miller
2020-09-15 20:49 ` Oded Gabbay
2020-09-16 6:26 ` Greg Kroah-Hartman
2020-09-16 6:36 ` Oded Gabbay
2020-09-16 7:42 ` Greg Kroah-Hartman
2020-09-16 8:02 ` Oded Gabbay
2020-09-16 8:22 ` Greg Kroah-Hartman
2020-09-16 8:47 ` Oded Gabbay
2020-09-16 12:00 ` Greg Kroah-Hartman
2020-09-20 16:45 ` Daniel Vetter
2020-09-16 23:04 ` Williams, Dan J
2020-09-18 12:00 ` Jason Gunthorpe
2020-09-18 12:01 ` Oded Gabbay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200918141909.GU8409@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=SW_Drivers@habana.ai \
--cc=andrew@lunn.ch \
--cc=davem@davemloft.net \
--cc=f.fainelli@gmail.com \
--cc=galpress@amazon.com \
--cc=gregkh@linuxfoundation.org \
--cc=izur@habana.ai \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=oded.gabbay@gmail.com \
--cc=olof@lixom.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).