Re: [PATCH 0/2] Lima DRM driver

[Daniel Vetter <daniel@ffwll.ch>]

On Thu, Feb 07, 2019 at 11:21:52PM +0800, Qiang Yu wrote:
> On Thu, Feb 7, 2019 at 5:09 PM Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > On Wed, Feb 06, 2019 at 09:14:55PM +0800, Qiang Yu wrote:
> > > Kernel DRM driver for ARM Mali 400/450 GPUs.
> > >
> > > Since last RFC, all feedback has been addressed. Most Mali DTS
> > > changes are already upstreamed by SoC maintainers. The kernel
> > > driver and user-kernel interface are quite stable for several
> > > months, so I think it's ready to be upstreamed.
> > >
> > > This implementation mainly take amdgpu DRM driver as reference.
> > >
> > > - Mali 4xx GPUs have two kinds of processors GP and PP. GP is for
> > >   OpenGL vertex shader processing and PP is for fragment shader
> > >   processing. Each processor has its own MMU so prcessors work in
> > >   virtual address space.
> > > - There's only one GP but multiple PP (max 4 for mali 400 and 8
> > >   for mali 450) in the same mali 4xx GPU. All PPs are grouped
> > >   togather to handle a single fragment shader task divided by
> > >   FB output tiled pixels. Mali 400 user space driver is
> > >   responsible for assign target tiled pixels to each PP, but mali
> > >   450 has a HW module called DLBU to dynamically balance each
> > >   PP's load.
> > > - User space driver allocate buffer object and map into GPU
> > >   virtual address space, upload command stream and draw data with
> > >   CPU mmap of the buffer object, then submit task to GP/PP with
> > >   a register frame indicating where is the command stream and misc
> > >   settings.
> > > - There's no command stream validation/relocation due to each user
> > >   process has its own GPU virtual address space. GP/PP's MMU switch
> > >   virtual address space before running two tasks from different
> > >   user process. Error or evil user space code just get MMU fault
> > >   or GP/PP error IRQ, then the HW/SW will be recovered.
> > > - Use TTM as MM. TTM_PL_TT type memory is used as the content of
> > >   lima buffer object which is allocated from TTM page pool. all
> > >   lima buffer object gets pinned with TTM_PL_FLAG_NO_EVICT when
> > >   allocation, so there's no buffer eviction and swap for now.
> >
> > All other render gpu drivers that have unified memory (aka is on the SoC)
> > use GEM directly, with some of the helpers we have. So msm, etnaviv, vc4
> > (and i915 is kinda the same too really). TTM makes sense if you have some
> > discrete memory to manage, but imo not in any other place really.
> >
> > What's the design choice behind this?
> To be honest, it's just because TTM offers more helpers. I did implement
> a GEM way with cma alloc at the beginning. But when implement paged mem,
> I found TTM has mem pool alloc, sync and mmap related helpers which covers
> much of my existing code. It's totally possible with GEM, but not as easy as
> TTM to me. And virtio-gpu seems an example to use TTM without discrete
> mem. Shouldn't TTM a super set of both unified mem and discrete mem?

virtio does have fake vram and migration afaiui. And sure, you can use TTM
without the vram migration, it's just that most of the complexity of TTM
is due to buffer placement and migration and all that stuff. If you never
need to move buffers, then you don't need that ever.

Wrt lack of helpers, what exactly are you looking for? A big part of these
for TTM is that TTM is a bid a midlayer, so reinvents a bunch of things
provided by e.g. dma-api. It's cleaner to use the dma-api directly. Basing
the lima kernel driver on vc4, freedreno or etnaviv (last one is probably
closest, since it doesn't have a display block either) would be better I
think.

> > From an upstream pov having all soc gpu drivers use similar approaches
> > should help with shared infrastructure and stuff like that.
> Hope GEM gets more helpers now if Iima have to use it.
> 
> >
> > Another one: What's the plan with extending this to panfrost? Or are the
> > architectures for command submission totally different, and we'll need
> > separate kernel drivers for utgard/midgard/bifrost?
> + Alyssa & Rob
> There is a gitlab issue about sharing kernel driver:
> https://gitlab.freedesktop.org/panfrost/linux/issues/1
> 
> But seems utgard (discrete shader arch) really differs with midgard/bifrost
> (unified shader arch), so that ARM also have different official kernel drivers.

Ok, makes sense, just wanted to check.
-Daniel

> 
> Thanks,
> Qiang
> 
> 
> 
> >
> > Thanks, Daniel
> >
> > > - Use drm_sched for GPU task schedule. Each OpenGL context should
> > >   have a lima context object in the kernel to distinguish tasks
> > >   from different user. drm_sched gets task from each lima context
> > >   in a fair way.
> > >
> > > This patch serial is based on 5.0-rc5 and squash all the commits.
> > > For whole history of this driver's development, see:
> > > https://gitlab.freedesktop.org/lima/linux/commits/lima-5.0-rc5
> > > https://gitlab.freedesktop.org/lima/linux/commits/lima-4.17-rc4
> > >
> > > Mesa driver is still in development and not ready for daily usage,
> > > but can run some simple tests like kmscube and glamrk2, and some
> > > single full screen application like kodi-gbm, see:
> > > https://gitlab.freedesktop.org/lima/mesa
> > >
> > > [rfc]
> > > https://lists.freedesktop.org/archives/dri-devel/2018-May/177314.html
> > >
> > > Lima Project Developers (1):
> > >   drm/lima: driver for ARM Mali4xx GPUs
> > >
> > > Qiang Yu (1):
> > >   drm/fourcc: add ARM tiled format modifier
> > >
> > >  drivers/gpu/drm/Kconfig               |   2 +
> > >  drivers/gpu/drm/Makefile              |   1 +
> > >  drivers/gpu/drm/lima/Kconfig          |  10 +
> > >  drivers/gpu/drm/lima/Makefile         |  22 ++
> > >  drivers/gpu/drm/lima/lima_bcast.c     |  46 +++
> > >  drivers/gpu/drm/lima/lima_bcast.h     |  14 +
> > >  drivers/gpu/drm/lima/lima_ctx.c       | 124 +++++++
> > >  drivers/gpu/drm/lima/lima_ctx.h       |  33 ++
> > >  drivers/gpu/drm/lima/lima_device.c    | 384 ++++++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_device.h    | 116 ++++++
> > >  drivers/gpu/drm/lima/lima_dlbu.c      |  56 +++
> > >  drivers/gpu/drm/lima/lima_dlbu.h      |  18 +
> > >  drivers/gpu/drm/lima/lima_drv.c       | 459 ++++++++++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_drv.h       |  59 ++++
> > >  drivers/gpu/drm/lima/lima_gem.c       | 485 +++++++++++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_gem.h       |  25 ++
> > >  drivers/gpu/drm/lima/lima_gem_prime.c | 144 ++++++++
> > >  drivers/gpu/drm/lima/lima_gem_prime.h |  18 +
> > >  drivers/gpu/drm/lima/lima_gp.c        | 280 +++++++++++++++
> > >  drivers/gpu/drm/lima/lima_gp.h        |  16 +
> > >  drivers/gpu/drm/lima/lima_l2_cache.c  |  79 +++++
> > >  drivers/gpu/drm/lima/lima_l2_cache.h  |  14 +
> > >  drivers/gpu/drm/lima/lima_mmu.c       | 135 +++++++
> > >  drivers/gpu/drm/lima/lima_mmu.h       |  16 +
> > >  drivers/gpu/drm/lima/lima_object.c    | 103 ++++++
> > >  drivers/gpu/drm/lima/lima_object.h    |  72 ++++
> > >  drivers/gpu/drm/lima/lima_pmu.c       |  61 ++++
> > >  drivers/gpu/drm/lima/lima_pmu.h       |  12 +
> > >  drivers/gpu/drm/lima/lima_pp.c        | 419 ++++++++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_pp.h        |  19 +
> > >  drivers/gpu/drm/lima/lima_regs.h      | 298 ++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_sched.c     | 486 ++++++++++++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_sched.h     | 108 ++++++
> > >  drivers/gpu/drm/lima/lima_ttm.c       | 319 +++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_ttm.h       |  24 ++
> > >  drivers/gpu/drm/lima/lima_vm.c        | 354 +++++++++++++++++++
> > >  drivers/gpu/drm/lima/lima_vm.h        |  59 ++++
> > >  include/uapi/drm/drm_fourcc.h         |   9 +
> > >  include/uapi/drm/lima_drm.h           | 193 ++++++++++
> > >  39 files changed, 5092 insertions(+)
> > >  create mode 100644 drivers/gpu/drm/lima/Kconfig
> > >  create mode 100644 drivers/gpu/drm/lima/Makefile
> > >  create mode 100644 drivers/gpu/drm/lima/lima_bcast.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_bcast.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_ctx.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_ctx.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_device.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_device.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_dlbu.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_dlbu.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_drv.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_drv.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_gem.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_gem.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_gem_prime.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_gp.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_gp.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_l2_cache.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_mmu.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_mmu.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_object.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_object.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_pmu.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_pmu.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_pp.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_pp.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_regs.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_sched.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_sched.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_ttm.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_ttm.h
> > >  create mode 100644 drivers/gpu/drm/lima/lima_vm.c
> > >  create mode 100644 drivers/gpu/drm/lima/lima_vm.h
> > >  create mode 100644 include/uapi/drm/lima_drm.h
> > >
> > > --
> > > 2.17.1
> > >
> >
> > --
> > Daniel Vetter
> > Software Engineer, Intel Corporation
> > http://blog.ffwll.ch

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel