Re: [PATCH v3 3/6] drm/i915 Implement LMEM backup and restore for suspend / resume

["Thomas Hellström" <thomas.hellstrom@linux.intel.com>]

On Mon, 2021-09-20 at 11:49 +0100, Matthew Auld wrote:
> On 14/09/2021 20:31, Thomas Hellström wrote:
> > Just evict unpinned objects to system. For pinned LMEM objects,
> > make a backup system object and blit the contents to that.
> > 
> > Backup is performed in three steps,
> > 1: Opportunistically evict evictable objects using the gpu blitter.
> > 2: After gt idle, evict evictable objects using the gpu blitter.
> > This will
> > be modified in an upcoming patch to backup pinned objects that are
> > not used
> > by the blitter itself.
> > 3: Backup remaining pinned objects using memcpy.
> > 
> > Also move uC suspend to after 2) to make sure we have a functional
> > GuC
> > during 2) if using GuC submission.
> > 
> > v2:
> > - Major refactor to make sure gem_exec_suspend@hang-SX subtests
> > work, and
> >    suspend / resume works with a slightly modified GuC submission
> > enabling
> >    patch series.
> > 
> > v3:
> > - Fix a potential use-after-free (Matthew Auld)
> > - Use i915_gem_object_create_shmem() instead of
> >    i915_gem_object_create_region (Matthew Auld)
> > - Minor simplifications (Matthew Auld)
> > - Fix up kerneldoc for i195_ttm_restore_region().
> > - Final lmem_suspend() call moved to i915_gem_backup_suspend from
> >    i915_gem_suspend_late, since the latter gets called at driver
> > unload
> >    and we don't unnecessarily want to run it at that time.
> > 
> > Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> 
> <snip>
> 
> > +
> > +static int i915_ttm_restore(struct i915_gem_apply_to_region
> > *apply,
> > +                           struct drm_i915_gem_object *obj)
> > +{
> > +       struct i915_gem_ttm_pm_apply *pm_apply =
> > +               container_of(apply, typeof(*pm_apply), base);
> > +       struct drm_i915_gem_object *backup = obj->ttm.backup;
> > +       struct ttm_buffer_object *backup_bo =
> > i915_gem_to_ttm(backup);
> > +       struct ttm_operation_ctx ctx = {};
> > +       int err;
> > +
> > +       if (!backup)
> > +               return 0;
> > +
> > +       if (!pm_apply->allow_gpu && (obj->flags &
> > I915_BO_ALLOC_USER))
> > +               return 0;
> 
> Hmm, do we ever hit this? I would presume anything that userspace 
> directly allocated in lmem can be kicked out with
> ttm_bo_validate(sys) 
> i.e backup == NULL?

At this point, (before patch 6/6) I think we might do. Typical
candidates are dma-buf objects that have become pinned on exporting,
and perhaps framebuffers that are pinned, not sure they are unpinned
before we back them up.

But it might be that we should remove this after patch 6/6 where we
require a special flag for early recovers using memcpy.

/Thomas


> 
> > +
> > +       err = i915_gem_object_lock(backup, apply->ww);
> > +       if (err)
> > +               return err;
> > +
> > +       /* Content may have been swapped. */
> > +       err = ttm_tt_populate(backup_bo->bdev, backup_bo->ttm,
> > &ctx);
> > +       if (!err) {
> > +               err = i915_gem_obj_copy_ttm(obj, backup, pm_apply-
> > >allow_gpu,
> > +                                           false);
> > +               GEM_WARN_ON(err);
> > +
> > +               obj->ttm.backup = NULL;
> > +               err = 0;
> > +       }
> > +
> > +       i915_gem_ww_unlock_single(backup);
> > +
> > +       if (!err)
> > +               i915_gem_object_put(backup);
> > +
> > +       return err;
> > +}
> > +