Re: [Intel-xe] [PATCH v2] drm/xe: Use fast virtual copy engine for migrate engine on PVC

[Matthew Brost <matthew.brost@intel.com>]

On Fri, Mar 24, 2023 at 07:42:33AM +0100, Mauro Carvalho Chehab wrote:
> On Thu, 23 Mar 2023 18:23:29 -0700
> Matthew Brost <matthew.brost@intel.com> wrote:
> 
> > Some copy hardware engine instances are faster than others on PVC, use a
> > virtual engine of these plus the reserved instance for the migrate
> > engine on PVC. The idea being if a fast instance is available it will be
> > used and the throughput of kernel copies, clears, and pagefault
> > servicing will be higher.
> > 
> > v2: Include local change of correct mask for fast instances
> > 
> > Cc: Bruce Chang <yu.bruce.chang@intel.com>
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_engine.h    |  2 ++
> >  drivers/gpu/drm/xe/xe_hw_engine.c | 20 ++++++++++++++++++++
> >  drivers/gpu/drm/xe/xe_migrate.c   |  7 ++++---
> >  3 files changed, 26 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_engine.h b/drivers/gpu/drm/xe/xe_engine.h
> > index 1cf7f23c4afd..0a9c35ea3d34 100644
> > --- a/drivers/gpu/drm/xe/xe_engine.h
> > +++ b/drivers/gpu/drm/xe/xe_engine.h
> > @@ -26,6 +26,8 @@ void xe_engine_destroy(struct kref *ref);
> >  
> >  struct xe_engine *xe_engine_lookup(struct xe_file *xef, u32 id);
> >  
> > +u32 xe_hw_engine_fast_copy_logical_mask(struct xe_gt *gt);
> > +
> >  static inline struct xe_engine *xe_engine_get(struct xe_engine *engine)
> >  {
> >  	kref_get(&engine->refcount);
> > diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
> > index 63a4efd5edcc..d2b43b189b14 100644
> > --- a/drivers/gpu/drm/xe/xe_hw_engine.c
> > +++ b/drivers/gpu/drm/xe/xe_hw_engine.c
> > @@ -600,3 +600,23 @@ bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe)
> >  	return xe->info.supports_usm && hwe->class == XE_ENGINE_CLASS_COPY &&
> >  		hwe->instance == gt->usm.reserved_bcs_instance;
> >  }
> > +
> > +u32 xe_hw_engine_fast_copy_logical_mask(struct xe_gt *gt)
> > +{
> > +	struct xe_device *xe = gt_to_xe(gt);
> > +	struct xe_hw_engine *hwe;
> > +	const u32 fast_physical_mask = 0xab;	/* 0, 1, 3, 5, 7 */
> 
> Since this is PVC-only, I would use, instead:
> 
> 	const u32 pvc_fast_physical_mask = BIT(0) | BIT(1) | BIT(3) | BIT(5) | BIT(7);
> 
> as:
>  - it would avoid the comment;
>  - it will document that other GPU models may need different physical masks.
> 

Agree that is better, will change.

> 
> > +	u32 fast_logical_mask = 0;
> > +	enum xe_hw_engine_id id;
> > +
> > +	/* XXX: We only support this function on PVC for now */
> > +	XE_BUG_ON(!(xe->info.platform == XE_PVC));
> 
> Why bug on? Since this future is PVC only (for now), other platforms
> may not need. So, I would do, instead (as before):
> 
> 	if (!(xe->info.platform == XE_PVC))
> 		return (hwe->logical_instance);
> 
> perhaps calling a drm_dbg() to report it, just in case newer platforms
> might require it as well.
>

I added the BUG_ON so if we add new platforms that support USM this
function blows and we immediately fix it. I prefer to keep it as is.

Matt

> > +
> > +	for_each_hw_engine(hwe, gt, id) {
> > +		if ((fast_physical_mask | gt->usm.reserved_bcs_instance) &
> > +		    BIT(hwe->instance))
> > +			fast_logical_mask |= hwe->logical_instance;
> > +	}
> > +
> > +	return fast_logical_mask;
> > +}
> > diff --git a/drivers/gpu/drm/xe/xe_migrate.c b/drivers/gpu/drm/xe/xe_migrate.c
> > index 11c8af9c6c92..4a7fec5d619d 100644
> > --- a/drivers/gpu/drm/xe/xe_migrate.c
> > +++ b/drivers/gpu/drm/xe/xe_migrate.c
> > @@ -345,11 +345,12 @@ struct xe_migrate *xe_migrate_init(struct xe_gt *gt)
> >  							   XE_ENGINE_CLASS_COPY,
> >  							   gt->usm.reserved_bcs_instance,
> >  							   false);
> > -		if (!hwe)
> > +		u32 logical_mask = xe_hw_engine_fast_copy_logical_mask(gt);
> > +
> > +		if (!hwe || !logical_mask)
> >  			return ERR_PTR(-EINVAL);
> >  
> > -		m->eng = xe_engine_create(xe, vm,
> > -					  BIT(hwe->logical_instance), 1,
> > +		m->eng = xe_engine_create(xe, vm, logical_mask, 1,
> >  					  hwe, ENGINE_FLAG_KERNEL);
> >  	} else {
> >  		m->eng = xe_engine_create_class(xe, gt, vm,