From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C3A85C54E5D for ; Fri, 8 Mar 2024 05:08:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 71F0810F7AB; Fri, 8 Mar 2024 05:08:08 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="jwY9/CO9"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2F5ED10FCF0 for ; Fri, 8 Mar 2024 05:07:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709874473; x=1741410473; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XASWRAMWTkXL5l2w+NZN8bq24E2Zwy/qDgzQH1yi40s=; b=jwY9/CO90AE+NQA/4HMFu48UYnBIQeaPpYU7Rq5hxUcqzFdo3QEv0ouY hID1IuYQrj2F8CK7DbxaH9oqauXr1BFbW/hOj2XkIfVo9uwmcsBP0D8QN aOz+xlXaEdQPv9dI5NsvucID9Ydd4P6xCsR9DaXhhAn6AVWw51nwnuZSh f3pLSw3h8a9zfUtxRhgWJz51ZDUe76vV5XjQ12cevgcEdImEdU8ZCkN+c srFQOwhLoNvftFxYaHtv8yNoj1O/c0pqij4yLqBcgeoX6Q+TRXUHd4QS/ zENb7C1O2M3ATcSJRvs+Aj006+sxa4sdlbYpuFEZfvd2TsuQK3wRqV0kq Q==; X-IronPort-AV: E=McAfee;i="6600,9927,11006"; a="4761959" X-IronPort-AV: E=Sophos;i="6.07,108,1708416000"; d="scan'208";a="4761959" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Mar 2024 21:07:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,108,1708416000"; d="scan'208";a="10402980" Received: from lstrano-desk.jf.intel.com ([10.54.39.91]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Mar 2024 21:07:51 -0800 From: Matthew Brost To: Cc: Matthew Brost Subject: [PATCH v4 05/30] drm/xe: Update xe_vm_rebind to use dummy VMA operations Date: Thu, 7 Mar 2024 21:07:41 -0800 Message-Id: <20240308050806.577176-6-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240308050806.577176-1-matthew.brost@intel.com> References: <20240308050806.577176-1-matthew.brost@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" All bind interfaces are transitioning to use VMA ops, update xe_vm_rebind to use VMA ops. Signed-off-by: Matthew Brost --- drivers/gpu/drm/xe/xe_vm.c | 47 +++++--- drivers/gpu/drm/xe/xe_vm_types.h | 189 ++++++++++++++++--------------- 2 files changed, 132 insertions(+), 104 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index e342af6b51b1..0bb807c05d7b 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -755,10 +755,22 @@ int xe_vm_userptr_check_repin(struct xe_vm *vm) list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN; } -static struct dma_fence * -xe_vm_bind_vma(struct xe_vma *vma, struct xe_exec_queue *q, - struct xe_sync_entry *syncs, u32 num_syncs, - bool first_op, bool last_op); +static void xe_vm_populate_dummy_rebind(struct xe_vm *vm, struct xe_vma *vma) +{ + vm->dummy_ops.op.base.op = DRM_GPUVA_OP_MAP; + vm->dummy_ops.op.base.map.va.addr = vma->gpuva.va.addr; + vm->dummy_ops.op.base.map.va.range = vma->gpuva.va.range; + vm->dummy_ops.op.base.map.gem.obj = vma->gpuva.gem.obj; + vm->dummy_ops.op.base.map.gem.offset = vma->gpuva.gem.offset; + vm->dummy_ops.op.map.vma = vma; + vm->dummy_ops.op.map.immediate = true; + vm->dummy_ops.op.map.dumpable = vma->gpuva.flags & XE_VMA_DUMPABLE; + vm->dummy_ops.op.map.is_null = xe_vma_is_null(vma); +} + +static struct dma_fence *ops_execute(struct xe_vm *vm, + struct xe_vma_ops *vops, + bool cleanup); struct dma_fence *xe_vm_rebind(struct xe_vm *vm, bool rebind_worker) { @@ -780,7 +792,9 @@ struct dma_fence *xe_vm_rebind(struct xe_vm *vm, bool rebind_worker) trace_xe_vma_rebind_worker(vma); else trace_xe_vma_rebind_exec(vma); - fence = xe_vm_bind_vma(vma, NULL, NULL, 0, false, false); + + xe_vm_populate_dummy_rebind(vm, vma); + fence = ops_execute(vm, &vm->dummy_ops.vops, false); if (IS_ERR(fence)) return fence; } @@ -1289,6 +1303,11 @@ static void xe_vm_free_scratch(struct xe_vm *vm) } } +static void xe_vma_ops_init(struct xe_vma_ops *vops) +{ + INIT_LIST_HEAD(&vops->list); +} + struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags) { struct drm_gem_object *vm_resv_obj; @@ -1310,6 +1329,10 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags) init_rwsem(&vm->lock); mutex_init(&vm->snap_mutex); + xe_vma_ops_init(&vm->dummy_ops.vops); + INIT_LIST_HEAD(&vm->dummy_ops.op.link); + list_add(&vm->dummy_ops.op.link, &vm->dummy_ops.vops.list); + INIT_LIST_HEAD(&vm->rebind_list); INIT_LIST_HEAD(&vm->userptr.repin_list); @@ -2140,6 +2163,7 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo, struct xe_vma_op *op = gpuva_op_to_vma_op(__op); if (__op->op == DRM_GPUVA_OP_MAP) { + op->map.immediate = !xe_vm_in_fault_mode(vm); op->map.is_null = flags & DRM_XE_VM_BIND_FLAG_NULL; op->map.dumpable = flags & DRM_XE_VM_BIND_FLAG_DUMPABLE; op->map.pat_index = pat_index; @@ -2465,7 +2489,7 @@ static struct dma_fence *op_execute(struct xe_vm *vm, struct xe_vma *vma, { struct dma_fence *fence = NULL; - lockdep_assert_held_write(&vm->lock); + lockdep_assert_held(&vm->lock); xe_vm_assert_held(vm); xe_bo_assert_held(xe_vma_bo(vma)); @@ -2473,7 +2497,7 @@ static struct dma_fence *op_execute(struct xe_vm *vm, struct xe_vma *vma, case DRM_GPUVA_OP_MAP: fence = xe_vm_bind(vm, vma, op->q, xe_vma_bo(vma), op->syncs, op->num_syncs, - !xe_vm_in_fault_mode(vm), + op->map.immediate, op->flags & XE_VMA_OP_FIRST, op->flags & XE_VMA_OP_LAST); break; @@ -2554,7 +2578,7 @@ __xe_vma_op_execute(struct xe_vm *vm, struct xe_vma *vma, retry_userptr: fence = op_execute(vm, vma, op); if (IS_ERR(fence) && PTR_ERR(fence) == -EAGAIN) { - lockdep_assert_held_write(&vm->lock); + lockdep_assert_held(&vm->lock); if (op->base.op == DRM_GPUVA_OP_REMAP) { if (!op->remap.unmap_done) @@ -2583,7 +2607,7 @@ xe_vma_op_execute(struct xe_vm *vm, struct xe_vma_op *op) { struct dma_fence *fence = ERR_PTR(-ENOMEM); - lockdep_assert_held_write(&vm->lock); + lockdep_assert_held(&vm->lock); switch (op->base.op) { case DRM_GPUVA_OP_MAP: @@ -2992,11 +3016,6 @@ static int vm_bind_ioctl_signal_fences(struct xe_vm *vm, return err; } -static void xe_vma_ops_init(struct xe_vma_ops *vops) -{ - INIT_LIST_HEAD(&vops->list); -} - int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) { struct xe_device *xe = to_xe_device(dev); diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h index cc3dce893f1e..7ef9e632154a 100644 --- a/drivers/gpu/drm/xe/xe_vm_types.h +++ b/drivers/gpu/drm/xe/xe_vm_types.h @@ -18,6 +18,7 @@ #include "xe_range_fence.h" struct xe_bo; +struct xe_device; struct xe_sync_entry; struct xe_user_fence; struct xe_vm; @@ -124,7 +125,96 @@ struct xe_userptr_vma { struct xe_userptr userptr; }; -struct xe_device; +/** struct xe_vma_op_map - VMA map operation */ +struct xe_vma_op_map { + /** @vma: VMA to map */ + struct xe_vma *vma; + /** @immediate: Immediate bind */ + bool immediate; + /** @is_null: is NULL binding */ + bool is_null; + /** @dumpable: whether BO is dumped on GPU hang */ + bool dumpable; + /** @pat_index: The pat index to use for this operation. */ + u16 pat_index; +}; + +/** struct xe_vma_op_remap - VMA remap operation */ +struct xe_vma_op_remap { + /** @prev: VMA preceding part of a split mapping */ + struct xe_vma *prev; + /** @next: VMA subsequent part of a split mapping */ + struct xe_vma *next; + /** @start: start of the VMA unmap */ + u64 start; + /** @range: range of the VMA unmap */ + u64 range; + /** @skip_prev: skip prev rebind */ + bool skip_prev; + /** @skip_next: skip next rebind */ + bool skip_next; + /** @unmap_done: unmap operation in done */ + bool unmap_done; +}; + +/** struct xe_vma_op_prefetch - VMA prefetch operation */ +struct xe_vma_op_prefetch { + /** @region: memory region to prefetch to */ + u32 region; +}; + +/** enum xe_vma_op_flags - flags for VMA operation */ +enum xe_vma_op_flags { + /** @XE_VMA_OP_FIRST: first VMA operation for a set of syncs */ + XE_VMA_OP_FIRST = BIT(0), + /** @XE_VMA_OP_LAST: last VMA operation for a set of syncs */ + XE_VMA_OP_LAST = BIT(1), + /** @XE_VMA_OP_COMMITTED: VMA operation committed */ + XE_VMA_OP_COMMITTED = BIT(2), + /** @XE_VMA_OP_PREV_COMMITTED: Previous VMA operation committed */ + XE_VMA_OP_PREV_COMMITTED = BIT(3), + /** @XE_VMA_OP_NEXT_COMMITTED: Next VMA operation committed */ + XE_VMA_OP_NEXT_COMMITTED = BIT(4), +}; + +/** struct xe_vma_op - VMA operation */ +struct xe_vma_op { + /** @base: GPUVA base operation */ + struct drm_gpuva_op base; + /** + * @ops: GPUVA ops, when set call drm_gpuva_ops_free after this + * operations is processed + */ + struct drm_gpuva_ops *ops; + /** @q: exec queue for this operation */ + struct xe_exec_queue *q; + /** + * @syncs: syncs for this operation, only used on first and last + * operation + */ + struct xe_sync_entry *syncs; + /** @num_syncs: number of syncs */ + u32 num_syncs; + /** @link: async operation link */ + struct list_head link; + /** @flags: operation flags */ + enum xe_vma_op_flags flags; + + union { + /** @map: VMA map operation specific data */ + struct xe_vma_op_map map; + /** @remap: VMA remap operation specific data */ + struct xe_vma_op_remap remap; + /** @prefetch: VMA prefetch operation specific data */ + struct xe_vma_op_prefetch prefetch; + }; +}; + +/** struct xe_vma_ops - VMA operations */ +struct xe_vma_ops { + /** @list: list of VMA operations */ + struct list_head list; +}; struct xe_vm { /** @gpuvm: base GPUVM used to track VMAs */ @@ -267,99 +357,18 @@ struct xe_vm { bool capture_once; } error_capture; + /** @dummy_ops: dummy VMA ops to issue rebinds */ + struct { + /** @dummy_ops.ops: dummy VMA ops */ + struct xe_vma_ops vops; + /** @dummy_ops.op: dummy VMA op */ + struct xe_vma_op op; + } dummy_ops; + /** @batch_invalidate_tlb: Always invalidate TLB before batch start */ bool batch_invalidate_tlb; /** @xef: XE file handle for tracking this VM's drm client */ struct xe_file *xef; }; -/** struct xe_vma_op_map - VMA map operation */ -struct xe_vma_op_map { - /** @vma: VMA to map */ - struct xe_vma *vma; - /** @is_null: is NULL binding */ - bool is_null; - /** @dumpable: whether BO is dumped on GPU hang */ - bool dumpable; - /** @pat_index: The pat index to use for this operation. */ - u16 pat_index; -}; - -/** struct xe_vma_op_remap - VMA remap operation */ -struct xe_vma_op_remap { - /** @prev: VMA preceding part of a split mapping */ - struct xe_vma *prev; - /** @next: VMA subsequent part of a split mapping */ - struct xe_vma *next; - /** @start: start of the VMA unmap */ - u64 start; - /** @range: range of the VMA unmap */ - u64 range; - /** @skip_prev: skip prev rebind */ - bool skip_prev; - /** @skip_next: skip next rebind */ - bool skip_next; - /** @unmap_done: unmap operation in done */ - bool unmap_done; -}; - -/** struct xe_vma_op_prefetch - VMA prefetch operation */ -struct xe_vma_op_prefetch { - /** @region: memory region to prefetch to */ - u32 region; -}; - -/** enum xe_vma_op_flags - flags for VMA operation */ -enum xe_vma_op_flags { - /** @XE_VMA_OP_FIRST: first VMA operation for a set of syncs */ - XE_VMA_OP_FIRST = BIT(0), - /** @XE_VMA_OP_LAST: last VMA operation for a set of syncs */ - XE_VMA_OP_LAST = BIT(1), - /** @XE_VMA_OP_COMMITTED: VMA operation committed */ - XE_VMA_OP_COMMITTED = BIT(2), - /** @XE_VMA_OP_PREV_COMMITTED: Previous VMA operation committed */ - XE_VMA_OP_PREV_COMMITTED = BIT(3), - /** @XE_VMA_OP_NEXT_COMMITTED: Next VMA operation committed */ - XE_VMA_OP_NEXT_COMMITTED = BIT(4), -}; - -/** struct xe_vma_op - VMA operation */ -struct xe_vma_op { - /** @base: GPUVA base operation */ - struct drm_gpuva_op base; - /** - * @ops: GPUVA ops, when set call drm_gpuva_ops_free after this - * operations is processed - */ - struct drm_gpuva_ops *ops; - /** @q: exec queue for this operation */ - struct xe_exec_queue *q; - /** - * @syncs: syncs for this operation, only used on first and last - * operation - */ - struct xe_sync_entry *syncs; - /** @num_syncs: number of syncs */ - u32 num_syncs; - /** @link: async operation link */ - struct list_head link; - /** @flags: operation flags */ - enum xe_vma_op_flags flags; - - union { - /** @map: VMA map operation specific data */ - struct xe_vma_op_map map; - /** @remap: VMA remap operation specific data */ - struct xe_vma_op_remap remap; - /** @prefetch: VMA prefetch operation specific data */ - struct xe_vma_op_prefetch prefetch; - }; -}; - -/** struct xe_vma_ops - VMA operations */ -struct xe_vma_ops { - /** @list: list of VMA operations */ - struct list_head list; -}; - #endif -- 2.34.1