[Intel-gfx] [RFC 0/2] drm/doc/rfc: i915 VM_BIND feature design + uapi

[Intel-gfx] [RFC 0/2] drm/doc/rfc: i915 VM_BIND feature design + uapi

[Niranjana Vishwanathapura]

This is the i915 driver VM_BIND feature design RFC patch series along
with the required uapi definition and description of intended use cases.

Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>

Niranjana Vishwanathapura (2):
  drm/doc/rfc: VM_BIND feature design document
  drm/doc/rfc: VM_BIND uapi definition

 Documentation/gpu/rfc/i915_vm_bind.h   | 113 +++++++++++++++++++++
 Documentation/gpu/rfc/i915_vm_bind.rst | 132 +++++++++++++++++++++++++
 Documentation/gpu/rfc/index.rst        |   4 +
 3 files changed, 249 insertions(+)
 create mode 100644 Documentation/gpu/rfc/i915_vm_bind.h
 create mode 100644 Documentation/gpu/rfc/i915_vm_bind.rst

-- 
2.21.0.rc0.32.g243a4c7e27

[Intel-gfx] [RFC 1/2] drm/doc/rfc: VM_BIND feature design document

[Niranjana Vishwanathapura]

VM_BIND design document with description of intended use cases.

Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
---
 Documentation/gpu/rfc/i915_vm_bind.rst | 126 +++++++++++++++++++++++++
 Documentation/gpu/rfc/index.rst        |   4 +
 2 files changed, 130 insertions(+)
 create mode 100644 Documentation/gpu/rfc/i915_vm_bind.rst

diff --git a/Documentation/gpu/rfc/i915_vm_bind.rst b/Documentation/gpu/rfc/i915_vm_bind.rst
new file mode 100644
index 000000000000..dbc35262a554
--- /dev/null
+++ b/Documentation/gpu/rfc/i915_vm_bind.rst
@@ -0,0 +1,126 @@
+==========================================
+I915 VM_BIND feature design and use cases
+==========================================
+
+VM_BIND feature
+================
+DRM_I915_GEM_VM_BIND/UNBIND ioctls allows UMD to bind/unbind GEM buffer
+objects (BOs) or sections of a BOs at specified GPU virtual addresses on
+a specified address space (VM).
+
+These mappings (also referred to as persistent mappings) will be persistent
+across multiple GPU submissions (execbuff) issued by the UMD, making execbuff
+path leaner with fast path submission latency of O(1) w.r.t the number of
+objects required for that submission.
+
+UMDs can still send BOs of these persistent mappings in execlist of execbuff
+for specifying BO dependencies (implicit fencing) and to use BO as a batch.
+
+The persistent mappings are not individually tracked, instead the address
+space (VM) they are mapped in is tracked to determine if the mappings are
+being referred by GPU job (active) or not.
+
+VM_BIND features include:
+- Different VA mappings can map to the same physical pages of an object
+  (aliasing).
+- VA mapping can map to a partial section of the BO (partial binding).
+- Support capture of mapping in the dump upon GPU error.
+- TLB is flushed upon unbind completion.
+- Asynchronous vm_bind and vm_unbind support.
+- VM_BIND uses user/memory fence mechanism (explained below) for signaling
+  bind completion.
+
+
+User/Memory Fence
+==================
+The idea is to take a user process virtual address and install an interrupt
+handler to wake up the current task when the memory location passes the user
+supplied filter.
+
+It also allows the user to emit their own MI_FLUSH/PIPE_CONTROL notify
+interrupt within their batches after updating the value on the GPU to
+have sub-batch precision on the wakeup.
+
+User/Memory fence <user address and value pair> can also be supplied to the
+kernel driver to signal/wake up the user process after completion of an
+asynchronous operation.
+
+This feature will be derived from the below original work:
+https://patchwork.freedesktop.org/patch/349417/
+
+When VM_BIND ioctl was provided with a user/memory fence via SYNC_FENCE
+extension, it will be signaled upon the completion of binding of that
+mapping. All async binds/unbinds are serialized, hence signaling of
+user/memory fence also indicate the completion of all previous binds/unbinds.
+
+
+TODOs
+======
+- Rebase VM_BIND on top of ongoing i915 TTM adoption changes including
+  eviction support.
+- Various optimizations like around LRU ordering of persistent mappings,
+  batching of TLB flushes etc.
+
+
+Intended use cases
+===================
+
+Debugger
+---------
+With debug event interface user space process (debugger) is able to keep track
+of and act upon resources created by another process (debuggee) and attached
+to GPU via vm_bind interface.
+
+Mesa/Valkun
+------------
+VM_BIND can potentially reduce the CPU-overhead in Mesa thus improving
+performance. For Vulkan it should be straightforward to use VM_BIND.
+For Iris implicit buffer tracking must be implemented before we can harness
+VM_BIND benefits. With increasing GPU hardware performance reducing CPU
+overhead becomes more important.
+
+Page level hints settings
+--------------------------
+VM_BIND allows any hints setting per mapping instead of per BO.
+Possible hints include read-only, placement and atomicity.
+Sub-BO level placement hint will be even more relevant with
+upcoming GPU on-demand page fault support.
+
+Page level Cache/CLOS settings
+-------------------------------
+VM_BIND allows cache/CLOS settings per mapping instead of per BO.
+
+Compute
+--------
+Usage of dma-fence expects that they complete in reasonable amount of time.
+Compute on the other hand can be long running. Hence it is appropriate for
+compute to use user/memory fence (explained above) and dma-fence usage will
+be limited to in kernel consumption only. Compute must opt-in for this
+mechanism during context creation time with a 'compute_ctx' flag.
+
+Where GPU page faults are not available, kernel driver upon buffer invalidation
+must initiate a compute context suspend with a dma-fence attached to it.
+And upon completion of that suspend fence, finish the invalidation and then
+resume the compute context.
+
+This is much easier to support with VM_BIND instead of the current heavier
+execbuff path resource attachment.
+
+Low Latency Submission
+-----------------------
+Allow compute UMDs to directly submit GPU jobs instead of through execbuff
+ioctl. VM_BIND allows map/unmap of BOs required for directly submitted jobs.
+
+Shared Virtual Memory (SVM) support
+------------------------------------
+VM_BIND interface can be used to map system memory directly (without gem BO
+abstraction) using the HMM interface.
+
+
+Links:
+======
+- Reference WIP VM_BIND implementation can be found here.
+  https://gitlab.freedesktop.org/nvishwa1/nvishwa1-drm-tip
+
+  NOTE: It is WIP and not fully functional. There are known issues which
+  are being worked upon.
diff --git a/Documentation/gpu/rfc/index.rst b/Documentation/gpu/rfc/index.rst
index 91e93a705230..7d10c36b268d 100644
--- a/Documentation/gpu/rfc/index.rst
+++ b/Documentation/gpu/rfc/index.rst
@@ -23,3 +23,7 @@ host such documentation:
 .. toctree::
 
     i915_scheduler.rst
+
+.. toctree::
+
+    i915_vm_bind.rst
-- 
2.21.0.rc0.32.g243a4c7e27

[Intel-gfx] [RFC 2/2] drm/doc/rfc: VM_BIND uapi definition

[Niranjana Vishwanathapura]

VM_BIND and GEM_WAIT_USER_FENCE uapi document

Signed-off-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
---
 Documentation/gpu/rfc/i915_vm_bind.h   | 113 +++++++++++++++++++++++++
 Documentation/gpu/rfc/i915_vm_bind.rst |   6 ++
 2 files changed, 119 insertions(+)
 create mode 100644 Documentation/gpu/rfc/i915_vm_bind.h

diff --git a/Documentation/gpu/rfc/i915_vm_bind.h b/Documentation/gpu/rfc/i915_vm_bind.h
new file mode 100644
index 000000000000..3aaf66e62aa0
--- /dev/null
+++ b/Documentation/gpu/rfc/i915_vm_bind.h
@@ -0,0 +1,113 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2021 Intel Corporation
+ */
+
+/* VM_BIND feature availability through drm_i915_getparam */
+#define I915_PARAM_HAS_VM_BIND          59
+
+/**
+ * struct drm_i915_gem_vm_bind - VA to object/buffer mapping to [un]bind.
+ */
+struct drm_i915_gem_vm_bind {
+	/** vm to [un]bind **/
+	__u32 vm_id;
+
+	/** BO handle or file descriptor. 'fd' to -1 for system pages (SVM) **/
+	union {
+		__u32 handle;
+		__s32 fd;
+	}
+
+	/** VA start to [un]bind **/
+	__u64 start;
+
+	/** Offset in object to [un]bind **/
+	__u64 offset;
+
+	/** VA length to [un]bind **/
+	__u64 length;
+
+	/** Flags **/
+	__u64 flags;
+	/** Bind the mapping immediately instead of during next submission */
+#define I915_GEM_VM_BIND_IMMEDIATE   (1 << 0)
+	/** Read-only mapping */
+#define I915_GEM_VM_BIND_READONLY    (1 << 1)
+	/** Capture this mapping in the dump upon GPU error */
+#define I915_GEM_VM_BIND_CAPTURE     (1 << 2)
+
+	/**
+	 * Zero-terminated chain of extensions.
+	 *
+	 * No current extensions defined; mbz.
+	 */
+	__u64 extensions;
+};
+
+/**
+ * struct drm_i915_vm_bind_ext_sync_fence - Bind completion signaling extension.
+ */
+struct drm_i915_vm_bind_ext_sync_fence {
+#define I915_VM_BIND_EXT_SYNC_FENCE     0
+	/** @base: Extension link. See struct i915_user_extension. */
+	struct i915_user_extension base;
+
+	/** User/Memory fence address */
+	__u64 addr;
+
+	/** User/Memory fence value to be written after bind completion */
+	__u64 val;
+};
+
+/**
+ * struct drm_i915_gem_wait_user_fence
+ *
+ * Wait on user/memory fence. User/Memory fence can be woken up either by,
+ *    1. GPU context indicated by 'ctx_id', or,
+ *    2. Kerrnel driver async worker upon I915_UFENCE_WAIT_SOFT.
+ *       'ctx_id' is ignored when this flag is set.
+ *
+ * Wakeup when below condition is true.
+ * (*addr & MASK) OP (VALUE & MASK)
+ *
+ */
+struct drm_i915_gem_wait_user_fence {
+	/** @base: Extension link. See struct i915_user_extension. */
+	__u64 extensions;
+
+	/** User/Memory fence address */
+	__u64 addr;
+
+	/** Id of the Context which will signal the fence. */
+	__u32 ctx_id;
+
+	/** Wakeup condition operator */
+	__u16 op;
+#define I915_UFENCE_WAIT_EQ      0
+#define I915_UFENCE_WAIT_NEQ     1
+#define I915_UFENCE_WAIT_GT      2
+#define I915_UFENCE_WAIT_GTE     3
+#define I915_UFENCE_WAIT_LT      4
+#define I915_UFENCE_WAIT_LTE     5
+#define I915_UFENCE_WAIT_BEFORE  6
+#define I915_UFENCE_WAIT_AFTER   7
+
+	/** Flags */
+	__u16 flags;
+#define I915_UFENCE_WAIT_SOFT    0x1
+#define I915_UFENCE_WAIT_ABSTIME 0x2
+
+	/** Wakeup value */
+	__u64 value;
+
+	/** Wakeup mask */
+	__u64 mask;
+#define I915_UFENCE_WAIT_U8     0xffu
+#define I915_UFENCE_WAIT_U16    0xffffu
+#define I915_UFENCE_WAIT_U32    0xfffffffful
+#define I915_UFENCE_WAIT_U64    0xffffffffffffffffull
+
+	/** Timeout */
+	__s64 timeout;
+};
diff --git a/Documentation/gpu/rfc/i915_vm_bind.rst b/Documentation/gpu/rfc/i915_vm_bind.rst
index dbc35262a554..dc843e32a1cd 100644
--- a/Documentation/gpu/rfc/i915_vm_bind.rst
+++ b/Documentation/gpu/rfc/i915_vm_bind.rst
@@ -117,6 +117,12 @@ VM_BIND interface can be used to map system memory directly (without gem BO
 abstraction) using the HMM interface.
 
 
+UAPI
+=====
+Uapi definiton can be found here:
+.. kernel-doc:: Documentation/gpu/rfc/i915_vm_bind.h
+
+
 Links:
 ======
 - Reference WIP VM_BIND implementation can be found here.
-- 
2.21.0.rc0.32.g243a4c7e27

[Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/doc/rfc: i915 VM_BIND feature design + uapi

[Patchwork]

== Series Details ==

Series: drm/doc/rfc: i915 VM_BIND feature design + uapi
URL   : https://patchwork.freedesktop.org/series/93447/
State : warning

== Summary ==

$ dim checkpatch origin/drm-tip
4bebdd1fc544 drm/doc/rfc: VM_BIND feature design document
-:11: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#11: 
new file mode 100644

-:16: WARNING:SPDX_LICENSE_TAG: Missing or malformed SPDX-License-Identifier tag in line 1
#16: FILE: Documentation/gpu/rfc/i915_vm_bind.rst:1:
+==========================================

total: 0 errors, 2 warnings, 0 checks, 133 lines checked
dd0b34c891dd drm/doc/rfc: VM_BIND uapi definition
-:11: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#11: 
new file mode 100644

total: 0 errors, 1 warnings, 0 checks, 125 lines checked

[Intel-gfx] ✗ Fi.CI.DOCS: warning for drm/doc/rfc: i915 VM_BIND feature design + uapi

[Patchwork]

== Series Details ==

Series: drm/doc/rfc: i915 VM_BIND feature design + uapi
URL   : https://patchwork.freedesktop.org/series/93447/
State : warning

== Summary ==

$ make htmldocs 2>&1 > /dev/null | grep i915
/home/cidrm/kernel/Documentation/gpu/rfc/i915_vm_bind.rst:25: WARNING: Unexpected indentation.
/home/cidrm/kernel/Documentation/gpu/rfc/i915_vm_bind.rst:26: WARNING: Block quote ends without a blank line; unexpected unindent.

[Intel-gfx] ✗ Fi.CI.BAT: failure for drm/doc/rfc: i915 VM_BIND feature design + uapi

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 2390 bytes --]

== Series Details ==

Series: drm/doc/rfc: i915 VM_BIND feature design + uapi
URL   : https://patchwork.freedesktop.org/series/93447/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_10454 -> Patchwork_20780
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with Patchwork_20780 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in Patchwork_20780, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20780/index.html

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in Patchwork_20780:

### IGT changes ###

#### Possible regressions ####

  * igt@core_hotunplug@unbind-rebind:
    - fi-rkl-guc:         [PASS][1] -> [DMESG-WARN][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10454/fi-rkl-guc/igt@core_hotunplug@unbind-rebind.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20780/fi-rkl-guc/igt@core_hotunplug@unbind-rebind.html

  
Known issues
------------

  Here are the changes found in Patchwork_20780 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@runner@aborted:
    - fi-rkl-guc:         NOTRUN -> [FAIL][3] ([i915#1602])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20780/fi-rkl-guc/igt@runner@aborted.html

  
  [i915#1602]: https://gitlab.freedesktop.org/drm/intel/issues/1602


Participating hosts (37 -> 34)
------------------------------

  Missing    (3): fi-bdw-samus fi-bsw-cyan bat-jsl-1 


Build changes
-------------

  * Linux: CI_DRM_10454 -> Patchwork_20780

  CI-20190529: 20190529
  CI_DRM_10454: 224f5b80ba7d0caf6c4899e30d13cabde980bf49 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGT_6160: 4287344dd6a39d9036c5fb9a047a7d8f10bee981 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  Patchwork_20780: dd0b34c891dddd111cbe130d60b465fb03827a77 @ git://anongit.freedesktop.org/gfx-ci/linux


== Linux commits ==

dd0b34c891dd drm/doc/rfc: VM_BIND uapi definition
4bebdd1fc544 drm/doc/rfc: VM_BIND feature design document

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_20780/index.html

[-- Attachment #2: Type: text/html, Size: 3028 bytes --]