From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jerome Glisse Subject: Re: [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers Date: Fri, 22 Jun 2018 13:26:14 -0400 Message-ID: <20180622172614.GD3497@redhat.com> References: <20180622150242.16558-1-mhocko@kernel.org> <152968180950.11773.3374981930722769733@mail.alporthouse.com> <20180622155716.GE10465@dhcp22.suse.cz> <20180622161845.GA3497@redhat.com> <20180622164026.GA23674@dhcp22.suse.cz> <20180622164243.GB23674@dhcp22.suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <20180622164243.GB23674@dhcp22.suse.cz> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: Michal Hocko Cc: kvm@vger.kernel.org, =?us-ascii?B?PT9VVEYtOD9xP1JhZGltPTIwS3I9QzQ9OERtPUMzPUExPUM1PTk5Pz0=?= , David Airlie , Sudeep Dutt , dri-devel@lists.freedesktop.org, linux-mm@kvack.org, Andrea Arcangeli , "David (ChunMing) Zhou" , Dimitri Sivanich , linux-rdma@vger.kernel.org, amd-gfx@lists.freedesktop.org, Jason Gunthorpe , Doug Ledford , David Rientjes , xen-devel@lists.xenproject.org, intel-gfx@lists.freedesktop.org, Rodrigo Vivi , Boris Ostrovsky , Juergen Gross , Mike Marciniszyn , Dennis Dalessandro , Ashutosh Dixit , Alex Deucher List-Id: linux-rdma@vger.kernel.org T24gRnJpLCBKdW4gMjIsIDIwMTggYXQgMDY6NDI6NDNQTSArMDIwMCwgTWljaGFsIEhvY2tvIHdy b3RlOgo+IFtSZXNuZGluZyB3aXRoIHRoZSBDQyBsaXN0IGZpeGVkXQo+IAo+IE9uIEZyaSAyMi0w Ni0xOCAxODo0MDoyNiwgTWljaGFsIEhvY2tvIHdyb3RlOgo+ID4gT24gRnJpIDIyLTA2LTE4IDEy OjE4OjQ2LCBKZXJvbWUgR2xpc3NlIHdyb3RlOgo+ID4gPiBPbiBGcmksIEp1biAyMiwgMjAxOCBh dCAwNTo1NzoxNlBNICswMjAwLCBNaWNoYWwgSG9ja28gd3JvdGU6Cj4gPiA+ID4gT24gRnJpIDIy LTA2LTE4IDE2OjM2OjQ5LCBDaHJpcyBXaWxzb24gd3JvdGU6Cj4gPiA+ID4gPiBRdW90aW5nIE1p Y2hhbCBIb2NrbyAoMjAxOC0wNi0yMiAxNjowMjo0MikKPiA+ID4gPiA+ID4gSGksCj4gPiA+ID4g PiA+IHRoaXMgaXMgYW4gUkZDIGFuZCBub3QgdGVzdGVkIGF0IGFsbC4gSSBhbSBub3QgdmVyeSBm YW1pbGlhciB3aXRoIHRoZQo+ID4gPiA+ID4gPiBtbXUgbm90aWZpZXJzIHNlbWFudGljcyB2ZXJ5 IG11Y2ggc28gdGhpcyBpcyBhIGNydWRlIGF0dGVtcHQgdG8gYWNoaWV2ZQo+ID4gPiA+ID4gPiB3 aGF0IEkgbmVlZCBiYXNpY2FsbHkuIEl0IG1pZ2h0IGJlIGNvbXBsZXRlbHkgd3JvbmcgYnV0IEkg d291bGQgbGlrZQo+ID4gPiA+ID4gPiB0byBkaXNjdXNzIHdoYXQgd291bGQgYmUgYSBiZXR0ZXIg d2F5IGlmIHRoYXQgaXMgdGhlIGNhc2UuCj4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiBnZXRfbWFp bnRhaW5lcnMgZ2F2ZSBtZSBxdWl0ZSBsYXJnZSBsaXN0IG9mIHBlb3BsZSB0byBDQyBzbyBJIGhh ZCB0byB0cmltCj4gPiA+ID4gPiA+IGl0IGRvd24uIElmIHlvdSB0aGluayBJIGhhdmUgZm9yZ290 IHNvbWVib2R5LCBwbGVhc2UgbGV0IG1lIGtub3cKPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiBkaWZm IC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2k5MTUvaTkxNV9nZW1fdXNlcnB0ci5jIGIvZHJpdmVy cy9ncHUvZHJtL2k5MTUvaTkxNV9nZW1fdXNlcnB0ci5jCj4gPiA+ID4gPiA+IGluZGV4IDg1NGJk NTFiOTQ3OC4uNTI4NWRmOTMzMWZhIDEwMDY0NAo+ID4gPiA+ID4gPiAtLS0gYS9kcml2ZXJzL2dw dS9kcm0vaTkxNS9pOTE1X2dlbV91c2VycHRyLmMKPiA+ID4gPiA+ID4gKysrIGIvZHJpdmVycy9n cHUvZHJtL2k5MTUvaTkxNV9nZW1fdXNlcnB0ci5jCj4gPiA+ID4gPiA+IEBAIC0xMTIsMTAgKzEx MiwxMSBAQCBzdGF0aWMgdm9pZCBkZWxfb2JqZWN0KHN0cnVjdCBpOTE1X21tdV9vYmplY3QgKm1v KQo+ID4gPiA+ID4gPiAgICAgICAgIG1vLT5hdHRhY2hlZCA9IGZhbHNlOwo+ID4gPiA+ID4gPiAg fQo+ID4gPiA+ID4gPiAgCj4gPiA+ID4gPiA+IC1zdGF0aWMgdm9pZCBpOTE1X2dlbV91c2VycHRy X21uX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqX21uLAo+ID4g PiA+ID4gPiArc3RhdGljIGludCBpOTE1X2dlbV91c2VycHRyX21uX2ludmFsaWRhdGVfcmFuZ2Vf c3RhcnQoc3RydWN0IG1tdV9ub3RpZmllciAqX21uLAo+ID4gPiA+ID4gPiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IG1tX3N0cnVj dCAqbW0sCj4gPiA+ID4gPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIHN0YXJ0LAo+ID4gPiA+ID4gPiAtICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdW5zaWduZWQg bG9uZyBlbmQpCj4gPiA+ID4gPiA+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICB1bnNpZ25lZCBsb25nIGVuZCwKPiA+ID4gPiA+ID4gKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGJvb2wgYmxv Y2thYmxlKQo+ID4gPiA+ID4gPiAgewo+ID4gPiA+ID4gPiAgICAgICAgIHN0cnVjdCBpOTE1X21t dV9ub3RpZmllciAqbW4gPQo+ID4gPiA+ID4gPiAgICAgICAgICAgICAgICAgY29udGFpbmVyX29m KF9tbiwgc3RydWN0IGk5MTVfbW11X25vdGlmaWVyLCBtbik7Cj4gPiA+ID4gPiA+IEBAIC0xMjQs NyArMTI1LDcgQEAgc3RhdGljIHZvaWQgaTkxNV9nZW1fdXNlcnB0cl9tbl9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXIgKl9tbiwKPiA+ID4gPiA+ID4gICAgICAgICBM SVNUX0hFQUQoY2FuY2VsbGVkKTsKPiA+ID4gPiA+ID4gIAo+ID4gPiA+ID4gPiAgICAgICAgIGlm IChSQl9FTVBUWV9ST09UKCZtbi0+b2JqZWN0cy5yYl9yb290KSkKPiA+ID4gPiA+ID4gLSAgICAg ICAgICAgICAgIHJldHVybjsKPiA+ID4gPiA+ID4gKyAgICAgICAgICAgICAgIHJldHVybiAwOwo+ ID4gPiA+ID4gCj4gPiA+ID4gPiBUaGUgcHJpbmNpcGxlIHdhaXQgaGVyZSBpcyBmb3IgdGhlIEhX IChldmVuIGFmdGVyIGZpeGluZyBhbGwgdGhlIGxvY2tzCj4gPiA+ID4gPiB0byBiZSBub3Qgc28g Y29hcnNlLCB3ZSBzdGlsbCBoYXZlIHRvIHdhaXQgZm9yIHRoZSBIVyB0byBmaW5pc2ggaXRzCj4g PiA+ID4gPiBhY2Nlc3MpLgo+ID4gPiA+IAo+ID4gPiA+IElzIHRoaXMgd2FpdCBib3VuZCBvciBp dCBjYW4gdGFrZSBiYXNpY2FsbHkgYXJiaXRyYXJ5IGFtb3VudCBvZiB0aW1lPwo+ID4gPiAKPiA+ ID4gQXJiaXRyYXJ5IGFtb3VudCBvZiB0aW1lIGJ1dCBpbiBkZXNrdG9wIHVzZSBjYXNlIHlvdSBj YW4gYXNzdW1lIHRoYXQKPiA+ID4gaXQgc2hvdWxkIG5ldmVyIGdvIGFib3ZlIDE2bXMgZm9yIGEg NjBmcmFtZSBwZXIgc2Vjb25kIHJlbmRlcmluZyBvZgo+ID4gPiB5b3VyIGRlc2t0b3AgKGluIEdQ VSBjb21wdXRlIGNhc2UgdGhpcyBraW5kIG9mIGFzc3VtcHRpb24gZG9lcyBub3QKPiA+ID4gaG9s ZCkuIElzIHRoZSBwcm9jZXNzIGV4aXRfc3RhdGUgYWxyZWFkeSB1cGRhdGVkIGJ5IHRoZSB0aW1l IHRoaXMgbW11Cj4gPiA+IG5vdGlmaWVyIGNhbGxiYWNrcyBoYXBwZW4gPwo+ID4gCj4gPiBXaGF0 IGRvIHlvdSBtZWFuPyBUaGUgcHJvY2VzcyBpcyBraWxsZWQgKGJ5IFNJR0tJTEwpIGF0IHRoZSB0 aW1lIGJ1dCB3ZQo+ID4gZG8gbm90IGtub3cgbXVjaCBtb3JlIHRoYW4gdGhhdC4gVGhlIHRhc2sg bWlnaHQgYmUgc3R1Y2sgYW55d2hlcmUgaW4gdGhlCj4gPiBrZXJuZWwgYmVmb3JlIGhhbmRsaW5n IHRoYXQgc2lnbmFsLgoKSSB3YXMgd29uZGVyaW5nIGlmIGFub3RoZXIgdGhyZWFkIG1pZ2h0IHN0 aWxsIGJlIGRlcmVmZXJlbmNpbmcgYW55IG9mCnRoZSBzdHJ1Y3R1cmUgY29uY3VycmVudGx5IHdp dGggdGhlIE9PTSBtbXUgbm90aWZpZXIgY2FsbGJhY2suIFNhZGRseQp5ZXMsIGl0IHdvdWxkIGJl IHNpbXBsZXIgaWYgd2UgY291bGQgbWFrZSBzdWNoIGFzc3VtcHRpb24uCgo+ID4gCj4gPiA+ID4g PiBUaGUgZmlyc3QgcGFzcyB3b3VsZCBiZSB0aGVuIHRvIG5vdCBkbyBhbnl0aGluZyBoZXJlIGlm Cj4gPiA+ID4gPiAhYmxvY2thYmxlLgo+ID4gPiA+IAo+ID4gPiA+IHNvbWV0aGluZyBsaWtlIHRo aXM/IChpbmNyZW1lbnRhbCBkaWZmKQo+ID4gPiAKPiA+ID4gV2hhdCBpIHdhbnRlZCB0byBkbyB3 aXRoIEhNTSBhbmQgbW11IG5vdGlmaWVyIGlzIHNwbGl0IHRoZSBpbnZhbGlkYXRpb24KPiA+ID4g aW4gMiBwYXNzLiBGaXJzdCBwYXNzIHRlbGwgdGhlIGRyaXZlcnMgdG8gc3RvcC9jYW5jZWwgcGVu ZGluZyBqb2JzIHRoYXQKPiA+ID4gZGVwZW5kcyBvbiB0aGUgcmFuZ2UgYW5kIGludmFsaWRhdGUg aW50ZXJuYWwgZHJpdmVyIHN0YXRlcyAobGlrZSBjbGVhcgo+ID4gPiBidWZmZXIgb2JqZWN0IHBh Z2VzIGFycmF5IGluIGNhc2Ugb2YgR1BVIGJ1dCBub3QgR1BVIHBhZ2UgdGFibGUpLiBXaGlsZQo+ ID4gPiB0aGUgc2Vjb25kIGNhbGxiYWNrIHdvdWxkIGRvIHRoZSBhY3R1YWwgd2FpdCBvbiB0aGUg R1BVIHRvIGJlIGRvbmUgYW5kCj4gPiA+IHVwZGF0ZSB0aGUgR1BVIHBhZ2UgdGFibGUuCj4gPiAK PiA+IFdoYXQgY2FuIHlvdSBkbyBhZnRlciB0aGUgZmlyc3QgcGhhc2U/IENhbiBJIHVubWFwIHRo ZSByYW5nZT8KCk5vIHlvdSBjYW4ndCBkbyBhbnl0aGluZyBidXQgdGhpcyBmb3JjZSBzeW5jaHJv bml6YXRpb24gYXMgYW55IG90aGVyCnRocmVhZCB0aGF0IGNvbmN1cnJlbnRseSB0cmllIHRvIGRv IHNvbWV0aGluZyB3aXRoIHRob3NlIHdvdWxkIHF1ZXVlCnVwLiBTbyBpdCB3b3VsZCBzZXJpYWxp emUgdGhpbmcuIEFsc28gbWFpbiBtb3RpdmF0aW9uIG9uIG15IHNpZGUgaXMKbXVsdGktR1BVLCBy aWdodCBub3cgbXVsdGktR1BVIGFyZSBub3QgdGhhdCBjb21tb24gYnV0IHRoaXMgaXMgY2hhbmdp bmcKcXVpY2tseSBhbmQgd2hhdCB3ZSBzZWUgb24gaGlnaCBlbmQgKDQsIDggb3IgMTYgR1BVcyBw ZXIgc29ja2V0KSBpcwpzcHJlYWRpbmcgaW50byBtb3JlIGNvbmZpZ3VyYXRpb25zLiBIZXJlIGlu IG11dGxpIEdQVSBjYXNlIHNwbGl0dGluZwppbiB0d28gd291bGQgYXZvaWQgaGF2aW5nIHRvIGZ1 bGx5IHdhaXQgb24gZmlyc3QgR1BVIGJlZm9yZSB0cnlpbmcgdG8KaW52YWlkYXRlIG9uIHNlY29u ZCBHUFUsIHNvIG9uIGFuZCBzbyBmb3J0aC4KCj4gPiAKPiA+ID4gTm93IGluIHRoaXMgc2NoZW1l IGluIGNhc2UgdGhlIHRhc2sgaXMgYWxyZWFkeSBpbiBzb21lIGV4aXQgc3RhdGUgYW5kCj4gPiA+ IHRoYXQgYWxsIENQVSB0aHJlYWRzIGFyZSBmcm96ZW4va2lsbCB0aGVuIHdlIGNhbiBwcm9iYWJs eSBmaW5kIGEgd2F5IHRvCj4gPiA+IGRvIHRoZSBmaXJzdCBwYXRoIG1vc3RseSBsb2NrIGxlc3Mu IEFGQUlDUiBub3IgQU1EIG5vciBJbnRlbCBhbGxvdyB0bwo+ID4gPiBzaGFyZSB1c2VycHRyIGJv IGhlbmNlIGEgdXB0ciBibyBzaG91bGQgb25seSBldmVyIGJlIGFjY2VzcyB0aHJvdWdoCj4gPiA+ IGlvY3RsIHN1Ym1pdGVkIGJ5IHRoZSBwcm9jZXNzLgo+ID4gPiAKPiA+ID4gVGhlIHNlY29uZCBj YWxsIGNhbiB0aGVuIGJlIGRlbGF5ZWQgYW5kIHBpbmcgZnJvbSB0aW1lIHRvIHRpbWUgdG8gc2Vl Cj4gPiA+IGlmIEdQVSBqb2JzIGFyZSBkb25lLgo+ID4gPiAKPiA+ID4gCj4gPiA+IE5vdGUgdGhh dCB3aGF0IHlvdSBwcm9wb3NlIG1pZ2h0IHN0aWxsIGJlIHVzZWZ1bCBhcyBpbiBjYXNlIHRoZXJl IGlzCj4gPiA+IG5vIGJ1ZmZlciBvYmplY3QgZm9yIGEgcmFuZ2UgdGhlbiBPT00gY2FuIG1ha2Ug cHJvZ3Jlc3MgaW4gZnJlZWluZyBhCj4gPiA+IHJhbmdlIG9mIG1lbW9yeS4gSXQgaXMgdmVyeSBs aWtlbHkgdGhhdCBzaWduaWZpY2FudCB2aXJ0dWFsIGFkZHJlc3MKPiA+ID4gcmFuZ2Ugb2YgYSBw cm9jZXNzIGFuZCBiYWNraW5nIG1lbW9yeSBjYW4gYmUgcmVjbGFpbSB0aGF0IHdheS4gVGhpcwo+ ID4gPiBhc3N1bWUgT09NIHJlY2xhaW0gdm1hIGJ5IHZtYSBvciBpbiBzb21lIGZvcm0gb2YgZ3Jh bnVsYXJpdHkgbGlrZQo+ID4gPiByZWNsYWltaW5nIDFHQiBieSAxR0IuIE9yIHdlIGNvdWxkIGFs c28gdXBkYXRlIGJsb2NraW5nIGNhbGxiYWNrIHRvCj4gPiA+IHJldHVybiByYW5nZSB0aGF0IGFy ZSBibG9ja2luZyB0aGF0IHdheSBPT00gY2FuIHJlY2xhaW0gYXJvdW5kLgo+ID4gCj4gPiBFeGFj dGx5IG15IHBvaW50LiBXaGF0IHdlIGhhdmUgcmlnaHQgbm93IGlzIGFsbCBvciBub3RoaW5nIHdo aWNoIGlzCj4gPiBvYnZpb3VzbHkgdG9vIGNvYXJzZSB0byBiZSB1c2VmdWwuCgpZZXMgaSB0aGlu ayBpdCBpcyBhIGdvb2Qgc3RlcCBpbiB0aGUgcmlnaHQgZGlyZWN0aW9uLgoKQ2hlZXJzLApKw6ly w7RtZQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpJbnRl bC1nZnggbWFpbGluZyBsaXN0CkludGVsLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6 Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9pbnRlbC1nZngK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f198.google.com (mail-qk0-f198.google.com [209.85.220.198]) by kanga.kvack.org (Postfix) with ESMTP id A3E2B6B0003 for ; Fri, 22 Jun 2018 13:26:19 -0400 (EDT) Received: by mail-qk0-f198.google.com with SMTP id w74-v6so6198170qka.4 for ; Fri, 22 Jun 2018 10:26:19 -0700 (PDT) Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id r5-v6si3038817qvc.200.2018.06.22.10.26.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Jun 2018 10:26:18 -0700 (PDT) Date: Fri, 22 Jun 2018 13:26:14 -0400 From: Jerome Glisse Subject: Re: [Intel-gfx] [RFC PATCH] mm, oom: distinguish blockable mode for mmu notifiers Message-ID: <20180622172614.GD3497@redhat.com> References: <20180622150242.16558-1-mhocko@kernel.org> <152968180950.11773.3374981930722769733@mail.alporthouse.com> <20180622155716.GE10465@dhcp22.suse.cz> <20180622161845.GA3497@redhat.com> <20180622164026.GA23674@dhcp22.suse.cz> <20180622164243.GB23674@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180622164243.GB23674@dhcp22.suse.cz> Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko Cc: "David (ChunMing) Zhou" , Paolo Bonzini , =?us-ascii?B?PT9VVEYtOD9xP1JhZGltPTIwS3I9QzQ9OERtPUMzPUExPUM1PTk5Pz0=?= , Alex Deucher , =?us-ascii?B?PT9VVEYtOD9xP0NocmlzdGlhbj0yMEs9QzM9QjZuaWc/PQ==?= , David Airlie , Jani Nikula , Joonas Lahtinen , Rodrigo Vivi , Doug Ledford , Jason Gunthorpe , Mike Marciniszyn , Dennis Dalessandro , Sudeep Dutt , Ashutosh Dixit , Dimitri Sivanich , Boris Ostrovsky , Juergen Gross , Andrea Arcangeli , kvm@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-rdma@vger.kernel.org, xen-devel@lists.xenproject.org, linux-mm@kvack.org, David Rientjes On Fri, Jun 22, 2018 at 06:42:43PM +0200, Michal Hocko wrote: > [Resnding with the CC list fixed] > > On Fri 22-06-18 18:40:26, Michal Hocko wrote: > > On Fri 22-06-18 12:18:46, Jerome Glisse wrote: > > > On Fri, Jun 22, 2018 at 05:57:16PM +0200, Michal Hocko wrote: > > > > On Fri 22-06-18 16:36:49, Chris Wilson wrote: > > > > > Quoting Michal Hocko (2018-06-22 16:02:42) > > > > > > Hi, > > > > > > this is an RFC and not tested at all. I am not very familiar with the > > > > > > mmu notifiers semantics very much so this is a crude attempt to achieve > > > > > > what I need basically. It might be completely wrong but I would like > > > > > > to discuss what would be a better way if that is the case. > > > > > > > > > > > > get_maintainers gave me quite large list of people to CC so I had to trim > > > > > > it down. If you think I have forgot somebody, please let me know > > > > > > > > > > > diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > > index 854bd51b9478..5285df9331fa 100644 > > > > > > --- a/drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > > +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > > @@ -112,10 +112,11 @@ static void del_object(struct i915_mmu_object *mo) > > > > > > mo->attached = false; > > > > > > } > > > > > > > > > > > > -static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, > > > > > > +static int i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, > > > > > > struct mm_struct *mm, > > > > > > unsigned long start, > > > > > > - unsigned long end) > > > > > > + unsigned long end, > > > > > > + bool blockable) > > > > > > { > > > > > > struct i915_mmu_notifier *mn = > > > > > > container_of(_mn, struct i915_mmu_notifier, mn); > > > > > > @@ -124,7 +125,7 @@ static void i915_gem_userptr_mn_invalidate_range_start(struct mmu_notifier *_mn, > > > > > > LIST_HEAD(cancelled); > > > > > > > > > > > > if (RB_EMPTY_ROOT(&mn->objects.rb_root)) > > > > > > - return; > > > > > > + return 0; > > > > > > > > > > The principle wait here is for the HW (even after fixing all the locks > > > > > to be not so coarse, we still have to wait for the HW to finish its > > > > > access). > > > > > > > > Is this wait bound or it can take basically arbitrary amount of time? > > > > > > Arbitrary amount of time but in desktop use case you can assume that > > > it should never go above 16ms for a 60frame per second rendering of > > > your desktop (in GPU compute case this kind of assumption does not > > > hold). Is the process exit_state already updated by the time this mmu > > > notifier callbacks happen ? > > > > What do you mean? The process is killed (by SIGKILL) at the time but we > > do not know much more than that. The task might be stuck anywhere in the > > kernel before handling that signal. I was wondering if another thread might still be dereferencing any of the structure concurrently with the OOM mmu notifier callback. Saddly yes, it would be simpler if we could make such assumption. > > > > > > > The first pass would be then to not do anything here if > > > > > !blockable. > > > > > > > > something like this? (incremental diff) > > > > > > What i wanted to do with HMM and mmu notifier is split the invalidation > > > in 2 pass. First pass tell the drivers to stop/cancel pending jobs that > > > depends on the range and invalidate internal driver states (like clear > > > buffer object pages array in case of GPU but not GPU page table). While > > > the second callback would do the actual wait on the GPU to be done and > > > update the GPU page table. > > > > What can you do after the first phase? Can I unmap the range? No you can't do anything but this force synchronization as any other thread that concurrently trie to do something with those would queue up. So it would serialize thing. Also main motivation on my side is multi-GPU, right now multi-GPU are not that common but this is changing quickly and what we see on high end (4, 8 or 16 GPUs per socket) is spreading into more configurations. Here in mutli GPU case splitting in two would avoid having to fully wait on first GPU before trying to invaidate on second GPU, so on and so forth. > > > > > Now in this scheme in case the task is already in some exit state and > > > that all CPU threads are frozen/kill then we can probably find a way to > > > do the first path mostly lock less. AFAICR nor AMD nor Intel allow to > > > share userptr bo hence a uptr bo should only ever be access through > > > ioctl submited by the process. > > > > > > The second call can then be delayed and ping from time to time to see > > > if GPU jobs are done. > > > > > > > > > Note that what you propose might still be useful as in case there is > > > no buffer object for a range then OOM can make progress in freeing a > > > range of memory. It is very likely that significant virtual address > > > range of a process and backing memory can be reclaim that way. This > > > assume OOM reclaim vma by vma or in some form of granularity like > > > reclaiming 1GB by 1GB. Or we could also update blocking callback to > > > return range that are blocking that way OOM can reclaim around. > > > > Exactly my point. What we have right now is all or nothing which is > > obviously too coarse to be useful. Yes i think it is a good step in the right direction. Cheers, Jerome