From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Deucher Subject: Re: [PATCH v6 5/5] drm/amdgpu: resize VRAM BAR for CPU access v2 Date: Tue, 6 Jun 2017 09:53:34 -0400 Message-ID: References: <1494348547-1465-1-git-send-email-deathsimple@vodafone.de> <1494348547-1465-6-git-send-email-deathsimple@vodafone.de> <20170601201430.GC12257@bhelgaas-glaptop.roam.corp.google.com> <72e061a5-9684-06e2-3021-80de8ca97bd7@vodafone.de> <20170602202631.GA1452@bhelgaas-glaptop.roam.corp.google.com> <779a883a-265a-ae04-b0cd-8cb3599f0dc0@vodafone.de> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <779a883a-265a-ae04-b0cd-8cb3599f0dc0-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: =?UTF-8?Q?Christian_K=C3=B6nig?= Cc: David Airlie , Linux PCI , amd-gfx list , platform-driver-x86-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Bjorn Helgaas , Maling list - DRI developers , Alex Deucher List-Id: platform-driver-x86.vger.kernel.org T24gVHVlLCBKdW4gNiwgMjAxNyBhdCA3OjUxIEFNLCBDaHJpc3RpYW4gS8O2bmlnIDxkZWF0aHNp bXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6Cj4gQW0gMDIuMDYuMjAxNyB1bSAyMjoyNiBzY2hyaWVi IEJqb3JuIEhlbGdhYXM6Cj4+Cj4+IE9uIEZyaSwgSnVuIDAyLCAyMDE3IGF0IDExOjMyOjIxQU0g KzAyMDAsIENocmlzdGlhbiBLw7ZuaWcgd3JvdGU6Cj4+Pgo+Pj4gSGkgQmpvcm4sCj4+Pgo+Pj4g c29ycnkgZm9yIG5vdCByZXNwb25kaW5nIGVhcmxpZXIgYW5kIHRoYW5rcyBmb3IgcGlja2luZyB0 aGlzIHRocmVhZAo+Pj4gdXAgYWdhaW4uCj4+Pgo+Pj4gQW0gMDEuMDYuMjAxNyB1bSAyMjoxNCBz Y2hyaWViIEJqb3JuIEhlbGdhYXM6Cj4+Pj4KPj4+PiBbK2NjIEFETUdQVSwgRFJNIGZvbGtzXQo+ Pj4+Cj4+Pj4gT24gVHVlLCBNYXkgMDksIDIwMTcgYXQgMDY6NDk6MDdQTSArMDIwMCwgQ2hyaXN0 aWFuIEvDtm5pZyB3cm90ZToKPj4+Pj4KPj4+Pj4gW1NOSVBdCj4+Pj4+ICsvKioKPj4+Pj4gKyAq IGFtZGdwdV9yZXNpemVfYmFyMCAtIHRyeSB0byByZXNpemUgQkFSMAo+Pj4+PiArICoKPj4+Pj4g KyAqIEBhZGV2OiBhbWRncHVfZGV2aWNlIHBvaW50ZXIKPj4+Pj4gKyAqCj4+Pj4+ICsgKiBUcnkg dG8gcmVzaXplIEJBUjAgdG8gbWFrZSBhbGwgVlJBTSBDUFUgYWNjZXNzaWJsZS4KPj4+Pj4gKyAq Lwo+Pj4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFk ZXYpCj4+Pj4+ICt7Cj4+Pj4+ICsgICAgICAgdTY0IHNwYWNlX25lZWRlZCA9IHJvdW5kdXBfcG93 X29mX3R3byhhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSk7Cj4+Pj4+ICsgICAgICAgdTMyIHJiYXJf c2l6ZSA9IG9yZGVyX2Jhc2VfMigoKHNwYWNlX25lZWRlZCA+PiAyMCkgfCAxKSkgLTE7Cj4+Pj4+ ICsgICAgICAgdTE2IGNtZDsKPj4+Pj4gKyAgICAgICBpbnQgcjsKPj4+Pj4gKwo+Pj4+PiArICAg ICAgIC8qIEZyZWUgdGhlIGRvb3JiZWxsIG1hcHBpbmcsIGl0IG1vc3QgbGlrZWx5IG5lZWRzIHRv IG1vdmUgYXMKPj4+Pj4gd2VsbCAqLwo+Pj4+PiArICAgICAgIGFtZGdwdV9kb29yYmVsbF9maW5p KGFkZXYpOwo+Pj4+PiArICAgICAgIHBjaV9yZWxlYXNlX3Jlc291cmNlKGFkZXYtPnBkZXYsIDIp Owo+Pj4+PiArCj4+Pj4+ICsgICAgICAgLyogRGlzYWJsZSBtZW1vcnkgZGVjb2Rpbmcgd2hpbGUg d2UgY2hhbmdlIHRoZSBCQVIgYWRkcmVzc2VzCj4+Pj4+IGFuZCBzaXplICovCj4+Pj4+ICsgICAg ICAgcGNpX3JlYWRfY29uZmlnX3dvcmQoYWRldi0+cGRldiwgUENJX0NPTU1BTkQsICZjbWQpOwo+ Pj4+PiArICAgICAgIHBjaV93cml0ZV9jb25maWdfd29yZChhZGV2LT5wZGV2LCBQQ0lfQ09NTUFO RCwKPj4+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY21kICYgflBDSV9DT01NQU5E X01FTU9SWSk7Cj4+Pj4+ICsKPj4+Pj4gKyAgICAgICByID0gcGNpX3Jlc2l6ZV9yZXNvdXJjZShh ZGV2LT5wZGV2LCAwLCByYmFyX3NpemUpOwo+Pj4+PiArICAgICAgIGlmIChyID09IC1FTk9TUEMp Cj4+Pj4+ICsgICAgICAgICAgICAgICBEUk1fSU5GTygiTm90IGVub3VnaCBQQ0kgYWRkcmVzcyBz cGFjZSBmb3IgYSBsYXJnZQo+Pj4+PiBCQVIuIik7Cj4+Pj4+ICsgICAgICAgZWxzZSBpZiAociAm JiByICE9IC1FTk9UU1VQUCkKPj4+Pj4gKyAgICAgICAgICAgICAgIERSTV9FUlJPUigiUHJvYmxl bSByZXNpemluZyBCQVIwICglZCkuIiwgcik7Cj4+Pj4+ICsKPj4+Pj4gKyAgICAgICBwY2lfd3Jp dGVfY29uZmlnX3dvcmQoYWRldi0+cGRldiwgUENJX0NPTU1BTkQsIGNtZCk7Cj4+Pj4+ICsKPj4+ Pj4gKyAgICAgICAvKiBXaGVuIHRoZSBkb29yYmVsbCBCQVIgaXNuJ3QgYXZhaWxhYmxlIHdlIGhh dmUgbm8gY2hhbmNlIG9mCj4+Pj4+ICsgICAgICAgICogdXNpbmcgdGhlIGRldmljZS4KPj4+Pj4g KyAgICAgICAgKi8KPj4+Pj4gKyAgICAgICBCVUdfT04oYW1kZ3B1X2Rvb3JiZWxsX2luaXQoYWRl dikpOwo+Pj4+Cj4+Pj4gVGhpcyBhbWRncHVfZG9vcmJlbGxfZmluaSgpL2FtZGdwdV9kb29yYmVs bF9pbml0KCkgdGhpbmcgZG9lc24ndCBsb29rCj4+Pj4gcmlnaHQuICBhbWRncHVfZGV2aWNlX2lu aXQoKSBvbmx5IGNhbGxzIGFtZGdwdV9kb29yYmVsbF9pbml0KCkgZm9yCj4+Pj4gImFkZXYtPmFz aWNfdHlwZSA+PSBDSElQX0JPTkFJUkUiLCBidXQgd2UgY2FsbCBpdCB1bmNvbmRpdGlvbmFsbHkK Pj4+PiBoZXJlLgo+Pj4+Cj4+Pj4gVGhpcyBpcyB0aGUgY2FsbCBncmFwaDoKPj4+Pgo+Pj4+ICAg IGFtZGdwdV9kZXZpY2VfaW5pdAo+Pj4+ICAgICAgYWRldi0+cm1taW9fYmFzZSA9IHBjaV9yZXNv dXJjZV9zdGFydChhZGV2LT5wZGV2LCA1KSAgICMgMiBmb3IgPAo+Pj4+IEJPTkFJUkUKPj4+PiAg ICAgIGFkZXYtPnJtbWlvID0gaW9yZW1hcChhZGV2LT5ybW1pb19iYXNlLCAuLi4pCj4+Pj4gICAg ICBEUk1fSU5GTygicmVnaXN0ZXIgbW1pbyBiYXNlOiAweCUwOFhcbiIsCj4+Pj4gKHVpbnQzMl90 KWFkZXYtPnJtbWlvX2Jhc2UpCj4+Pj4gICAgICBpZiAoYWRldi0+YXNpY190eXBlID49IENISVBf Qk9OQUlSRSkgewo+Pj4+ICAgICAgICBhbWRncHVfZG9vcmJlbGxfaW5pdAo+Pj4+ICAgICAgICAg YWRldi0+ZG9vcmJlbGwuYmFzZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAyKQo+ Pj4+ICAgICAgICAgYWRldi0+ZG9vcmJlbGwucHRyID0gaW9yZW1hcChhZGV2LT5kb29yYmVsbC5i YXNlLCAuLi4pCj4+Pj4gICAgICB9Cj4+Pj4gICAgICBhbWRncHVfaW5pdAo+Pj4+ICAgICAgICBn bWNfdjdfMF9zd19pbml0ICAgICAgICAgICAgICAjIGdtY192N18wX2lwX2Z1bmNzLnN3X2luaXQK Pj4+PiAgICAgICAgIGdtY192N18wX21jX2luaXQKPj4+PiAgICArICAgICAgICAgYW1kZ3B1X3Jl c2l6ZV9iYXIwCj4+Pj4gICAgKyAgICAgICAgICAgYW1kZ3B1X2Rvb3JiZWxsX2ZpbmkKPj4+PiAg ICArICAgICAgICAgICBwY2lfcmVsZWFzZV9yZXNvdXJjZShhZGV2LT5wZGV2LCAyKQo+Pj4+ICAg ICsgICAgICAgICAgIHBjaV9yZXNpemVfcmVzb3VyY2UoYWRldi0+cGRldiwgMCwgc2l6ZSkKPj4+ PiAgICArICAgICAgICAgICBhbWRncHVfZG9vcmJlbGxfaW5pdAo+Pj4+ICAgICAgICAgICBhZGV2 LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCkKPj4+Pgo+ Pj4+IElmICJhc2ljX3R5cGUgPCBDSElQX0JPTkFJUkUiLCB3ZSBpb3JlbWFwcGVkIEJBUiAyIGlu Cj4+Pj4gYW1kZ3B1X2RldmljZV9pbml0KCksIHRoZW4gd2UgcmVsZWFzZWQgdGhlIHJlc291cmNl IGhlcmUgYW5kIG5ldmVyCj4+Pj4gdXBkYXRlZCB0aGUgaW9yZW1hcC4KPj4+Cj4+PiBUaGUgZmly c3QgaGFyZHdhcmUgd2l0aCBhIHJlc2l6ZWFibGUgQkFSIEkgY291bGQgZmluZCBpcyBhIFRvbmdh LAo+Pj4gYW5kIHRoYXQgaXMgZXZlbiBhIGdlbmVyYXRpb24gbGF0ZXIgdGhhbiBCb25haXJlLgo+ Pj4KPj4+IFNvIHdlIGFyZSBuZXZlciBnb2luZyB0byBjYWxsIHRoaXMgY29kZSBvbiBlYXJsaWVy IGhhcmR3YXJlIGdlbmVyYXRpb25zLgo+Pgo+PiBUaGUgcHJvYmxlbSB3aXRoIHRoYXQgaXMgdGhh dCBpdCdzIGltcG9zc2libGUgZm9yIGEgY29kZSByZWFkZXIgdG8KPj4gdmVyaWZ5IHRoYXQuICBT byBhZGRpbmcgYSBjaGVjayBpcyB1Z2x5IGJ1dCBJIHRoaW5rIG1ha2VzIGl0IG1vcmUKPj4gcmVh ZGFibGUuCj4KPgo+IEdvb2QgcG9pbnQuIEkgd2lsbCBqdXN0IG1vdmUgdGhlIGNoZWNrIGludG8g dGhlIGZ1bmN0aW9uIGl0c2VsZiwgdGhhdCBzaG91bGQKPiBoYW5kbGUgYWxsIHN1Y2ggY2FzZXMu Cj4KPj4+PiAgRnJvbSB0aGUgUENJIGNvcmUgcGVyc3BlY3RpdmUsIGl0IHdvdWxkIGJlIG11Y2gg Y2xlYW5lciB0byBkbyB0aGUgQkFSCj4+Pj4gcmVzaXplIGJlZm9yZSB0aGUgZHJpdmVyIGNhbGxz IHBjaV9lbmFibGVfZGV2aWNlKCkuICBJZiB0aGF0IGNvdWxkIGJlCj4+Pj4gZG9uZSwgdGhlcmUg d291bGQgYmUgbm8gbmVlZCBmb3IgdGhpcyBzb3J0IG9mIHNodXRkb3duL3JlaW5pdCBzdHVmZgo+ Pj4+IGFuZCB3ZSB3b3VsZG4ndCBoYXZlIHRvIHdvcnJ5IGFib3V0IGlzc3VlcyBsaWtlIHRoZXNl LiAgVGhlIGFtZGdwdQo+Pj4+IGluaXQgcGF0aCBpcyBwcmV0dHkgY29tcGxpY2F0ZWQsIHNvIEkg ZG9uJ3Qga25vdyB3aGV0aGVyIHRoaXMgaXMKPj4+PiBwb3NzaWJsZS4KPj4+Cj4+PiBJIGNvbXBs ZXRlbHkgYWdyZWUgb24gdGhpcyBhbmQgaXQgaXMgYWN0dWFsbHkgdGhlIGFwcHJvYWNoIEkgdHJp ZWQgZmlyc3QuCj4+Pgo+Pj4gVGhlcmUgYXJlIGp1c3QgdHdvIHByb2JsZW1zIHdpdGggdGhpcyBh cHByb2FjaDoKPj4+IDEuIFdoZW4gdGhlIGFtZGdwdSBkcml2ZXIgaXMgbG9hZGVkIHRoZXJlIGNh biBhbHJlYWR5IGJlIHRoZSBWR0EKPj4+IGNvbnNvbGUsIFZlc2Egb3IgRUZJIGRyaXZlciBhY3Rp dmUgZm9yIHRoZSBkZXZpY2UgYW5kIGRpc3BsYXlpbmcgdGhlCj4+PiBzcGxhc2ggc2NyZWVuLgo+ Pj4KPj4+IFdoZW4gd2UgcmVzaXplIGFuZCBtb3N0IGxpa2VseSByZWxvY2F0ZSB0aGUgQkFSIHdo aWxlIHRob3NlIGRyaXZlcnMKPj4+IGFyZSBhY3RpdmUgaXQgd2lsbCBjZXJ0YWlubHkgY2F1c2Ug cHJvYmxlbXMuCj4+Pgo+Pj4gV2hhdCBhbWRncHUgZG9lcyBiZWZvcmUgdHJ5aW5nIHRvIHJlc2l6 ZSB0aGUgQkFSIGlzIGtpY2tpbmcgb3V0Cj4+PiBvdGhlciBkcml2ZXIgYW5kIG1ha2luZyBzdXJl IGl0IGhhcyBleGNsdXNpdmUgYWNjZXNzIHRvIHRoZQo+Pj4gaGFyZHdhcmUuCj4+Cj4+IEkgZG9u J3QgdW5kZXJzdGFuZCB0aGUgcHJvYmxlbSBoZXJlIHlldC4gIElmIHlvdSBuZWVkIHRvIGVuYWJs ZSB0aGUKPj4gZGV2aWNlLCB0aGVuIGRpc2FibGUgaXQsIHJlc2l6ZSwgYW5kIHJlLWVuYWJsZSBp dCwgdGhhdCdzIGZpbmUuCj4KPgo+IFRoZSBpc3N1ZSBpcyB3ZSBuZXZlciBlbmFibGUgdGhlIGRl dmljZSBvdXJzZWxmIGluIGFtZGdwdSwgZXhjZXB0IGZvciBzb21lCj4gcmFyZSBjYXNlcyBkdXJp bmcgcmVzdW1lLgo+Cj4gSW4gbW9zdCBvZiB0aGUgY2FzZXMgd2UgaGF2ZSB0byBoYW5kbGUgdGhp cyBpcyB0aGUgcHJpbWFyeSBkaXNwbGF5IGRldmljZQo+IHdoaWNoIGlzIGVuYWJsZWQgYnkgZWl0 aGVyIHRoZSBCSU9TLCBWR0EgY29uc29sZSwgVmVzYUZCIG9yIEVGSUZCLiBBbWRncHUKPiBqdXN0 IGtpY2tzIG91dCB3aGF0ZXZlciBkcml2ZXIgd2FzIHJlc3BvbnNpYmxlIGZvciB0aGUgZGV2aWNl IHByZXZpb3VzbHkgYW5kCj4gdGFrZXMgb3Zlci4KPgo+IEkgY291bGQgb2YgY291cnNlIGRvIHRo ZSBkaXNhYmxlL3Jlc2l6ZS9yZWVuYWJsZSBkYW5jZSwgYnV0IEkgd291bGQgcmF0aGVyCj4gd2Fu dCB0byBhdm9pZCB0aGF0Lgo+Cj4gVGhlIGhhcmR3YXJlIGlzIG1vc3QgbGlrZWx5IGFscmVhZHkg ZGlzcGxheWluZyBhIGJvb3Qgc3BsYXNoIGFuZCB3ZSB3YW50IHRvCj4gdHJhbnNpdCB0byB0aGUg ZGVza3RvcCB3aXRob3V0IGFueSBmbGlja2VyaW5nIChhdCBsZWFzdCB0aGF0J3MgdGhlIGxvbmcg dGVybQo+IGdvYWwpLiBDb21wbGV0ZWx5IGRpc2FibGluZyB0aGUgZGV2aWNlIHRvIGRvIHRoaXMg ZG9lc24ndCBzb3VuZHMgbGlrZSBhIGdvb2QKPiBpZGVhIGlmIHdlIHdhbnQgdGhhdC4KPgo+PiBU aGUgaW1wb3J0YW50IHRoaW5nIEknbSBsb29raW5nIGZvciBpcyB0aGF0IHRoZSByZXNpemUgaGFw cGVucyBiZWZvcmUKPj4gYSBwY2lfZW5hYmxlX2RldmljZSgpLCBiZWNhdXNlIHBjaV9lbmFibGVf ZGV2aWNlKCkgaXMgdGhlIHN5bmMgcG9pbnQKPj4gd2hlcmUgdGhlIFBDSSBjb3JlIGVuYWJsZXMg cmVzb3VyY2VzIGFuZCBtYWtlcyB0aGVtIGF2YWlsYWJsZSB0byB0aGUKPj4gZHJpdmVyLiAgRHJp dmVycyBrbm93IHRoYXQgdGhleSBjYW4ndCBsb29rIGF0IHRoZSByZXNvdXJjZXMgYmVmb3JlCj4+ IHRoYXQgcG9pbnQuICBUaGVyZSdzIGEgbGl0dGxlIGJpdCBvZiB0ZXh0IGFib3V0IHRoaXMgaW4g WzFdLgo+Cj4KPiBZZWFoLCBJIHVuZGVyc3RhbmQgdGhhdC4gQnV0IHdvdWxkbid0IGl0IGJlIHN1 ZmZpY2llbnQgdG8ganVzdCBkaXNhYmxlCj4gbWVtb3J5IGRlY29kaW5nIGR1cmluZyB0aGUgcmVz aXplPwo+Cj4gSSBjYW4gZWFzaWx5IGd1YXJhbnRlZSB0aGF0IHRoZSBDUFUgaXNuJ3QgYWNjZXNz aW5nIHRoZSBCQVIgZHVyaW5nIHRoZSB0aW1lCj4gKHdlIG5lZWQgdG8gZG8gdGhpcyBmb3IgY2hh bmdpbmcgdGhlIG1lbW9yeSBjbG9ja3MgYXMgd2VsbCksIGJ1dCBJIGhhdmUgYQo+IGJhZCBndXQg ZmVlbGluZyBjb21wbGV0ZWx5IHR1cm5pbmcgb2YgdGhlIGRldmljZSB3aGlsZSB3ZSBhcmUgc3Rp bGwKPiBkaXNwbGF5aW5nIHN0dWZmLgo+Cj4+PiAyLiBXaXRob3V0IHRha2luZyBhIGxvb2sgYXQg dGhlIHJlZ2lzdGVycyB5b3UgZG9uJ3Qga25vdyBob3cgbXVjaAo+Pj4gbWVtb3J5IHRoZXJlIGFj dHVhbGx5IGlzIG9uIHRoZSBib2FyZC4KPj4+Cj4+PiBXZSBjb3VsZCBhbHdheXMgcmVzaXplIGl0 IHRvIHRoZSBtYXhpbXVtIHN1cHBvcnRlZCwgYnV0IHRoYXQgd291bGQKPj4+IG1lYW4gd2UgY291 bGQgZWFzaWx5IHdhc3RlIDEyOEdCIG9mIGFkZHJlc3Mgc3BhY2Ugd2hpbGUgdGhlIGhhcmR3YXJl Cj4+PiBvbmx5IGhhcyA4R0Igb2YgVlJBTS4KPj4+Cj4+PiBUaGF0IHdvdWxkIG5vdCBuZWNlc3Nh cmlseSBodXJ0IHVzIHdoZW4gd2UgaGF2ZSBlbm91Z2ggYWRkcmVzcwo+Pj4gc3BhY2UsIGJ1dCBh dCBsZWFzdCBraW5kIG9mIHN1Y2tzLgo+Pgo+PiBFbmFibGUsIHJlYWQgcmVncywgZGlzYWJsZSwg a2ljayBvdXQgb3RoZXIgZHJpdmVycywgcmVzaXplLCBlbmFibGUuCj4+IERvZXMgdGhhdCBzb2x2 ZSB0aGlzIHByb2JsZW0/Cj4KPgo+IFllYWgsIHRoYXQgc291bmRzIGxpa2UgaXQgc2hvdWxkIGRv IGl0LiBJIHdpbGwgdHJ5IGFuZCB0YWtlIGEgbG9vayBpZiB0aGF0Cj4gd29ya3Mgb3Igbm90Lgo+ Cj4KPj4+PiBJIHdvdWxkIGFsc28gbGlrZSB0byBzaW1wbGlmeSB0aGUgZHJpdmVyIHVzYWdlIG1v ZGVsIGFuZCBnZXQgdGhlCj4+Pj4gUENJX0NPTU1BTkRfTUVNT1JZIHR3aWRkbGluZyBpbnRvIHRo ZSBQQ0kgY29yZSBpbnN0ZWFkIG9mIHRoZSBkcml2ZXIuCj4+Pj4gSWRlYWxseSwgdGhlIGRyaXZl ciB3b3VsZCBkbyBzb21ldGhpbmcgbGlrZSB0aGlzOgo+Pj4+Cj4+Pj4gICAgcGNpX3Jlc2l6ZV9y ZXNvdXJjZShhZGV2LT5wZGV2LCAwLCByYmFyX3NpemUpOwo+Pj4+ICAgIHBjaV9lbmFibGVfZGV2 aWNlKGFkZXYtPnBkZXYpOwo+Pj4+Cj4+Pj4gQW5kIHRoZSBQQ0kgY29yZSB3b3VsZCBiZSBzb21l dGhpbmcgYWxvbmcgdGhlc2UgbGluZXM6Cj4+Pj4KPj4+PiAgICBpbnQgcGNpX3Jlc2l6ZV9yZXNv dXJjZShkZXYsIGJhciwgc2l6ZSkKPj4+PiAgICB7Cj4+Pj4gICAgICBpZiAocGNpX2lzX2VuYWJs ZWQoZGV2KSkKPj4+PiAgICAgICAgcmV0dXJuIC1FQlVTWTsKPj4+Pgo+Pj4+ICAgICAgcGNpX2Rp c2FibGVfZGVjb2RpbmcoZGV2KTsgICAgICAgICAgIyB0dXJuIG9mZiBNRU0sIElPIGRlY29kaW5n Cj4+Pj4gICAgICBwY2lfcmVsZWFzZV9yZXNvdXJjZXMoZGV2KTsgICAgICAgICAjIChhbGwgb2Yg dGhlbSkKPj4+PiAgICAgIGVyciA9IHBjaV9yZXNpemVfYmFyKGRldiwgYmFyLCBzaXplKTsgICAg ICMgY2hhbmdlIEJBUiBzaXplIChvbmx5KQo+Pj4+ICAgICAgaWYgKGVycikKPj4+PiAgICAgICAg cmV0dXJuIGVycjsKPj4+Pgo+Pj4+ICAgICAgcGNpX2Fzc2lnbl9yZXNvdXJjZXMoZGV2KTsgICAg ICAgICAgIyByZWFzc2lnbiBhbGwgImRldiIgcmVzb3VyY2VzCj4+Pj4gICAgICByZXR1cm4gMDsK Pj4+PiAgICB9Cj4+Pgo+Pj4gSSBhbHJlYWR5IHRyaWVkIHRoZSBhcHByb2FjaCB3aXRoIHJlbGVh c2luZyBhbGwgcmVzb3VyY2VzLCBidXQgaXQKPj4+IGRpZG4ndCB3b3JrZWQgc28gd2VsbC4KPj4+ Cj4+PiBXaGVuIHJlc2l6aW5nIGZhaWxzIGJlY2F1c2Ugd2UgZG9uJ3QgaGF2ZSBlbm91Z2ggYWRk cmVzcyBzcGFjZSB0aGVuCj4+PiB3ZSBhdCBsZWFzdCB3YW50IHRvIGdldCBiYWNrIHRvIGEgd29y a2luZyBjb25maWcuCj4+Pgo+Pj4gUmVsZWFzaW5nIGV2ZXJ5dGhpbmcgbWFrZXMgdGhhdCByYXRo ZXIgdHJpY2t5LCBzaW5jZSBJIHdvdWxkIHRoZW4KPj4+IG5lZWQgdG8ga2VlcCBhIGJhY2t1cCBv ZiB0aGUgb2xkIGNvbmZpZyBhbmQgdHJ5IHRvIHJlc3RvcmUgaXQuCj4+Cj4+IElmIHJlc2l6aW5n IGZhaWxzIGJlY2F1c2Ugb2YgbGFjayBvZiBhZGRyZXNzIHNwYWNlLCBJIHdvdWxkIGV4cGVjdCB0 aGUKPj4gUENJIGNvcmUgdG8gYXQgbGVhc3QgcmVzdG9yZSB0byB0aGUgcHJldmlvdXMgc3RhdGUu ICBJZiBpdCBkb2Vzbid0LCBJCj4+IHRoaW5rIHRoYXQgd291bGQgYmUgYSBkZWZlY3QuCj4KPgo+ IEkgY29tcGxldGVseSBhZ3JlZSwgYnV0IHRoaXMgaXMgdW5mb3J0dW5hdGVseSB3aGF0IGhhcHBl bnMuCj4KPiBUaGUgYWxsb2NhdGlvbiBhbmQgYWxpZ25tZW50IGZ1bmN0aW9ucyBpbiB0aGUgUENJ IGNvcmUgc29tZXRpbWVzIGRvZXNuJ3QKPiBzZWVtIHRvIGJlIGFibGUgdG8gcmVzdG9yZSB0aGUg b2xkIGNvbmZpZy4KPgo+IEkgbWVhbiBqdXN0IHRyeSBwY2k9cmVhbGxvYy4gVGhlIGNvcmUgc29t ZXRpbWVzIGRvZXNuJ3Qgc2VlbSB0byBiZSBhYmxlIHRvCj4gY29tZSB1cCB3aXRoIGEgdmFsaWQg Y29uZmlnIHdpdGggdGhhdC4KPgo+PiBIYXZpbmcgdGhlIGRyaXZlciBzcGVjaWZ5IHRoZSBCQVJz IGl0IHRoaW5rcyBtaWdodCBjYXVzZSBpc3N1ZXMgZmVlbHMKPj4gbGlrZSBhIGNydXRjaC4KPgo+ IFdoeT8gVGhlIGRyaXZlciBqdXN0IHJlbGVhc2VzIHdoYXQgaXMgbmVjZXNzYXJ5IHRvIG1vdmUg dGhlIEJBUi4KPgo+Pgo+Pj4gQWRkaXRpb25hbCB0byB0aGF0IEknbSBub3Qgc3VyZSBpZiByZWxl YXNpbmcgdGhlIHJlZ2lzdGVyIEJBUiBhbmQKPj4+IHJlbG9jYXRpbmcgaXQgd29ya3Mgd2l0aCBh bWRncHUuCj4+Cj4+IElmIHRoZSBCQVIgY2FuJ3QgYmUgcmVsb2NhdGVkLCB0aGF0IHNvdW5kcyBs aWtlIGEgaGFyZHdhcmUgZGVmZWN0LiAgSWYKPj4gdGhhdCdzIHJlYWxseSB0aGUgY2FzZSwgeW91 IGNvdWxkIG1hcmsgaXQgSU9SRVNPVVJDRV9QQ0lfRklYRUQgc28gd2UKPj4gZG9uJ3QgbW92ZSBp dC4gIE9yIGlmIGl0J3MgYW4gYW1kZ3B1IGRlZmVjdCwgZS5nLiwgaWYgYW1kZ3B1IGRvZXNuJ3QK Pj4gcmUtcmVhZCB0aGUgcmVzb3VyY2UgYWRkcmVzc2VzIGFmdGVyIHBjaV9lbmFibGVfZGV2aWNl KCksIHlvdSBzaG91bGQKPj4gZml4IGFtZGdwdS4KPgo+Cj4gTm8sIHRoZSBpc3N1ZSBpcyBub3Qg dGhlIGhhcmR3YXJlIGJ1dCByYXRoZXIgdGhlIGZhY3QgdGhhdCB0aGUgSS9PIEJBUiBtaWdodAo+ IGJlIHVzZWQgYnkgbW9yZSB0aGFuIG9uZSBkcml2ZXIgaW4gdGhlIHN5c3RlbS4KPgo+IFdlIGFs c28gaGF2ZSBhbiBhbHNhIGF1ZGlvIGRyaXZlciB1c2luZyB0aGlzIHRvIHByb2dyYW0gaXQncyBE TUEgZm9yIEhETUkvRFAKPiBhdWRpbyBhbmQgdGhlcmUgaXMgYWxzbyB0aGUga2lja2VkIG91dCBk cml2ZXIgd2hpY2ggbWlnaHQgc2hvdWxkIHRha2Ugb3Zlcgo+IGFnYWluIGlmIGFtZGdwdSBldmVy IHVubG9hZHMuCgpJIHRoaW5rIHlvdSdsbCBvbmx5IHNlZSB0aGlzIG9uIGNoaXBzIHdpdGggQUNQ IChDYXJyaXpvIGFuZCBTdG9uZXkpCmFuZCBJIHRoaW5rIHRoZSBzb2MgYWxzYSBkcml2ZXIgZ2V0 cyBsb2FkZWQgdmlhIGEgaG90cGx1ZyBjYWxsIGZyb20KYW1kZ3B1IHdoZW4gaXQgZGV0ZWN0cyBB Q1Agc28gYXMgbG9uZyBhcyB3ZSBkbyB0aGUgcmVzaXplIGJlZm9yZQpwcm9iaW5nIEFDUCB3ZSBz aG91bGQgYmUgZmluZS4gIE9uIFJWLCBBQ1AgbW92ZXMgdG8gaXQncyBvd24gcGNpCmRldmljZSBh bmQgdGhlIEhEQSBhdWRpbyBvbiBvdGhlciBkZXZpY2VzIGlzIGV4cG9zZWQgdmlhIGl0J3Mgb3du IHBjaQpkZXZpY2UuCgpBbGV4CgoKPgo+IEkgbmVlZCB0byBkb3VibGUgY2hlY2sgaG93IHRob3Nl IHR3byB3b3JrLgo+Cj4gUmVnYXJkcywKPiBDaHJpc3RpYW4uCj4KPgo+Pgo+PiBbMV0KPj4gaHR0 cHM6Ly9naXQua2VybmVsLm9yZy9wdWIvc2NtL2xpbnV4L2tlcm5lbC9naXQvdG9ydmFsZHMvbGlu dXguZ2l0L3RyZWUvRG9jdW1lbnRhdGlvbi9QQ0kvcGNpLnR4dCNuMjU1Cj4KPgo+Cj4gX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPiBhbWQtZ2Z4IG1haWxp bmcgbGlzdAo+IGFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCj4gaHR0cHM6Ly9saXN0cy5m cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4Cl9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmFtZC1nZnggbWFpbGluZyBsaXN0CmFtZC1n ZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21h aWxtYW4vbGlzdGluZm8vYW1kLWdmeAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: MIME-Version: 1.0 In-Reply-To: <779a883a-265a-ae04-b0cd-8cb3599f0dc0@vodafone.de> References: <1494348547-1465-1-git-send-email-deathsimple@vodafone.de> <1494348547-1465-6-git-send-email-deathsimple@vodafone.de> <20170601201430.GC12257@bhelgaas-glaptop.roam.corp.google.com> <72e061a5-9684-06e2-3021-80de8ca97bd7@vodafone.de> <20170602202631.GA1452@bhelgaas-glaptop.roam.corp.google.com> <779a883a-265a-ae04-b0cd-8cb3599f0dc0@vodafone.de> From: Alex Deucher Date: Tue, 6 Jun 2017 09:53:34 -0400 Message-ID: Subject: Re: [PATCH v6 5/5] drm/amdgpu: resize VRAM BAR for CPU access v2 To: =?UTF-8?Q?Christian_K=C3=B6nig?= Cc: Bjorn Helgaas , David Airlie , Linux PCI , Maling list - DRI developers , platform-driver-x86@vger.kernel.org, amd-gfx list , Alex Deucher Content-Type: text/plain; charset="UTF-8" List-ID: On Tue, Jun 6, 2017 at 7:51 AM, Christian K=C3=B6nig wrote: > Am 02.06.2017 um 22:26 schrieb Bjorn Helgaas: >> >> On Fri, Jun 02, 2017 at 11:32:21AM +0200, Christian K=C3=B6nig wrote: >>> >>> Hi Bjorn, >>> >>> sorry for not responding earlier and thanks for picking this thread >>> up again. >>> >>> Am 01.06.2017 um 22:14 schrieb Bjorn Helgaas: >>>> >>>> [+cc ADMGPU, DRM folks] >>>> >>>> On Tue, May 09, 2017 at 06:49:07PM +0200, Christian K=C3=B6nig wrote: >>>>> >>>>> [SNIP] >>>>> +/** >>>>> + * amdgpu_resize_bar0 - try to resize BAR0 >>>>> + * >>>>> + * @adev: amdgpu_device pointer >>>>> + * >>>>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>>>> + */ >>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >>>>> +{ >>>>> + u64 space_needed =3D roundup_pow_of_two(adev->mc.real_vram_si= ze); >>>>> + u32 rbar_size =3D order_base_2(((space_needed >> 20) | 1)) -1= ; >>>>> + u16 cmd; >>>>> + int r; >>>>> + >>>>> + /* Free the doorbell mapping, it most likely needs to move as >>>>> well */ >>>>> + amdgpu_doorbell_fini(adev); >>>>> + pci_release_resource(adev->pdev, 2); >>>>> + >>>>> + /* Disable memory decoding while we change the BAR addresses >>>>> and size */ >>>>> + pci_read_config_word(adev->pdev, PCI_COMMAND, &cmd); >>>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, >>>>> + cmd & ~PCI_COMMAND_MEMORY); >>>>> + >>>>> + r =3D pci_resize_resource(adev->pdev, 0, rbar_size); >>>>> + if (r =3D=3D -ENOSPC) >>>>> + DRM_INFO("Not enough PCI address space for a large >>>>> BAR."); >>>>> + else if (r && r !=3D -ENOTSUPP) >>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>>>> + >>>>> + pci_write_config_word(adev->pdev, PCI_COMMAND, cmd); >>>>> + >>>>> + /* When the doorbell BAR isn't available we have no chance of >>>>> + * using the device. >>>>> + */ >>>>> + BUG_ON(amdgpu_doorbell_init(adev)); >>>> >>>> This amdgpu_doorbell_fini()/amdgpu_doorbell_init() thing doesn't look >>>> right. amdgpu_device_init() only calls amdgpu_doorbell_init() for >>>> "adev->asic_type >=3D CHIP_BONAIRE", but we call it unconditionally >>>> here. >>>> >>>> This is the call graph: >>>> >>>> amdgpu_device_init >>>> adev->rmmio_base =3D pci_resource_start(adev->pdev, 5) # 2 for = < >>>> BONAIRE >>>> adev->rmmio =3D ioremap(adev->rmmio_base, ...) >>>> DRM_INFO("register mmio base: 0x%08X\n", >>>> (uint32_t)adev->rmmio_base) >>>> if (adev->asic_type >=3D CHIP_BONAIRE) { >>>> amdgpu_doorbell_init >>>> adev->doorbell.base =3D pci_resource_start(adev->pdev, 2) >>>> adev->doorbell.ptr =3D ioremap(adev->doorbell.base, ...) >>>> } >>>> amdgpu_init >>>> gmc_v7_0_sw_init # gmc_v7_0_ip_funcs.sw_init >>>> gmc_v7_0_mc_init >>>> + amdgpu_resize_bar0 >>>> + amdgpu_doorbell_fini >>>> + pci_release_resource(adev->pdev, 2) >>>> + pci_resize_resource(adev->pdev, 0, size) >>>> + amdgpu_doorbell_init >>>> adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0) >>>> >>>> If "asic_type < CHIP_BONAIRE", we ioremapped BAR 2 in >>>> amdgpu_device_init(), then we released the resource here and never >>>> updated the ioremap. >>> >>> The first hardware with a resizeable BAR I could find is a Tonga, >>> and that is even a generation later than Bonaire. >>> >>> So we are never going to call this code on earlier hardware generations= . >> >> The problem with that is that it's impossible for a code reader to >> verify that. So adding a check is ugly but I think makes it more >> readable. > > > Good point. I will just move the check into the function itself, that sho= uld > handle all such cases. > >>>> From the PCI core perspective, it would be much cleaner to do the BAR >>>> resize before the driver calls pci_enable_device(). If that could be >>>> done, there would be no need for this sort of shutdown/reinit stuff >>>> and we wouldn't have to worry about issues like these. The amdgpu >>>> init path is pretty complicated, so I don't know whether this is >>>> possible. >>> >>> I completely agree on this and it is actually the approach I tried firs= t. >>> >>> There are just two problems with this approach: >>> 1. When the amdgpu driver is loaded there can already be the VGA >>> console, Vesa or EFI driver active for the device and displaying the >>> splash screen. >>> >>> When we resize and most likely relocate the BAR while those drivers >>> are active it will certainly cause problems. >>> >>> What amdgpu does before trying to resize the BAR is kicking out >>> other driver and making sure it has exclusive access to the >>> hardware. >> >> I don't understand the problem here yet. If you need to enable the >> device, then disable it, resize, and re-enable it, that's fine. > > > The issue is we never enable the device ourself in amdgpu, except for som= e > rare cases during resume. > > In most of the cases we have to handle this is the primary display device > which is enabled by either the BIOS, VGA console, VesaFB or EFIFB. Amdgpu > just kicks out whatever driver was responsible for the device previously = and > takes over. > > I could of course do the disable/resize/reenable dance, but I would rathe= r > want to avoid that. > > The hardware is most likely already displaying a boot splash and we want = to > transit to the desktop without any flickering (at least that's the long t= erm > goal). Completely disabling the device to do this doesn't sounds like a g= ood > idea if we want that. > >> The important thing I'm looking for is that the resize happens before >> a pci_enable_device(), because pci_enable_device() is the sync point >> where the PCI core enables resources and makes them available to the >> driver. Drivers know that they can't look at the resources before >> that point. There's a little bit of text about this in [1]. > > > Yeah, I understand that. But wouldn't it be sufficient to just disable > memory decoding during the resize? > > I can easily guarantee that the CPU isn't accessing the BAR during the ti= me > (we need to do this for changing the memory clocks as well), but I have a > bad gut feeling completely turning of the device while we are still > displaying stuff. > >>> 2. Without taking a look at the registers you don't know how much >>> memory there actually is on the board. >>> >>> We could always resize it to the maximum supported, but that would >>> mean we could easily waste 128GB of address space while the hardware >>> only has 8GB of VRAM. >>> >>> That would not necessarily hurt us when we have enough address >>> space, but at least kind of sucks. >> >> Enable, read regs, disable, kick out other drivers, resize, enable. >> Does that solve this problem? > > > Yeah, that sounds like it should do it. I will try and take a look if tha= t > works or not. > > >>>> I would also like to simplify the driver usage model and get the >>>> PCI_COMMAND_MEMORY twiddling into the PCI core instead of the driver. >>>> Ideally, the driver would do something like this: >>>> >>>> pci_resize_resource(adev->pdev, 0, rbar_size); >>>> pci_enable_device(adev->pdev); >>>> >>>> And the PCI core would be something along these lines: >>>> >>>> int pci_resize_resource(dev, bar, size) >>>> { >>>> if (pci_is_enabled(dev)) >>>> return -EBUSY; >>>> >>>> pci_disable_decoding(dev); # turn off MEM, IO decoding >>>> pci_release_resources(dev); # (all of them) >>>> err =3D pci_resize_bar(dev, bar, size); # change BAR size (on= ly) >>>> if (err) >>>> return err; >>>> >>>> pci_assign_resources(dev); # reassign all "dev" resource= s >>>> return 0; >>>> } >>> >>> I already tried the approach with releasing all resources, but it >>> didn't worked so well. >>> >>> When resizing fails because we don't have enough address space then >>> we at least want to get back to a working config. >>> >>> Releasing everything makes that rather tricky, since I would then >>> need to keep a backup of the old config and try to restore it. >> >> If resizing fails because of lack of address space, I would expect the >> PCI core to at least restore to the previous state. If it doesn't, I >> think that would be a defect. > > > I completely agree, but this is unfortunately what happens. > > The allocation and alignment functions in the PCI core sometimes doesn't > seem to be able to restore the old config. > > I mean just try pci=3Drealloc. The core sometimes doesn't seem to be able= to > come up with a valid config with that. > >> Having the driver specify the BARs it thinks might cause issues feels >> like a crutch. > > Why? The driver just releases what is necessary to move the BAR. > >> >>> Additional to that I'm not sure if releasing the register BAR and >>> relocating it works with amdgpu. >> >> If the BAR can't be relocated, that sounds like a hardware defect. If >> that's really the case, you could mark it IORESOURCE_PCI_FIXED so we >> don't move it. Or if it's an amdgpu defect, e.g., if amdgpu doesn't >> re-read the resource addresses after pci_enable_device(), you should >> fix amdgpu. > > > No, the issue is not the hardware but rather the fact that the I/O BAR mi= ght > be used by more than one driver in the system. > > We also have an alsa audio driver using this to program it's DMA for HDMI= /DP > audio and there is also the kicked out driver which might should take ove= r > again if amdgpu ever unloads. I think you'll only see this on chips with ACP (Carrizo and Stoney) and I think the soc alsa driver gets loaded via a hotplug call from amdgpu when it detects ACP so as long as we do the resize before probing ACP we should be fine. On RV, ACP moves to it's own pci device and the HDA audio on other devices is exposed via it's own pci device. Alex > > I need to double check how those two work. > > Regards, > Christian. > > >> >> [1] >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/= Documentation/PCI/pci.txt#n255 > > > > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx