From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32B6AC433E0 for ; Wed, 27 Jan 2021 21:53:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EC00B6146D for ; Wed, 27 Jan 2021 21:53:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235913AbhA0Vxt (ORCPT ); Wed, 27 Jan 2021 16:53:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236211AbhA0Vxl (ORCPT ); Wed, 27 Jan 2021 16:53:41 -0500 Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com [IPv6:2a00:1450:4864:20::231]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 564AEC061573 for ; Wed, 27 Jan 2021 13:53:01 -0800 (PST) Received: by mail-lj1-x231.google.com with SMTP id f2so3896286ljp.11 for ; Wed, 27 Jan 2021 13:53:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=I2V5qL7ef4qnigee9vLt+3og4OBtjgl/PNhaajxHfkY=; b=fp1l0zczuSsluJ5rNV82ImxxiKA1iV7UFIkHHxxlokIwffYPKVYGqGBFu1FBEi8XDA dUPXjRdAbfZvt5tMQTSl1zFktsLeu+gJuwmeq241CLUSi6iAvNpFunl8YNzIG5UyS7RI Fara4J5IgUT27agOZzW8MN0puhQYLOL8X/TISxefKHmLoSgtFw9gQJZRaiPGRyRpYhL3 XvOPfKGOGWNx7gPuGHDGiE6jpRZYul9IoMUnFWq0zd9J/c5SDA9mO5AW3crm0AMmqzvI WO1coAScJMIPUAUWKpdEVUex+CzOb1XL8h5e3E9OFlFHjluKyJR4PUmjQUCjhQBti3MB AV3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=I2V5qL7ef4qnigee9vLt+3og4OBtjgl/PNhaajxHfkY=; b=nfoPR58SlxhJQ9j9aDj6WTkPAtjYqzYe0dzIV/w3IhF+kYEiHZl7AAEK9eKcic95Gd qhUJF/BJDZuL04jXOYph6A0+SqvaalecxfEfF96AtNNr0BjcQYPzDtRni38pH0ndAvqZ BRKepKytzdxaFLE+qqtvfW7hKBe0bXXLVt8uNONEUKBdpVk5srugpZ/IYcxDYZ3O+VKr zy9YbIwmsjOqk1TaOYyUD3XWoMdkqFtQ0s5AfFuVnhI4TspEILMl2Z4AVVSnTybj88sw 1m9xZZw9vg5afDRCFZV4qzyQjqB2xdW8YyyC5Q8qHBh5pl6qrKbbY8oXEWQcENsIOYje 2HaQ== X-Gm-Message-State: AOAM530a9KHEH52AEDCJdp5CNFGDAcHJ69xRyeMyCvNZGNIBwsuf0xBz 2SXKFh6yq2dXguoXhpZriYk= X-Google-Smtp-Source: ABdhPJykpTqIXPlX22xvTJ7uWlP/yBGwsd3B47unBuHTo1XyPLU+9TV1fmfI8EPxij8D0d2qtyX+kA== X-Received: by 2002:a2e:b743:: with SMTP id k3mr6465157ljo.337.1611784379887; Wed, 27 Jan 2021 13:52:59 -0800 (PST) Received: from ?IPv6:2a00:1370:814d:ea25:a10:76ff:fe69:21b6? ([2a00:1370:814d:ea25:a10:76ff:fe69:21b6]) by smtp.googlemail.com with ESMTPSA id g21sm839837lfr.212.2021.01.27.13.52.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 27 Jan 2021 13:52:59 -0800 (PST) Subject: Re: [PATCH v5 00/21] support option where all commands are collected into a single,dedicated cmdstream To: Mikko Perttunen , Mikko Perttunen , thierry.reding@gmail.com, jonathanh@nvidia.com, airlied@linux.ie, daniel@ffwll.ch Cc: linux-tegra@vger.kernel.org, dri-devel@lists.freedesktop.org, talho@nvidia.com, bhuntsman@nvidia.com References: <20210111130019.3515669-1-mperttunen@nvidia.com> <2f999b6d-d781-503a-78f4-d444bce72c58@kapsi.fi> From: Dmitry Osipenko Message-ID: Date: Thu, 28 Jan 2021 00:52:58 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.2 MIME-Version: 1.0 In-Reply-To: <2f999b6d-d781-503a-78f4-d444bce72c58@kapsi.fi> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-tegra@vger.kernel.org 26.01.2021 05:45, Mikko Perttunen пишет: >> 6. We will need to allocate a host1x BO for a job's cmdstream and add a >> restart command to the end of the job's stream.  CDMA will jump into the >> job's stream from push buffer. >> >> We could add a flag for that to drm_tegra_submit_cmd_gather, saying that >> gather should be inlined into job's main cmdstream. >> >> This will remove a need to have a large push buffer that will easily >> overflow, it's a real problem and upstream driver even has a bug where >> it locks up on overflow. >> >> How it will look from CDMA perspective: >> >> PUSHBUF | >> --------- >> ...     |      | JOB   | >>          |      ---------       | JOB GATHER | >> RESTART    ------> CMD    |       -------------- >>          |      |GATHER -------> DATA        | >> ... <---------- RESTART|       |            | >>          |      |       | >> > > Let me check if I understood you correctly: > - You would like to have the job's cmdbuf have further GATHER opcodes > that jump into smaller gathers? I want jobs to be a self-contained. Instead of pushing commands to the PB of a kernel driver, we'll push them to job's cmdstream. This means that for each new job we'll need to allocate a host1x buffer. > I assume this is needed because currently WAITs are placed into the > pushbuffer, so the job will take a lot of space in the pushbuffer if > there are a lot of waits (and GATHERs in between these waits)? Yes, and with drm-sched we will just need to limit the max number of jobs in the h/w queue (i.e. push buffer) and then push buffer won't ever overflow. Problem solved. > If so, perhaps as a simpler alternative we could change the firewall to > allow SETCLASS into HOST1X for waiting specifically, then userspace > could just submit one big cmdbuf taking only little space in the > pushbuffer? Although that would only allow direct ID/threshold waits. My solution doesn't require changes to firewall, not sure whether it's easier. > In any case, it seems that this can be added in a later patch, so we > should omit it from this series for simplicity. If it is impossible for > the userspace to deal with it, we could disable the firewall > temporarily, or implement the above change in the firewall. I won't be able to test UAPI fully until all features are at least on par with the experimental driver of grate kernel. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81D0BC433DB for ; Thu, 28 Jan 2021 08:17:34 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3404D64DA1 for ; Thu, 28 Jan 2021 08:17:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3404D64DA1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 20C5C6E926; Thu, 28 Jan 2021 08:16:48 +0000 (UTC) Received: from mail-lj1-x236.google.com (mail-lj1-x236.google.com [IPv6:2a00:1450:4864:20::236]) by gabe.freedesktop.org (Postfix) with ESMTPS id 904BA89CCB for ; Wed, 27 Jan 2021 21:53:01 +0000 (UTC) Received: by mail-lj1-x236.google.com with SMTP id u4so2390811ljh.6 for ; Wed, 27 Jan 2021 13:53:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=I2V5qL7ef4qnigee9vLt+3og4OBtjgl/PNhaajxHfkY=; b=fp1l0zczuSsluJ5rNV82ImxxiKA1iV7UFIkHHxxlokIwffYPKVYGqGBFu1FBEi8XDA dUPXjRdAbfZvt5tMQTSl1zFktsLeu+gJuwmeq241CLUSi6iAvNpFunl8YNzIG5UyS7RI Fara4J5IgUT27agOZzW8MN0puhQYLOL8X/TISxefKHmLoSgtFw9gQJZRaiPGRyRpYhL3 XvOPfKGOGWNx7gPuGHDGiE6jpRZYul9IoMUnFWq0zd9J/c5SDA9mO5AW3crm0AMmqzvI WO1coAScJMIPUAUWKpdEVUex+CzOb1XL8h5e3E9OFlFHjluKyJR4PUmjQUCjhQBti3MB AV3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=I2V5qL7ef4qnigee9vLt+3og4OBtjgl/PNhaajxHfkY=; b=oZ/9bgz4FReWPKyHDD4NgB9Ph19TQK1Yt2mMsIOMJWfY9xROmZBC/s0I4em/BpbPf/ axA/CGuBsMh3uk45UFi9xdVpB9XDn1bsZl36miMASKL9ucZXgzx/KwoeAwmqywxtmG0D bHkOY1/CGKMHIw2+d0jz7Yo4D5ONvsl95J5v2/9IfQCcUdPk8UqARYGTcnoWTUH1pyA1 xMZRzzSgYodBVbo7MC1dXtp1yquvLW8xrtUDAt8o0CYqcFFuVS08Gs7ug0uRfTzJr/xb BugQW+wAvGVEcs3JQ64G03b9lp4izvOVs2qCA8ma+RpfcRz3c9AdKcEhphLh79J41xTn XElQ== X-Gm-Message-State: AOAM532SZjL960Im7nG4tR2FgyQyVdaEtbs9xtgrvkWtWwQXmjV5Y30V mT3XtiCWZdbwVHR8AGjgyck= X-Google-Smtp-Source: ABdhPJykpTqIXPlX22xvTJ7uWlP/yBGwsd3B47unBuHTo1XyPLU+9TV1fmfI8EPxij8D0d2qtyX+kA== X-Received: by 2002:a2e:b743:: with SMTP id k3mr6465157ljo.337.1611784379887; Wed, 27 Jan 2021 13:52:59 -0800 (PST) Received: from ?IPv6:2a00:1370:814d:ea25:a10:76ff:fe69:21b6? ([2a00:1370:814d:ea25:a10:76ff:fe69:21b6]) by smtp.googlemail.com with ESMTPSA id g21sm839837lfr.212.2021.01.27.13.52.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 27 Jan 2021 13:52:59 -0800 (PST) Subject: Re: [PATCH v5 00/21] support option where all commands are collected into a single,dedicated cmdstream To: Mikko Perttunen , Mikko Perttunen , thierry.reding@gmail.com, jonathanh@nvidia.com, airlied@linux.ie, daniel@ffwll.ch References: <20210111130019.3515669-1-mperttunen@nvidia.com> <2f999b6d-d781-503a-78f4-d444bce72c58@kapsi.fi> From: Dmitry Osipenko Message-ID: Date: Thu, 28 Jan 2021 00:52:58 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.2 MIME-Version: 1.0 In-Reply-To: <2f999b6d-d781-503a-78f4-d444bce72c58@kapsi.fi> Content-Language: en-US X-Mailman-Approved-At: Thu, 28 Jan 2021 08:15:35 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-tegra@vger.kernel.org, talho@nvidia.com, bhuntsman@nvidia.com, dri-devel@lists.freedesktop.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" MjYuMDEuMjAyMSAwNTo0NSwgTWlra28gUGVydHR1bmVuINC/0LjRiNC10YI6Cj4+IDYuIFdlIHdp bGwgbmVlZCB0byBhbGxvY2F0ZSBhIGhvc3QxeCBCTyBmb3IgYSBqb2IncyBjbWRzdHJlYW0gYW5k IGFkZCBhCj4+IHJlc3RhcnQgY29tbWFuZCB0byB0aGUgZW5kIG9mIHRoZSBqb2IncyBzdHJlYW0u wqAgQ0RNQSB3aWxsIGp1bXAgaW50byB0aGUKPj4gam9iJ3Mgc3RyZWFtIGZyb20gcHVzaCBidWZm ZXIuCj4+Cj4+IFdlIGNvdWxkIGFkZCBhIGZsYWcgZm9yIHRoYXQgdG8gZHJtX3RlZ3JhX3N1Ym1p dF9jbWRfZ2F0aGVyLCBzYXlpbmcgdGhhdAo+PiBnYXRoZXIgc2hvdWxkIGJlIGlubGluZWQgaW50 byBqb2IncyBtYWluIGNtZHN0cmVhbS4KPj4KPj4gVGhpcyB3aWxsIHJlbW92ZSBhIG5lZWQgdG8g aGF2ZSBhIGxhcmdlIHB1c2ggYnVmZmVyIHRoYXQgd2lsbCBlYXNpbHkKPj4gb3ZlcmZsb3csIGl0 J3MgYSByZWFsIHByb2JsZW0gYW5kIHVwc3RyZWFtIGRyaXZlciBldmVuIGhhcyBhIGJ1ZyB3aGVy ZQo+PiBpdCBsb2NrcyB1cCBvbiBvdmVyZmxvdy4KPj4KPj4gSG93IGl0IHdpbGwgbG9vayBmcm9t IENETUEgcGVyc3BlY3RpdmU6Cj4+Cj4+IFBVU0hCVUYgfAo+PiAtLS0tLS0tLS0KPj4gLi4uwqDC oMKgwqAgfMKgwqDCoMKgwqAgfCBKT0LCoMKgIHwKPj4gwqDCoMKgwqDCoMKgwqDCoCB8wqDCoMKg wqDCoCAtLS0tLS0tLS3CoMKgwqDCoMKgwqAgfCBKT0IgR0FUSEVSIHwKPj4gUkVTVEFSVMKgwqDC oCAtLS0tLS0+IENNRMKgwqDCoCB8wqDCoMKgwqDCoMKgIC0tLS0tLS0tLS0tLS0tCj4+IMKgwqDC oMKgwqDCoMKgwqAgfMKgwqDCoMKgwqAgfEdBVEhFUiAtLS0tLS0tPiBEQVRBwqDCoMKgwqDCoMKg wqAgfAo+PiAuLi4gPC0tLS0tLS0tLS0gUkVTVEFSVHzCoMKgwqDCoMKgwqAgfMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgfAo+PiDCoMKgwqDCoMKgwqDCoMKgIHzCoMKgwqDCoMKgIHzCoMKgwqDCoMKg wqAgfAo+Pgo+IAo+IExldCBtZSBjaGVjayBpZiBJIHVuZGVyc3Rvb2QgeW91IGNvcnJlY3RseToK PiAtIFlvdSB3b3VsZCBsaWtlIHRvIGhhdmUgdGhlIGpvYidzIGNtZGJ1ZiBoYXZlIGZ1cnRoZXIg R0FUSEVSIG9wY29kZXMKPiB0aGF0IGp1bXAgaW50byBzbWFsbGVyIGdhdGhlcnM/CgpJIHdhbnQg am9icyB0byBiZSBhIHNlbGYtY29udGFpbmVkLiBJbnN0ZWFkIG9mIHB1c2hpbmcgY29tbWFuZHMg dG8gdGhlClBCIG9mIGEga2VybmVsIGRyaXZlciwgd2UnbGwgcHVzaCB0aGVtIHRvIGpvYidzIGNt ZHN0cmVhbS4gVGhpcyBtZWFucwp0aGF0IGZvciBlYWNoIG5ldyBqb2Igd2UnbGwgbmVlZCB0byBh bGxvY2F0ZSBhIGhvc3QxeCBidWZmZXIuCgo+IEkgYXNzdW1lIHRoaXMgaXMgbmVlZGVkIGJlY2F1 c2UgY3VycmVudGx5IFdBSVRzIGFyZSBwbGFjZWQgaW50byB0aGUKPiBwdXNoYnVmZmVyLCBzbyB0 aGUgam9iIHdpbGwgdGFrZSBhIGxvdCBvZiBzcGFjZSBpbiB0aGUgcHVzaGJ1ZmZlciBpZgo+IHRo ZXJlIGFyZSBhIGxvdCBvZiB3YWl0cyAoYW5kIEdBVEhFUnMgaW4gYmV0d2VlbiB0aGVzZSB3YWl0 cyk/CgpZZXMsIGFuZCB3aXRoIGRybS1zY2hlZCB3ZSB3aWxsIGp1c3QgbmVlZCB0byBsaW1pdCB0 aGUgbWF4IG51bWJlciBvZgpqb2JzIGluIHRoZSBoL3cgcXVldWUgKGkuZS4gcHVzaCBidWZmZXIp IGFuZCB0aGVuIHB1c2ggYnVmZmVyIHdvbid0IGV2ZXIKb3ZlcmZsb3cuIFByb2JsZW0gc29sdmVk LgoKPiBJZiBzbywgcGVyaGFwcyBhcyBhIHNpbXBsZXIgYWx0ZXJuYXRpdmUgd2UgY291bGQgY2hh bmdlIHRoZSBmaXJld2FsbCB0bwo+IGFsbG93IFNFVENMQVNTIGludG8gSE9TVDFYIGZvciB3YWl0 aW5nIHNwZWNpZmljYWxseSwgdGhlbiB1c2Vyc3BhY2UKPiBjb3VsZCBqdXN0IHN1Ym1pdCBvbmUg YmlnIGNtZGJ1ZiB0YWtpbmcgb25seSBsaXR0bGUgc3BhY2UgaW4gdGhlCj4gcHVzaGJ1ZmZlcj8g QWx0aG91Z2ggdGhhdCB3b3VsZCBvbmx5IGFsbG93IGRpcmVjdCBJRC90aHJlc2hvbGQgd2FpdHMu CgpNeSBzb2x1dGlvbiBkb2Vzbid0IHJlcXVpcmUgY2hhbmdlcyB0byBmaXJld2FsbCwgbm90IHN1 cmUgd2hldGhlciBpdCdzCmVhc2llci4KCj4gSW4gYW55IGNhc2UsIGl0IHNlZW1zIHRoYXQgdGhp cyBjYW4gYmUgYWRkZWQgaW4gYSBsYXRlciBwYXRjaCwgc28gd2UKPiBzaG91bGQgb21pdCBpdCBm cm9tIHRoaXMgc2VyaWVzIGZvciBzaW1wbGljaXR5LiBJZiBpdCBpcyBpbXBvc3NpYmxlIGZvcgo+ IHRoZSB1c2Vyc3BhY2UgdG8gZGVhbCB3aXRoIGl0LCB3ZSBjb3VsZCBkaXNhYmxlIHRoZSBmaXJl d2FsbAo+IHRlbXBvcmFyaWx5LCBvciBpbXBsZW1lbnQgdGhlIGFib3ZlIGNoYW5nZSBpbiB0aGUg ZmlyZXdhbGwuCgpJIHdvbid0IGJlIGFibGUgdG8gdGVzdCBVQVBJIGZ1bGx5IHVudGlsIGFsbCBm ZWF0dXJlcyBhcmUgYXQgbGVhc3Qgb24KcGFyIHdpdGggdGhlIGV4cGVyaW1lbnRhbCBkcml2ZXIg b2YgZ3JhdGUga2VybmVsLgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXwpkcmktZGV2ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3Rv cC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9kcmkt ZGV2ZWwK