From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5550EC2D0CF for ; Mon, 16 Dec 2019 08:22:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 13E862072B for ; Mon, 16 Dec 2019 08:22:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="duCNJEib" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726949AbfLPIWB (ORCPT ); Mon, 16 Dec 2019 03:22:01 -0500 Received: from mail-ot1-f65.google.com ([209.85.210.65]:46623 "EHLO mail-ot1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726712AbfLPIWA (ORCPT ); Mon, 16 Dec 2019 03:22:00 -0500 Received: by mail-ot1-f65.google.com with SMTP id g18so8079509otj.13; Mon, 16 Dec 2019 00:21:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=B2zr+SxGgQJRKPjCI0ZlCNLHKl8Hq+JbhUCM9tDaYbk=; b=duCNJEibi83CQtLO3lJy8ZeTakoP8uvN3rYY/wkMtOGdcEAoS3hikmezHKWm0CA5Pm 9dL0Oz8Qoh2fSfXuethC2UUonF9dhfVvzIhos/7OF5APUXaGD+ozKlWRvy5oN71OKV97 O9chmuVrwmzS6M/f8BQsn6RDt+beAkjXHGQHMEqlTRiRl7F4YpovwtXu3z01fBv9t/JU m1RwuykieOwvhYNmnGjevq6+CmQSVn9QW96K+HLJbb3UkIfaKwPx/jivTNNGnMj89B0I oZ7BHnvjJJS4ZrWzDkrsyxIX+gEODf8o+n/a9w5+NEi4szuQ5RjtJHNummuyoNBNNIDb /spw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=B2zr+SxGgQJRKPjCI0ZlCNLHKl8Hq+JbhUCM9tDaYbk=; b=KGnonN9UZE3RiRnYo67kXkumhqBtpQ2tOElH3qEcFYfHIy09sRoTtQJbZlk76Ponor 78DavZiQ1uk12M/siczGyySH/V9LXSsV+2MGMeWHjHVX5mMzeYK+77JGGWXifzZObD3o ZaTYA1mFh2d+g3ds2nOY3ebH0AADElBKoV2b+QHzHAjru0tkZkXyJ3krtBTfhUQHd/5k CmJZNFJLyUJcKLMuv5C8mEb5Gi2RR4JUVXE8UUmIFV2/K5K1EbwPmWxMYBQHvV6ky7l+ mjUOpwmJciJkHEpG0yhpX1cb23dSOkSHKvwCTSU07pqzQZJ0ZKspLeLj2Ev/khA4Y9zx gHxQ== X-Gm-Message-State: APjAAAXJHXQNc4kNfis/AjL5nNYNYRe1NdoYKeMuoSRSnnit8REyJib4 WZmU8QkGQnwX6ilb4aGLJskcMCIQQo25s87TC20= X-Google-Smtp-Source: APXvYqxQ2QA/IJOqMGi2skPvLoXCAHWfqvnNsrhuIWnq8ZJXE69aNr5mcARZx/rneXf34MbVxtEMNdjkpDEu0iV67GE= X-Received: by 2002:a05:6830:147:: with SMTP id j7mr14990640otp.44.1576484518648; Mon, 16 Dec 2019 00:21:58 -0800 (PST) MIME-Version: 1.0 References: <20191213084748.11210-3-prabhakar.mahadev-lad.rj@bp.renesas.com> <20191213195727.GA170874@google.com> In-Reply-To: <20191213195727.GA170874@google.com> From: "Lad, Prabhakar" Date: Mon, 16 Dec 2019 08:21:31 +0000 Message-ID: Subject: Re: [v2 2/6] pci: endpoint: add support to handle features of outbound memory To: Bjorn Helgaas Cc: Rob Herring , Mark Rutland , Geert Uytterhoeven , Magnus Damm , Kishon Vijay Abraham I , Marek Vasut , Yoshihiro Shimoda , linux-pci , Catalin Marinas , Will Deacon , Lorenzo Pieralisi , Arnd Bergmann , Greg Kroah-Hartman , Andrew Murray , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , LKML , LAK , Linux-Renesas , Chris Paterson , Frank Rowand , Gustavo Pimentel , Jingoo Han , Simon Horman , Shawn Lin , Tom Joseph , Heiko Stuebner , linux-rockchip@lists.infradead.org, "Lad, Prabhakar" Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Bjorn, thank you for the review. On Fri, Dec 13, 2019 at 9:06 PM Bjorn Helgaas wrote: > > On Fri, Dec 13, 2019 at 08:47:44AM +0000, Lad Prabhakar wrote: > > From: "Lad, Prabhakar" > > > > rcar pcie controller has support to map multiple memory regions > > for mapping the outbound memory in local system, this feature > > inspires to add support for handling such features in endpoint > > framework. similar features exists on other controllers where > > outbound regions can be specifically used for low/high priority > > transactions, and regions can be flagged and used for allocation > > of large/small memory allocations. > > This patch adds support to handle such features, where the > > properties described for outbound regions are used whenever a > > request to memory is made. > > For this and the other patches, please: > > - start sentences with a capital letter > - leave a blank line between paragraphs > - wrap commit log text to use the whole 80 character line (I wrap to > 75 characters to account for "git log" indenting by 4 spaces) > - check your signed-off-by: it shows your name as "Lad, Prabhakar", > while your email From: line shows "Lad Prabhakar". Choose one :) > sure will fix it next version. > > diff --git a/drivers/pci/endpoint/pci-epc-mem.c b/drivers/pci/endpoint/pci-epc-mem.c > > index 2bf8bd1..4b610cd 100644 > > --- a/drivers/pci/endpoint/pci-epc-mem.c > > +++ b/drivers/pci/endpoint/pci-epc-mem.c > > > -int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, > > - size_t page_size) > > +int __pci_epc_mem_init(struct pci_epc *epc, struct pci_epc_mem_window *windows, > > + int num_windows, size_t page_size) > > { > > - int ret; > > - struct pci_epc_mem *mem; > > - unsigned long *bitmap; > > + struct pci_epc_mem *mem = NULL; > > + unsigned long *bitmap = NULL; > > unsigned int page_shift; > > - int pages; > > int bitmap_size; > > + int pages; > > + int ret; > > + int i; > > + > > + epc->mem_windows = 0; > > + > > + if (!windows) > > + return -EINVAL; > > + > > + if (num_windows <= 0) > > + return -EINVAL; > > Why is num_windows signed? > > > void pci_epc_mem_exit(struct pci_epc *epc) > > { > > - struct pci_epc_mem *mem = epc->mem; > > + struct pci_epc_mem *mem; > > + int i; > > + > > + if (!epc->mem_windows) > > + return; > > If you fix the loop below, why do you even need to test this? > yes makes sense will drop this check. > > + for (i = 0; i <= epc->mem_windows; i--) { > > Huh? "<="? "i--"? Surely you mean > > for (i = 0; i < epc->mem_windows; i++) { > oops my bad, will fix it. > > + mem = epc->mem[i]; > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > epc->mem = NULL; > > - kfree(mem->bitmap); > > - kfree(mem); > > + epc->mem_windows = 0; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_exit); > > > > +static int pci_epc_find_best_fit_window(struct pci_epc *epc, size_t size, > > + u32 flags) > > Can this just return a struct pci_epc_mem *, so the caller doesn't > have to lookup epc->mem[i] again? > yes makes sense will change it to return struct pci_epc_mem * > > +{ > > + size_t window_least_size = 0; > > + int best_fit_window = -1; > > + struct pci_epc_mem *mem; > > + size_t actual_size; > > + size_t avail_size; > > + u32 win_flags; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + win_flags = mem->window.flags; > > + > > + actual_size = ALIGN(size, mem->page_size); > > + avail_size = mem->window.size - mem->window.map_size; > > + > > + if (win_flags == 0x0) { > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } else { > > + if (mem->window.map_size && > > + (win_flags | PCI_EPC_WINDOW_FLAG_NON_MULTI_ALLOC)) > > + continue; > > + > > + if (!(win_flags | flags)) > > + continue; > > + > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } > > + } > > + > > + return best_fit_window; > > +} > > + > > /** > > * pci_epc_mem_alloc_addr() - allocate memory address from EPC addr space > > * @epc: the EPC device on which memory has to be allocated > > * @phys_addr: populate the allocated physical address here > > + * @window: populate the window here which will be used to map PCI address > > * @size: the size of the address space that has to be allocated > > + * @flags: look for window as requested in flags > > * > > * Invoke to allocate memory address from the EPC address space. This > > * is usually done to map the remote RC address into the local system. > > */ > > void __iomem *pci_epc_mem_alloc_addr(struct pci_epc *epc, > > - phys_addr_t *phys_addr, size_t size) > > + phys_addr_t *phys_addr, > > + int *window, size_t size, uint32_t flags) > > { > > + int best_fit = PCI_EPC_DEFAULT_WINDOW; > > + void __iomem *virt_addr = NULL; > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > int pageno; > > - void __iomem *virt_addr; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows <= 0) > > + return NULL; > > + > > + if (epc->mem_windows > 1) { > > Why bother testing epc->mem_windows here? Just make sure > pci_epc_find_best_fit_window() returns the correct thing for > "mem_windows == 0" and "mem_windows == 1", and remove both the tests > above. > will fix that. > > + best_fit = pci_epc_find_best_fit_window(epc, size, flags); > > + if (best_fit < 0) > > + return NULL; > > + } > > + > > + mem = epc->mem[best_fit]; > > size = ALIGN(size, mem->page_size); > > + if (size > (mem->window.size - mem->window.map_size)) > > + return NULL; > > + page_shift = ilog2(mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > > > pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order); > > if (pageno < 0) > > return NULL; > > > > - *phys_addr = mem->phys_base + (pageno << page_shift); > > + *phys_addr = mem->window.phys_base + (pageno << page_shift); > > virt_addr = ioremap(*phys_addr, size); > > - if (!virt_addr) > > + if (!virt_addr) { > > bitmap_release_region(mem->bitmap, pageno, order); > > + } else { > > + mem->window.map_size += size; > > + *window = best_fit; > > + } > > > > return virt_addr; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > > > +static int pci_epc_get_matching_window(struct pci_epc *epc, > > + phys_addr_t phys_addr) > > Return struct pci_epc_mem * again? > yes makes sense. > > +{ > > + struct pci_epc_mem *mem; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + if (mem->window.phys_base == phys_addr) > > + return i; > > + } > > + > > + return -EINVAL; > > +} > > + > > /** > > * pci_epc_mem_free_addr() - free the allocated memory address > > * @epc: the EPC device on which memory was allocated > > @@ -155,16 +281,26 @@ EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, > > void __iomem *virt_addr, size_t size) > > { > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > + int window = 0; > > int pageno; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows > 1) { > > Same here (unnecessary test). > will drop it. > > + window = pci_epc_get_matching_window(epc, phys_addr); > > + if (window < 0) > > + return; > > + } > > + > > + mem = epc->mem[window]; > > + page_shift = ilog2(mem->page_size); > > iounmap(virt_addr); > > - pageno = (phys_addr - mem->phys_base) >> page_shift; > > + pageno = (phys_addr - mem->window.phys_base) >> page_shift; > > size = ALIGN(size, mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > bitmap_release_region(mem->bitmap, pageno, order); > > + mem->window.map_size -= size; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr); > > > @@ -85,7 +126,8 @@ struct pci_epc_mem { > > * @dev: PCI EPC device > > * @pci_epf: list of endpoint functions present in this EPC device > > * @ops: function pointers for performing endpoint operations > > - * @mem: address space of the endpoint controller > > + * @mem: array of address space of the endpoint controller > > + * @mem_windows: number of windows supported by device > > * @max_functions: max number of functions that can be configured in this EPC > > * @group: configfs group representing the PCI EPC device > > * @lock: spinlock to protect pci_epc ops > > @@ -94,7 +136,8 @@ struct pci_epc { > > struct device dev; > > struct list_head pci_epf; > > const struct pci_epc_ops *ops; > > - struct pci_epc_mem *mem; > > + struct pci_epc_mem **mem; > > + int mem_windows; > > Can't this be unsigned int and then there's no need to check > "mem_windows < 0"? > yes will change it unsigned int. Cheers, --Prabhakar > > u8 max_functions; > > struct config_group *group; > > /* spinlock to protect against concurrent access of EP controller */ From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Lad, Prabhakar" Subject: Re: [v2 2/6] pci: endpoint: add support to handle features of outbound memory Date: Mon, 16 Dec 2019 08:21:31 +0000 Message-ID: References: <20191213084748.11210-3-prabhakar.mahadev-lad.rj@bp.renesas.com> <20191213195727.GA170874@google.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: In-Reply-To: <20191213195727.GA170874@google.com> Sender: linux-kernel-owner@vger.kernel.org To: Bjorn Helgaas Cc: Rob Herring , Mark Rutland , Geert Uytterhoeven , Magnus Damm , Kishon Vijay Abraham I , Marek Vasut , Yoshihiro Shimoda , linux-pci , Catalin Marinas , Will Deacon , Lorenzo Pieralisi , Arnd Bergmann , Greg Kroah-Hartman , Andrew Murray , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , LKML , LAK List-Id: linux-rockchip.vger.kernel.org Hi Bjorn, thank you for the review. On Fri, Dec 13, 2019 at 9:06 PM Bjorn Helgaas wrote: > > On Fri, Dec 13, 2019 at 08:47:44AM +0000, Lad Prabhakar wrote: > > From: "Lad, Prabhakar" > > > > rcar pcie controller has support to map multiple memory regions > > for mapping the outbound memory in local system, this feature > > inspires to add support for handling such features in endpoint > > framework. similar features exists on other controllers where > > outbound regions can be specifically used for low/high priority > > transactions, and regions can be flagged and used for allocation > > of large/small memory allocations. > > This patch adds support to handle such features, where the > > properties described for outbound regions are used whenever a > > request to memory is made. > > For this and the other patches, please: > > - start sentences with a capital letter > - leave a blank line between paragraphs > - wrap commit log text to use the whole 80 character line (I wrap to > 75 characters to account for "git log" indenting by 4 spaces) > - check your signed-off-by: it shows your name as "Lad, Prabhakar", > while your email From: line shows "Lad Prabhakar". Choose one :) > sure will fix it next version. > > diff --git a/drivers/pci/endpoint/pci-epc-mem.c b/drivers/pci/endpoint/pci-epc-mem.c > > index 2bf8bd1..4b610cd 100644 > > --- a/drivers/pci/endpoint/pci-epc-mem.c > > +++ b/drivers/pci/endpoint/pci-epc-mem.c > > > -int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, > > - size_t page_size) > > +int __pci_epc_mem_init(struct pci_epc *epc, struct pci_epc_mem_window *windows, > > + int num_windows, size_t page_size) > > { > > - int ret; > > - struct pci_epc_mem *mem; > > - unsigned long *bitmap; > > + struct pci_epc_mem *mem = NULL; > > + unsigned long *bitmap = NULL; > > unsigned int page_shift; > > - int pages; > > int bitmap_size; > > + int pages; > > + int ret; > > + int i; > > + > > + epc->mem_windows = 0; > > + > > + if (!windows) > > + return -EINVAL; > > + > > + if (num_windows <= 0) > > + return -EINVAL; > > Why is num_windows signed? > > > void pci_epc_mem_exit(struct pci_epc *epc) > > { > > - struct pci_epc_mem *mem = epc->mem; > > + struct pci_epc_mem *mem; > > + int i; > > + > > + if (!epc->mem_windows) > > + return; > > If you fix the loop below, why do you even need to test this? > yes makes sense will drop this check. > > + for (i = 0; i <= epc->mem_windows; i--) { > > Huh? "<="? "i--"? Surely you mean > > for (i = 0; i < epc->mem_windows; i++) { > oops my bad, will fix it. > > + mem = epc->mem[i]; > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > epc->mem = NULL; > > - kfree(mem->bitmap); > > - kfree(mem); > > + epc->mem_windows = 0; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_exit); > > > > +static int pci_epc_find_best_fit_window(struct pci_epc *epc, size_t size, > > + u32 flags) > > Can this just return a struct pci_epc_mem *, so the caller doesn't > have to lookup epc->mem[i] again? > yes makes sense will change it to return struct pci_epc_mem * > > +{ > > + size_t window_least_size = 0; > > + int best_fit_window = -1; > > + struct pci_epc_mem *mem; > > + size_t actual_size; > > + size_t avail_size; > > + u32 win_flags; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + win_flags = mem->window.flags; > > + > > + actual_size = ALIGN(size, mem->page_size); > > + avail_size = mem->window.size - mem->window.map_size; > > + > > + if (win_flags == 0x0) { > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } else { > > + if (mem->window.map_size && > > + (win_flags | PCI_EPC_WINDOW_FLAG_NON_MULTI_ALLOC)) > > + continue; > > + > > + if (!(win_flags | flags)) > > + continue; > > + > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } > > + } > > + > > + return best_fit_window; > > +} > > + > > /** > > * pci_epc_mem_alloc_addr() - allocate memory address from EPC addr space > > * @epc: the EPC device on which memory has to be allocated > > * @phys_addr: populate the allocated physical address here > > + * @window: populate the window here which will be used to map PCI address > > * @size: the size of the address space that has to be allocated > > + * @flags: look for window as requested in flags > > * > > * Invoke to allocate memory address from the EPC address space. This > > * is usually done to map the remote RC address into the local system. > > */ > > void __iomem *pci_epc_mem_alloc_addr(struct pci_epc *epc, > > - phys_addr_t *phys_addr, size_t size) > > + phys_addr_t *phys_addr, > > + int *window, size_t size, uint32_t flags) > > { > > + int best_fit = PCI_EPC_DEFAULT_WINDOW; > > + void __iomem *virt_addr = NULL; > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > int pageno; > > - void __iomem *virt_addr; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows <= 0) > > + return NULL; > > + > > + if (epc->mem_windows > 1) { > > Why bother testing epc->mem_windows here? Just make sure > pci_epc_find_best_fit_window() returns the correct thing for > "mem_windows == 0" and "mem_windows == 1", and remove both the tests > above. > will fix that. > > + best_fit = pci_epc_find_best_fit_window(epc, size, flags); > > + if (best_fit < 0) > > + return NULL; > > + } > > + > > + mem = epc->mem[best_fit]; > > size = ALIGN(size, mem->page_size); > > + if (size > (mem->window.size - mem->window.map_size)) > > + return NULL; > > + page_shift = ilog2(mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > > > pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order); > > if (pageno < 0) > > return NULL; > > > > - *phys_addr = mem->phys_base + (pageno << page_shift); > > + *phys_addr = mem->window.phys_base + (pageno << page_shift); > > virt_addr = ioremap(*phys_addr, size); > > - if (!virt_addr) > > + if (!virt_addr) { > > bitmap_release_region(mem->bitmap, pageno, order); > > + } else { > > + mem->window.map_size += size; > > + *window = best_fit; > > + } > > > > return virt_addr; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > > > +static int pci_epc_get_matching_window(struct pci_epc *epc, > > + phys_addr_t phys_addr) > > Return struct pci_epc_mem * again? > yes makes sense. > > +{ > > + struct pci_epc_mem *mem; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + if (mem->window.phys_base == phys_addr) > > + return i; > > + } > > + > > + return -EINVAL; > > +} > > + > > /** > > * pci_epc_mem_free_addr() - free the allocated memory address > > * @epc: the EPC device on which memory was allocated > > @@ -155,16 +281,26 @@ EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, > > void __iomem *virt_addr, size_t size) > > { > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > + int window = 0; > > int pageno; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows > 1) { > > Same here (unnecessary test). > will drop it. > > + window = pci_epc_get_matching_window(epc, phys_addr); > > + if (window < 0) > > + return; > > + } > > + > > + mem = epc->mem[window]; > > + page_shift = ilog2(mem->page_size); > > iounmap(virt_addr); > > - pageno = (phys_addr - mem->phys_base) >> page_shift; > > + pageno = (phys_addr - mem->window.phys_base) >> page_shift; > > size = ALIGN(size, mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > bitmap_release_region(mem->bitmap, pageno, order); > > + mem->window.map_size -= size; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr); > > > @@ -85,7 +126,8 @@ struct pci_epc_mem { > > * @dev: PCI EPC device > > * @pci_epf: list of endpoint functions present in this EPC device > > * @ops: function pointers for performing endpoint operations > > - * @mem: address space of the endpoint controller > > + * @mem: array of address space of the endpoint controller > > + * @mem_windows: number of windows supported by device > > * @max_functions: max number of functions that can be configured in this EPC > > * @group: configfs group representing the PCI EPC device > > * @lock: spinlock to protect pci_epc ops > > @@ -94,7 +136,8 @@ struct pci_epc { > > struct device dev; > > struct list_head pci_epf; > > const struct pci_epc_ops *ops; > > - struct pci_epc_mem *mem; > > + struct pci_epc_mem **mem; > > + int mem_windows; > > Can't this be unsigned int and then there's no need to check > "mem_windows < 0"? > yes will change it unsigned int. Cheers, --Prabhakar > > u8 max_functions; > > struct config_group *group; > > /* spinlock to protect against concurrent access of EP controller */ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56135C43603 for ; Mon, 16 Dec 2019 08:22:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 00588206D3 for ; Mon, 16 Dec 2019 08:22:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Tl8z6q1f"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="duCNJEib" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 00588206D3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8n8RjRDtWqAbPSKTIqwv93it1hQGTlYcNCk1lexYg1I=; b=Tl8z6q1fLjUJKy f8RAnQipMTxIRHM+6wPiJny195RE7eSUR4duF9l08IQVyF2gfP9+YWz3B85Lb6SGzAnfPK/XRq7Dx 9xixF/fXM8rcb6zfRN/PqeQTqx5qLxhRx5iyeyZyibV2bOJbgN8mrc++IxbkVmUpNSKdQYhwcvv3F pUpGHiRzXauWvRBXhUSIDoJDr0dI9IVklMs2Oqij4MeZ1BbWz84mHPcAVfl7mouRqtl0pmAo6Ue0E +2DzTbe2TbSVlusg5uINMSHoXX3cq2vsT1dcuGUOH+khXtbR17CsbXAnGq3mRCRVgbjbIEDA+Tnh5 1rBVz9TDnuyDJ6zi7aSw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1igldi-0002Ko-Uq; Mon, 16 Dec 2019 08:22:02 +0000 Received: from mail-ot1-x343.google.com ([2607:f8b0:4864:20::343]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1igldf-0002KO-FP; Mon, 16 Dec 2019 08:22:01 +0000 Received: by mail-ot1-x343.google.com with SMTP id k14so8148512otn.4; Mon, 16 Dec 2019 00:21:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=B2zr+SxGgQJRKPjCI0ZlCNLHKl8Hq+JbhUCM9tDaYbk=; b=duCNJEibi83CQtLO3lJy8ZeTakoP8uvN3rYY/wkMtOGdcEAoS3hikmezHKWm0CA5Pm 9dL0Oz8Qoh2fSfXuethC2UUonF9dhfVvzIhos/7OF5APUXaGD+ozKlWRvy5oN71OKV97 O9chmuVrwmzS6M/f8BQsn6RDt+beAkjXHGQHMEqlTRiRl7F4YpovwtXu3z01fBv9t/JU m1RwuykieOwvhYNmnGjevq6+CmQSVn9QW96K+HLJbb3UkIfaKwPx/jivTNNGnMj89B0I oZ7BHnvjJJS4ZrWzDkrsyxIX+gEODf8o+n/a9w5+NEi4szuQ5RjtJHNummuyoNBNNIDb /spw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=B2zr+SxGgQJRKPjCI0ZlCNLHKl8Hq+JbhUCM9tDaYbk=; b=eX9xLl4kCNNsNwtyKN4cZyw+izM7LSsxSfZUDA+Zq+K/9CBQZXz2gEskC2CtBGICG6 HQFoRWDSG6Fd5uPPffL/Z9sD9w9eamK8dqMrN5xYlAj7tpOtLabpOrh2aNeDwY/yg2iz n0halDxlf0dKxi+neN4vi0wLoJzDW/AHX5nCc4ePRQEqb4iu8IA15nmzqaNZfabcK3Qh dtrhKT3i9Ejkm1t/UAwoWvxzwnliY0xW9c6EYI7PoVMBSgWFUtk40b04Eahw3PdS2fsC uxAnbahQ21FzWa4JG/ZYgVr6x2cr/MI+RBRSWjNQcy2kASIQNX8IcqKGwb2VklCrzGrd 42iA== X-Gm-Message-State: APjAAAVxkawk5WPawDp4w1k0ntS+sr1NSBiPOFe/+H8F9vIYkqhixpYQ wgNvyZoEQHkDiuOJGk1J8LzRC1A7OW8V67hvyXc= X-Google-Smtp-Source: APXvYqxQ2QA/IJOqMGi2skPvLoXCAHWfqvnNsrhuIWnq8ZJXE69aNr5mcARZx/rneXf34MbVxtEMNdjkpDEu0iV67GE= X-Received: by 2002:a05:6830:147:: with SMTP id j7mr14990640otp.44.1576484518648; Mon, 16 Dec 2019 00:21:58 -0800 (PST) MIME-Version: 1.0 References: <20191213084748.11210-3-prabhakar.mahadev-lad.rj@bp.renesas.com> <20191213195727.GA170874@google.com> In-Reply-To: <20191213195727.GA170874@google.com> From: "Lad, Prabhakar" Date: Mon, 16 Dec 2019 08:21:31 +0000 Message-ID: Subject: Re: [v2 2/6] pci: endpoint: add support to handle features of outbound memory To: Bjorn Helgaas X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191216_002159_544932_C0493172 X-CRM114-Status: GOOD ( 32.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Heiko Stuebner , Geert Uytterhoeven , linux-pci , Shawn Lin , Frank Rowand , Marek Vasut , Lorenzo Pieralisi , Will Deacon , Magnus Damm , Kishon Vijay Abraham I , linux-rockchip@lists.infradead.org, Catalin Marinas , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , Chris Paterson , Arnd Bergmann , "Lad, Prabhakar" , Rob Herring , LAK , Greg Kroah-Hartman , Yoshihiro Shimoda , LKML , Linux-Renesas , Tom Joseph , Simon Horman , Jingoo Han , Andrew Murray , Gustavo Pimentel Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Bjorn, thank you for the review. On Fri, Dec 13, 2019 at 9:06 PM Bjorn Helgaas wrote: > > On Fri, Dec 13, 2019 at 08:47:44AM +0000, Lad Prabhakar wrote: > > From: "Lad, Prabhakar" > > > > rcar pcie controller has support to map multiple memory regions > > for mapping the outbound memory in local system, this feature > > inspires to add support for handling such features in endpoint > > framework. similar features exists on other controllers where > > outbound regions can be specifically used for low/high priority > > transactions, and regions can be flagged and used for allocation > > of large/small memory allocations. > > This patch adds support to handle such features, where the > > properties described for outbound regions are used whenever a > > request to memory is made. > > For this and the other patches, please: > > - start sentences with a capital letter > - leave a blank line between paragraphs > - wrap commit log text to use the whole 80 character line (I wrap to > 75 characters to account for "git log" indenting by 4 spaces) > - check your signed-off-by: it shows your name as "Lad, Prabhakar", > while your email From: line shows "Lad Prabhakar". Choose one :) > sure will fix it next version. > > diff --git a/drivers/pci/endpoint/pci-epc-mem.c b/drivers/pci/endpoint/pci-epc-mem.c > > index 2bf8bd1..4b610cd 100644 > > --- a/drivers/pci/endpoint/pci-epc-mem.c > > +++ b/drivers/pci/endpoint/pci-epc-mem.c > > > -int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, > > - size_t page_size) > > +int __pci_epc_mem_init(struct pci_epc *epc, struct pci_epc_mem_window *windows, > > + int num_windows, size_t page_size) > > { > > - int ret; > > - struct pci_epc_mem *mem; > > - unsigned long *bitmap; > > + struct pci_epc_mem *mem = NULL; > > + unsigned long *bitmap = NULL; > > unsigned int page_shift; > > - int pages; > > int bitmap_size; > > + int pages; > > + int ret; > > + int i; > > + > > + epc->mem_windows = 0; > > + > > + if (!windows) > > + return -EINVAL; > > + > > + if (num_windows <= 0) > > + return -EINVAL; > > Why is num_windows signed? > > > void pci_epc_mem_exit(struct pci_epc *epc) > > { > > - struct pci_epc_mem *mem = epc->mem; > > + struct pci_epc_mem *mem; > > + int i; > > + > > + if (!epc->mem_windows) > > + return; > > If you fix the loop below, why do you even need to test this? > yes makes sense will drop this check. > > + for (i = 0; i <= epc->mem_windows; i--) { > > Huh? "<="? "i--"? Surely you mean > > for (i = 0; i < epc->mem_windows; i++) { > oops my bad, will fix it. > > + mem = epc->mem[i]; > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > epc->mem = NULL; > > - kfree(mem->bitmap); > > - kfree(mem); > > + epc->mem_windows = 0; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_exit); > > > > +static int pci_epc_find_best_fit_window(struct pci_epc *epc, size_t size, > > + u32 flags) > > Can this just return a struct pci_epc_mem *, so the caller doesn't > have to lookup epc->mem[i] again? > yes makes sense will change it to return struct pci_epc_mem * > > +{ > > + size_t window_least_size = 0; > > + int best_fit_window = -1; > > + struct pci_epc_mem *mem; > > + size_t actual_size; > > + size_t avail_size; > > + u32 win_flags; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + win_flags = mem->window.flags; > > + > > + actual_size = ALIGN(size, mem->page_size); > > + avail_size = mem->window.size - mem->window.map_size; > > + > > + if (win_flags == 0x0) { > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } else { > > + if (mem->window.map_size && > > + (win_flags | PCI_EPC_WINDOW_FLAG_NON_MULTI_ALLOC)) > > + continue; > > + > > + if (!(win_flags | flags)) > > + continue; > > + > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } > > + } > > + > > + return best_fit_window; > > +} > > + > > /** > > * pci_epc_mem_alloc_addr() - allocate memory address from EPC addr space > > * @epc: the EPC device on which memory has to be allocated > > * @phys_addr: populate the allocated physical address here > > + * @window: populate the window here which will be used to map PCI address > > * @size: the size of the address space that has to be allocated > > + * @flags: look for window as requested in flags > > * > > * Invoke to allocate memory address from the EPC address space. This > > * is usually done to map the remote RC address into the local system. > > */ > > void __iomem *pci_epc_mem_alloc_addr(struct pci_epc *epc, > > - phys_addr_t *phys_addr, size_t size) > > + phys_addr_t *phys_addr, > > + int *window, size_t size, uint32_t flags) > > { > > + int best_fit = PCI_EPC_DEFAULT_WINDOW; > > + void __iomem *virt_addr = NULL; > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > int pageno; > > - void __iomem *virt_addr; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows <= 0) > > + return NULL; > > + > > + if (epc->mem_windows > 1) { > > Why bother testing epc->mem_windows here? Just make sure > pci_epc_find_best_fit_window() returns the correct thing for > "mem_windows == 0" and "mem_windows == 1", and remove both the tests > above. > will fix that. > > + best_fit = pci_epc_find_best_fit_window(epc, size, flags); > > + if (best_fit < 0) > > + return NULL; > > + } > > + > > + mem = epc->mem[best_fit]; > > size = ALIGN(size, mem->page_size); > > + if (size > (mem->window.size - mem->window.map_size)) > > + return NULL; > > + page_shift = ilog2(mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > > > pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order); > > if (pageno < 0) > > return NULL; > > > > - *phys_addr = mem->phys_base + (pageno << page_shift); > > + *phys_addr = mem->window.phys_base + (pageno << page_shift); > > virt_addr = ioremap(*phys_addr, size); > > - if (!virt_addr) > > + if (!virt_addr) { > > bitmap_release_region(mem->bitmap, pageno, order); > > + } else { > > + mem->window.map_size += size; > > + *window = best_fit; > > + } > > > > return virt_addr; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > > > +static int pci_epc_get_matching_window(struct pci_epc *epc, > > + phys_addr_t phys_addr) > > Return struct pci_epc_mem * again? > yes makes sense. > > +{ > > + struct pci_epc_mem *mem; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + if (mem->window.phys_base == phys_addr) > > + return i; > > + } > > + > > + return -EINVAL; > > +} > > + > > /** > > * pci_epc_mem_free_addr() - free the allocated memory address > > * @epc: the EPC device on which memory was allocated > > @@ -155,16 +281,26 @@ EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, > > void __iomem *virt_addr, size_t size) > > { > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > + int window = 0; > > int pageno; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows > 1) { > > Same here (unnecessary test). > will drop it. > > + window = pci_epc_get_matching_window(epc, phys_addr); > > + if (window < 0) > > + return; > > + } > > + > > + mem = epc->mem[window]; > > + page_shift = ilog2(mem->page_size); > > iounmap(virt_addr); > > - pageno = (phys_addr - mem->phys_base) >> page_shift; > > + pageno = (phys_addr - mem->window.phys_base) >> page_shift; > > size = ALIGN(size, mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > bitmap_release_region(mem->bitmap, pageno, order); > > + mem->window.map_size -= size; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr); > > > @@ -85,7 +126,8 @@ struct pci_epc_mem { > > * @dev: PCI EPC device > > * @pci_epf: list of endpoint functions present in this EPC device > > * @ops: function pointers for performing endpoint operations > > - * @mem: address space of the endpoint controller > > + * @mem: array of address space of the endpoint controller > > + * @mem_windows: number of windows supported by device > > * @max_functions: max number of functions that can be configured in this EPC > > * @group: configfs group representing the PCI EPC device > > * @lock: spinlock to protect pci_epc ops > > @@ -94,7 +136,8 @@ struct pci_epc { > > struct device dev; > > struct list_head pci_epf; > > const struct pci_epc_ops *ops; > > - struct pci_epc_mem *mem; > > + struct pci_epc_mem **mem; > > + int mem_windows; > > Can't this be unsigned int and then there's no need to check > "mem_windows < 0"? > yes will change it unsigned int. Cheers, --Prabhakar > > u8 max_functions; > > struct config_group *group; > > /* spinlock to protect against concurrent access of EP controller */ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel