From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AF40C33CA3 for ; Fri, 10 Jan 2020 18:08:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CF33820842 for ; Fri, 10 Jan 2020 18:08:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZpCuY6wQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728638AbgAJSIu (ORCPT ); Fri, 10 Jan 2020 13:08:50 -0500 Received: from mail-ot1-f67.google.com ([209.85.210.67]:36250 "EHLO mail-ot1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726767AbgAJSIt (ORCPT ); Fri, 10 Jan 2020 13:08:49 -0500 Received: by mail-ot1-f67.google.com with SMTP id 19so2826823otz.3; Fri, 10 Jan 2020 10:08:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=N0RmXH+9qj+QX8bAPpoUzZU5ql7ZmqOdGZKEvug39CU=; b=ZpCuY6wQE6lP+dHrYBkWaOk37LWQ92e9RTLQZxCkLE1bfWOyODU4NnVSqcCYGdgE05 NocwzU33V4qvXRXJFWg3PpP04z4Wm1Abphiqn7eI2b2zOL67LJbr77/qLPip1orP3LT6 zWcr/WzRZQs6CFi52+gt27CEA26DYHeLcozAJeFfD5Ey/Zoibwukit/MA1H2D94CfA/h 7tobQjySRq1tqis39rKpiFQcSxudEuSIb96jNJhqnJwi/SbT9i/sgDyJlq4nwwuHZHwB Qt4WSSJHrjN3KivUpy7s4BbF0XOrLvRdvWzB6WBGI5lncQMHA1j9akDhkdKbYPqnYWJa Ypmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=N0RmXH+9qj+QX8bAPpoUzZU5ql7ZmqOdGZKEvug39CU=; b=EiSrO+uy3w+yuf3SvrjW6kyFMUjHK1cZe12eHPnJcHOrf9osoq2yQFXy4rhn61/Qfl HWvbfo8C+iWBzpM0VXnJ/TDj2HDlvTq+M/q0v9Zl/xsBHteNeyAFM6AsaBCFdekBY5WL IgIY8AYxph46HTenBKkqBGOw3tmsgKWbmYVFBPAGlU1jJWk/V6jV1W6xDnafUnuvoVhu osJH38Gmo5+58acnWR4Nv+yyuY6FSqiUa07hR3QmV66q1iiQCY/YccAxRNST3Hf++vsR Nu/GKByUt7r5CEHeQYzMcbksVLMJ+uPAk0Sr82th11GUOiyzJ97RXqJtbB7wG+2SrBze BEJQ== X-Gm-Message-State: APjAAAU5mn3/XzIr4TdRBIDWddG4pxdiVIz33ZW66V6BnbxvJY35bwp4 7qgm3OnrkYhoC4IQpPt5FcpksgTLPlpATqlBdpE= X-Google-Smtp-Source: APXvYqx+6ZH/MWfoY/KTTSTSa7OKZSs98UA8BjDQzaK3ui04JhpUHnfYeMvGxsVFN11dFt8w+c0JffQRXh5xJLGatmM= X-Received: by 2002:a05:6830:1f95:: with SMTP id v21mr3714800otr.325.1578679728104; Fri, 10 Jan 2020 10:08:48 -0800 (PST) MIME-Version: 1.0 References: <20200108162211.22358-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20200108162211.22358-4-prabhakar.mahadev-lad.rj@bp.renesas.com> <53d74632-34ee-f7f7-656f-a93a6c10e7ba@ti.com> In-Reply-To: <53d74632-34ee-f7f7-656f-a93a6c10e7ba@ti.com> From: "Lad, Prabhakar" Date: Fri, 10 Jan 2020 18:08:21 +0000 Message-ID: Subject: Re: [v3 3/6] PCI: endpoint: Add support to handle multiple base for mapping outbound memory To: Kishon Vijay Abraham I Cc: Bjorn Helgaas , Rob Herring , Mark Rutland , Geert Uytterhoeven , Magnus Damm , Marek Vasut , Yoshihiro Shimoda , linux-pci , Catalin Marinas , Will Deacon , Lorenzo Pieralisi , Arnd Bergmann , Greg Kroah-Hartman , Andrew Murray , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , LKML , LAK , Linux-Renesas , Chris Paterson , Frank Rowand , Gustavo Pimentel , Jingoo Han , Simon Horman , Shawn Lin , Tom Joseph , Heiko Stuebner , "open list:ARM/Rockchip SoC..." , Lad Prabhakar Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Kishon, Thank you for the review. On Thu, Jan 9, 2020 at 6:25 AM Kishon Vijay Abraham I wrote: > > Hi Prabhakar, > > On 08/01/20 9:52 PM, Lad Prabhakar wrote: > > R-Car PCIe controller has support to map multiple memory regions for > > mapping the outbound memory in local system also the controller limits > > single allocation for each region (that is, once a chunk is used from the > > region it cannot be used to allocate a new one). This features inspires to > > add support for handling multiple memory bases in endpoint framework. > > > > With this patch pci_epc_mem_init() now accepts multiple regions, also > > page_size for each memory region is passed during initialization so as > > to handle single allocation for each region by setting the page_size to > > window_size. > > > > Signed-off-by: Lad Prabhakar > > --- > > .../pci/controller/cadence/pcie-cadence-ep.c | 12 +- > > .../pci/controller/dwc/pcie-designware-ep.c | 31 ++- > > drivers/pci/controller/pcie-rockchip-ep.c | 14 +- > > drivers/pci/endpoint/functions/pci-epf-test.c | 29 +-- > > drivers/pci/endpoint/pci-epc-core.c | 7 +- > > drivers/pci/endpoint/pci-epc-mem.c | 199 ++++++++++++++---- > > include/linux/pci-epc.h | 46 ++-- > > 7 files changed, 245 insertions(+), 93 deletions(-) > > > . > . > > . > . > > diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c > > index 2091508c1620..289c266c2d90 100644 > > --- a/drivers/pci/endpoint/pci-epc-core.c > > +++ b/drivers/pci/endpoint/pci-epc-core.c > > @@ -358,13 +358,15 @@ EXPORT_SYMBOL_GPL(pci_epc_unmap_addr); > > * @epc: the EPC device on which address is allocated > > * @func_no: the endpoint function number in the EPC device > > * @phys_addr: physical address of the local system > > + * @window: index to the window region where PCI address will be mapped > > * @pci_addr: PCI address to which the physical address should be mapped > > * @size: the size of the allocation > > * > > * Invoke to map CPU address with PCI address. > > */ > > int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, > > - phys_addr_t phys_addr, u64 pci_addr, size_t size) > > + phys_addr_t phys_addr, int window, > > + u64 pci_addr, size_t size) > > { > > int ret; > > unsigned long flags; > > @@ -376,7 +378,8 @@ int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, > > return 0; > > > > spin_lock_irqsave(&epc->lock, flags); > > - ret = epc->ops->map_addr(epc, func_no, phys_addr, pci_addr, size); > > + ret = epc->ops->map_addr(epc, func_no, phys_addr, > > + window, pci_addr, size); > > spin_unlock_irqrestore(&epc->lock, flags); > > > > return ret; > > diff --git a/drivers/pci/endpoint/pci-epc-mem.c b/drivers/pci/endpoint/pci-epc-mem.c > > index d2b174ce15de..f205f7819292 100644 > > --- a/drivers/pci/endpoint/pci-epc-mem.c > > +++ b/drivers/pci/endpoint/pci-epc-mem.c > > @@ -38,57 +38,77 @@ static int pci_epc_mem_get_order(struct pci_epc_mem *mem, size_t size) > > /** > > * __pci_epc_mem_init() - initialize the pci_epc_mem structure > > * @epc: the EPC device that invoked pci_epc_mem_init > > - * @phys_base: the physical address of the base > > - * @size: the size of the address space > > - * @page_size: size of each page > > + * @windows: pointer to windows supported by the device > > + * @num_windows: number of windows device supports > > * > > * Invoke to initialize the pci_epc_mem structure used by the > > * endpoint functions to allocate mapped PCI address. > > */ > > -int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, > > - size_t page_size) > > +int __pci_epc_mem_init(struct pci_epc *epc, struct pci_epc_mem_window *windows, > > + int num_windows) > > { > > - int ret; > > - struct pci_epc_mem *mem; > > - unsigned long *bitmap; > > + struct pci_epc_mem *mem = NULL; > > + unsigned long *bitmap = NULL; > > unsigned int page_shift; > > - int pages; > > + size_t page_size; > > int bitmap_size; > > + int pages; > > + int ret; > > + int i; > > > > - if (page_size < PAGE_SIZE) > > - page_size = PAGE_SIZE; > > + epc->mem_windows = 0; > > > > - page_shift = ilog2(page_size); > > - pages = size >> page_shift; > > - bitmap_size = BITS_TO_LONGS(pages) * sizeof(long); > > + if (!windows) > > + return -EINVAL; > > > > - mem = kzalloc(sizeof(*mem), GFP_KERNEL); > > - if (!mem) { > > - ret = -ENOMEM; > > - goto err; > > - } > > + if (num_windows <= 0) > > + return -EINVAL; > > > > - bitmap = kzalloc(bitmap_size, GFP_KERNEL); > > - if (!bitmap) { > > - ret = -ENOMEM; > > - goto err_mem; > > - } > > + epc->mem = kcalloc(num_windows, sizeof(*mem), GFP_KERNEL); > > + if (!epc->mem) > > + return -EINVAL; > > + > > + for (i = 0; i < num_windows; i++) { > > + page_size = windows[i].page_size; > > + if (page_size < PAGE_SIZE) > > + page_size = PAGE_SIZE; > > + page_shift = ilog2(page_size); > > + pages = windows[i].size >> page_shift; > > + bitmap_size = BITS_TO_LONGS(pages) * sizeof(long); > > + > > + mem = kzalloc(sizeof(*mem), GFP_KERNEL); > > + if (!mem) { > > + ret = -ENOMEM; > > + goto err_mem; > > + } > > > > - mem->bitmap = bitmap; > > - mem->phys_base = phys_base; > > - mem->page_size = page_size; > > - mem->pages = pages; > > - mem->size = size; > > + bitmap = kzalloc(bitmap_size, GFP_KERNEL); > > + if (!bitmap) { > > + ret = -ENOMEM; > > + goto err_mem; > > + } > > > > - epc->mem = mem; > > + mem->bitmap = bitmap; > > + mem->window.phys_base = windows[i].phys_base; > > + mem->page_size = page_size; > > + mem->pages = pages; > > + mem->window.size = windows[i].size; > > + mem->window.map_size = 0; > > + > > + epc->mem[i] = mem; > > + } > > + epc->mem_windows = num_windows; > > > > return 0; > > > > err_mem: > > - kfree(mem); > > + for (; i >= 0; i--) { > > mem has to be reinitialized for every iteration of the loop. not sure what exactly you mean here, could you please elaborate. > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > -err: > > -return ret; > > + return ret; > > } > > EXPORT_SYMBOL_GPL(__pci_epc_mem_init); > > > > @@ -101,48 +121,127 @@ EXPORT_SYMBOL_GPL(__pci_epc_mem_init); > > */ > > void pci_epc_mem_exit(struct pci_epc *epc) > > { > > - struct pci_epc_mem *mem = epc->mem; > > + struct pci_epc_mem *mem; > > + int i; > > + > > + if (!epc->mem_windows) > > + return; > > + > > + for (i = 0; i <= epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > epc->mem = NULL; > > - kfree(mem->bitmap); > > - kfree(mem); > > + epc->mem_windows = 0; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_exit); > > > > +static int pci_epc_find_best_fit_window(struct pci_epc *epc, size_t size) > > +{ > > + size_t window_least_size = 0; > > + int best_fit_window = -1; > > + struct pci_epc_mem *mem; > > + size_t actual_size; > > + size_t avail_size; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + actual_size = ALIGN(size, mem->page_size); > > + avail_size = mem->window.size - mem->window.map_size; > > + > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } > > + > > + return best_fit_window; > > +} > > This function shouldn't be required at all. Just loop over all the > windows invoking bitmap_find_free_region(), breaking the loop if it > succeeds and continuing if there is no free region. sure will do. > > + > > /** > > * pci_epc_mem_alloc_addr() - allocate memory address from EPC addr space > > * @epc: the EPC device on which memory has to be allocated > > * @phys_addr: populate the allocated physical address here > > + * @window: populate the window here which will be used to map PCI address > > * @size: the size of the address space that has to be allocated > > * > > * Invoke to allocate memory address from the EPC address space. This > > * is usually done to map the remote RC address into the local system. > > */ > > void __iomem *pci_epc_mem_alloc_addr(struct pci_epc *epc, > > - phys_addr_t *phys_addr, size_t size) > > + phys_addr_t *phys_addr, > > + int *window, size_t size) > > { > > + int best_fit = PCI_EPC_DEFAULT_WINDOW; > > + void __iomem *virt_addr = NULL; > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > int pageno; > > - void __iomem *virt_addr; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows <= 0) > > + return NULL; > > + > > + if (epc->mem_windows > 1) { > > + best_fit = pci_epc_find_best_fit_window(epc, size); > > + if (best_fit < 0) > > + return NULL; > > + } > > + > > + mem = epc->mem[best_fit]; > > size = ALIGN(size, mem->page_size); > > + if (size > (mem->window.size - mem->window.map_size)) > > + return NULL; > > + page_shift = ilog2(mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > > > pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order); > > if (pageno < 0) > > return NULL; > > This has to be invoked now in a loop for the number of windows. If > pageno < 0, continue the loop or else break. > > will fix that. > > - *phys_addr = mem->phys_base + ((phys_addr_t)pageno << page_shift); > > + *phys_addr = mem->window.phys_base + > > + ((phys_addr_t)pageno << page_shift); > > virt_addr = ioremap(*phys_addr, size); > > - if (!virt_addr) > > + if (!virt_addr) { > > bitmap_release_region(mem->bitmap, pageno, order); > > + } else { > > + mem->window.map_size += size; > > The map_size shouldn't be required. > > + *window = best_fit; > > Can't the platform driver get the window based on the address? Let's try > to avoid modifying API's for platform specific requirements. should be possible, will do that. > > + } > > > > return virt_addr; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > > > +static int pci_epc_get_matching_window(struct pci_epc *epc, > > + phys_addr_t phys_addr) > > +{ > > + struct pci_epc_mem *mem; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + if (mem->window.phys_base == phys_addr) > > + return i; > > + } > > + > > + return -EINVAL; > > +} > > + > > /** > > * pci_epc_mem_free_addr() - free the allocated memory address > > * @epc: the EPC device on which memory was allocated > > @@ -155,16 +254,26 @@ EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, > > void __iomem *virt_addr, size_t size) > > { > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > + int window = 0; > > int pageno; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows > 1) { > > + window = pci_epc_get_matching_window(epc, phys_addr); > > + if (window < 0) > > + return; > > + } > > + > > + mem = epc->mem[window]; > > + page_shift = ilog2(mem->page_size); > > iounmap(virt_addr); > > - pageno = (phys_addr - mem->phys_base) >> page_shift; > > + pageno = (phys_addr - mem->window.phys_base) >> page_shift; > > size = ALIGN(size, mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > bitmap_release_region(mem->bitmap, pageno, order); > > + mem->window.map_size -= size; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr); > > > > diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h > > index 56f1846b9d39..89daafa0d19f 100644 > > --- a/include/linux/pci-epc.h > > +++ b/include/linux/pci-epc.h > > @@ -48,7 +48,8 @@ struct pci_epc_ops { > > void (*clear_bar)(struct pci_epc *epc, u8 func_no, > > struct pci_epf_bar *epf_bar); > > int (*map_addr)(struct pci_epc *epc, u8 func_no, > > - phys_addr_t addr, u64 pci_addr, size_t size); > > + phys_addr_t addr, int window, > > + u64 pci_addr, size_t size); > > void (*unmap_addr)(struct pci_epc *epc, u8 func_no, > > phys_addr_t addr); > > int (*set_msi)(struct pci_epc *epc, u8 func_no, u8 interrupts); > > @@ -64,17 +65,31 @@ struct pci_epc_ops { > > struct module *owner; > > }; > > > > +#define PCI_EPC_DEFAULT_WINDOW 0 > > + > > +/** > > + * struct pci_epc_mem_window - address window of the endpoint controller > > + * @phys_base: physical base address of the PCI address window > > + * @size: the size of the PCI address window > > + * @map_size: size of allocated chunk in window > > + * @page_size: size of each page > > + */ > > +struct pci_epc_mem_window { > > + phys_addr_t phys_base; > > + size_t size; > > + size_t map_size; > > mem->bitmap already holds the map_size information. This structure can > be used only for passing info about the mem windows from platform to the > core. So map_size shouldn't be required. > agreed will drop it. Cheers, --Prabhakar From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Lad, Prabhakar" Subject: Re: [v3 3/6] PCI: endpoint: Add support to handle multiple base for mapping outbound memory Date: Fri, 10 Jan 2020 18:08:21 +0000 Message-ID: References: <20200108162211.22358-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20200108162211.22358-4-prabhakar.mahadev-lad.rj@bp.renesas.com> <53d74632-34ee-f7f7-656f-a93a6c10e7ba@ti.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: In-Reply-To: <53d74632-34ee-f7f7-656f-a93a6c10e7ba@ti.com> Sender: linux-pci-owner@vger.kernel.org To: Kishon Vijay Abraham I Cc: Bjorn Helgaas , Rob Herring , Mark Rutland , Geert Uytterhoeven , Magnus Damm , Marek Vasut , Yoshihiro Shimoda , linux-pci , Catalin Marinas , Will Deacon , Lorenzo Pieralisi , Arnd Bergmann , Greg Kroah-Hartman , Andrew Murray , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , LKML , LAK Linux-Renesas List-Id: linux-rockchip.vger.kernel.org Hi Kishon, Thank you for the review. On Thu, Jan 9, 2020 at 6:25 AM Kishon Vijay Abraham I wrote: > > Hi Prabhakar, > > On 08/01/20 9:52 PM, Lad Prabhakar wrote: > > R-Car PCIe controller has support to map multiple memory regions for > > mapping the outbound memory in local system also the controller limits > > single allocation for each region (that is, once a chunk is used from the > > region it cannot be used to allocate a new one). This features inspires to > > add support for handling multiple memory bases in endpoint framework. > > > > With this patch pci_epc_mem_init() now accepts multiple regions, also > > page_size for each memory region is passed during initialization so as > > to handle single allocation for each region by setting the page_size to > > window_size. > > > > Signed-off-by: Lad Prabhakar > > --- > > .../pci/controller/cadence/pcie-cadence-ep.c | 12 +- > > .../pci/controller/dwc/pcie-designware-ep.c | 31 ++- > > drivers/pci/controller/pcie-rockchip-ep.c | 14 +- > > drivers/pci/endpoint/functions/pci-epf-test.c | 29 +-- > > drivers/pci/endpoint/pci-epc-core.c | 7 +- > > drivers/pci/endpoint/pci-epc-mem.c | 199 ++++++++++++++---- > > include/linux/pci-epc.h | 46 ++-- > > 7 files changed, 245 insertions(+), 93 deletions(-) > > > . > . > > . > . > > diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c > > index 2091508c1620..289c266c2d90 100644 > > --- a/drivers/pci/endpoint/pci-epc-core.c > > +++ b/drivers/pci/endpoint/pci-epc-core.c > > @@ -358,13 +358,15 @@ EXPORT_SYMBOL_GPL(pci_epc_unmap_addr); > > * @epc: the EPC device on which address is allocated > > * @func_no: the endpoint function number in the EPC device > > * @phys_addr: physical address of the local system > > + * @window: index to the window region where PCI address will be mapped > > * @pci_addr: PCI address to which the physical address should be mapped > > * @size: the size of the allocation > > * > > * Invoke to map CPU address with PCI address. > > */ > > int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, > > - phys_addr_t phys_addr, u64 pci_addr, size_t size) > > + phys_addr_t phys_addr, int window, > > + u64 pci_addr, size_t size) > > { > > int ret; > > unsigned long flags; > > @@ -376,7 +378,8 @@ int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, > > return 0; > > > > spin_lock_irqsave(&epc->lock, flags); > > - ret = epc->ops->map_addr(epc, func_no, phys_addr, pci_addr, size); > > + ret = epc->ops->map_addr(epc, func_no, phys_addr, > > + window, pci_addr, size); > > spin_unlock_irqrestore(&epc->lock, flags); > > > > return ret; > > diff --git a/drivers/pci/endpoint/pci-epc-mem.c b/drivers/pci/endpoint/pci-epc-mem.c > > index d2b174ce15de..f205f7819292 100644 > > --- a/drivers/pci/endpoint/pci-epc-mem.c > > +++ b/drivers/pci/endpoint/pci-epc-mem.c > > @@ -38,57 +38,77 @@ static int pci_epc_mem_get_order(struct pci_epc_mem *mem, size_t size) > > /** > > * __pci_epc_mem_init() - initialize the pci_epc_mem structure > > * @epc: the EPC device that invoked pci_epc_mem_init > > - * @phys_base: the physical address of the base > > - * @size: the size of the address space > > - * @page_size: size of each page > > + * @windows: pointer to windows supported by the device > > + * @num_windows: number of windows device supports > > * > > * Invoke to initialize the pci_epc_mem structure used by the > > * endpoint functions to allocate mapped PCI address. > > */ > > -int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, > > - size_t page_size) > > +int __pci_epc_mem_init(struct pci_epc *epc, struct pci_epc_mem_window *windows, > > + int num_windows) > > { > > - int ret; > > - struct pci_epc_mem *mem; > > - unsigned long *bitmap; > > + struct pci_epc_mem *mem = NULL; > > + unsigned long *bitmap = NULL; > > unsigned int page_shift; > > - int pages; > > + size_t page_size; > > int bitmap_size; > > + int pages; > > + int ret; > > + int i; > > > > - if (page_size < PAGE_SIZE) > > - page_size = PAGE_SIZE; > > + epc->mem_windows = 0; > > > > - page_shift = ilog2(page_size); > > - pages = size >> page_shift; > > - bitmap_size = BITS_TO_LONGS(pages) * sizeof(long); > > + if (!windows) > > + return -EINVAL; > > > > - mem = kzalloc(sizeof(*mem), GFP_KERNEL); > > - if (!mem) { > > - ret = -ENOMEM; > > - goto err; > > - } > > + if (num_windows <= 0) > > + return -EINVAL; > > > > - bitmap = kzalloc(bitmap_size, GFP_KERNEL); > > - if (!bitmap) { > > - ret = -ENOMEM; > > - goto err_mem; > > - } > > + epc->mem = kcalloc(num_windows, sizeof(*mem), GFP_KERNEL); > > + if (!epc->mem) > > + return -EINVAL; > > + > > + for (i = 0; i < num_windows; i++) { > > + page_size = windows[i].page_size; > > + if (page_size < PAGE_SIZE) > > + page_size = PAGE_SIZE; > > + page_shift = ilog2(page_size); > > + pages = windows[i].size >> page_shift; > > + bitmap_size = BITS_TO_LONGS(pages) * sizeof(long); > > + > > + mem = kzalloc(sizeof(*mem), GFP_KERNEL); > > + if (!mem) { > > + ret = -ENOMEM; > > + goto err_mem; > > + } > > > > - mem->bitmap = bitmap; > > - mem->phys_base = phys_base; > > - mem->page_size = page_size; > > - mem->pages = pages; > > - mem->size = size; > > + bitmap = kzalloc(bitmap_size, GFP_KERNEL); > > + if (!bitmap) { > > + ret = -ENOMEM; > > + goto err_mem; > > + } > > > > - epc->mem = mem; > > + mem->bitmap = bitmap; > > + mem->window.phys_base = windows[i].phys_base; > > + mem->page_size = page_size; > > + mem->pages = pages; > > + mem->window.size = windows[i].size; > > + mem->window.map_size = 0; > > + > > + epc->mem[i] = mem; > > + } > > + epc->mem_windows = num_windows; > > > > return 0; > > > > err_mem: > > - kfree(mem); > > + for (; i >= 0; i--) { > > mem has to be reinitialized for every iteration of the loop. not sure what exactly you mean here, could you please elaborate. > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > -err: > > -return ret; > > + return ret; > > } > > EXPORT_SYMBOL_GPL(__pci_epc_mem_init); > > > > @@ -101,48 +121,127 @@ EXPORT_SYMBOL_GPL(__pci_epc_mem_init); > > */ > > void pci_epc_mem_exit(struct pci_epc *epc) > > { > > - struct pci_epc_mem *mem = epc->mem; > > + struct pci_epc_mem *mem; > > + int i; > > + > > + if (!epc->mem_windows) > > + return; > > + > > + for (i = 0; i <= epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > epc->mem = NULL; > > - kfree(mem->bitmap); > > - kfree(mem); > > + epc->mem_windows = 0; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_exit); > > > > +static int pci_epc_find_best_fit_window(struct pci_epc *epc, size_t size) > > +{ > > + size_t window_least_size = 0; > > + int best_fit_window = -1; > > + struct pci_epc_mem *mem; > > + size_t actual_size; > > + size_t avail_size; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + actual_size = ALIGN(size, mem->page_size); > > + avail_size = mem->window.size - mem->window.map_size; > > + > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } > > + > > + return best_fit_window; > > +} > > This function shouldn't be required at all. Just loop over all the > windows invoking bitmap_find_free_region(), breaking the loop if it > succeeds and continuing if there is no free region. sure will do. > > + > > /** > > * pci_epc_mem_alloc_addr() - allocate memory address from EPC addr space > > * @epc: the EPC device on which memory has to be allocated > > * @phys_addr: populate the allocated physical address here > > + * @window: populate the window here which will be used to map PCI address > > * @size: the size of the address space that has to be allocated > > * > > * Invoke to allocate memory address from the EPC address space. This > > * is usually done to map the remote RC address into the local system. > > */ > > void __iomem *pci_epc_mem_alloc_addr(struct pci_epc *epc, > > - phys_addr_t *phys_addr, size_t size) > > + phys_addr_t *phys_addr, > > + int *window, size_t size) > > { > > + int best_fit = PCI_EPC_DEFAULT_WINDOW; > > + void __iomem *virt_addr = NULL; > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > int pageno; > > - void __iomem *virt_addr; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows <= 0) > > + return NULL; > > + > > + if (epc->mem_windows > 1) { > > + best_fit = pci_epc_find_best_fit_window(epc, size); > > + if (best_fit < 0) > > + return NULL; > > + } > > + > > + mem = epc->mem[best_fit]; > > size = ALIGN(size, mem->page_size); > > + if (size > (mem->window.size - mem->window.map_size)) > > + return NULL; > > + page_shift = ilog2(mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > > > pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order); > > if (pageno < 0) > > return NULL; > > This has to be invoked now in a loop for the number of windows. If > pageno < 0, continue the loop or else break. > > will fix that. > > - *phys_addr = mem->phys_base + ((phys_addr_t)pageno << page_shift); > > + *phys_addr = mem->window.phys_base + > > + ((phys_addr_t)pageno << page_shift); > > virt_addr = ioremap(*phys_addr, size); > > - if (!virt_addr) > > + if (!virt_addr) { > > bitmap_release_region(mem->bitmap, pageno, order); > > + } else { > > + mem->window.map_size += size; > > The map_size shouldn't be required. > > + *window = best_fit; > > Can't the platform driver get the window based on the address? Let's try > to avoid modifying API's for platform specific requirements. should be possible, will do that. > > + } > > > > return virt_addr; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > > > +static int pci_epc_get_matching_window(struct pci_epc *epc, > > + phys_addr_t phys_addr) > > +{ > > + struct pci_epc_mem *mem; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + if (mem->window.phys_base == phys_addr) > > + return i; > > + } > > + > > + return -EINVAL; > > +} > > + > > /** > > * pci_epc_mem_free_addr() - free the allocated memory address > > * @epc: the EPC device on which memory was allocated > > @@ -155,16 +254,26 @@ EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, > > void __iomem *virt_addr, size_t size) > > { > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > + int window = 0; > > int pageno; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows > 1) { > > + window = pci_epc_get_matching_window(epc, phys_addr); > > + if (window < 0) > > + return; > > + } > > + > > + mem = epc->mem[window]; > > + page_shift = ilog2(mem->page_size); > > iounmap(virt_addr); > > - pageno = (phys_addr - mem->phys_base) >> page_shift; > > + pageno = (phys_addr - mem->window.phys_base) >> page_shift; > > size = ALIGN(size, mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > bitmap_release_region(mem->bitmap, pageno, order); > > + mem->window.map_size -= size; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr); > > > > diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h > > index 56f1846b9d39..89daafa0d19f 100644 > > --- a/include/linux/pci-epc.h > > +++ b/include/linux/pci-epc.h > > @@ -48,7 +48,8 @@ struct pci_epc_ops { > > void (*clear_bar)(struct pci_epc *epc, u8 func_no, > > struct pci_epf_bar *epf_bar); > > int (*map_addr)(struct pci_epc *epc, u8 func_no, > > - phys_addr_t addr, u64 pci_addr, size_t size); > > + phys_addr_t addr, int window, > > + u64 pci_addr, size_t size); > > void (*unmap_addr)(struct pci_epc *epc, u8 func_no, > > phys_addr_t addr); > > int (*set_msi)(struct pci_epc *epc, u8 func_no, u8 interrupts); > > @@ -64,17 +65,31 @@ struct pci_epc_ops { > > struct module *owner; > > }; > > > > +#define PCI_EPC_DEFAULT_WINDOW 0 > > + > > +/** > > + * struct pci_epc_mem_window - address window of the endpoint controller > > + * @phys_base: physical base address of the PCI address window > > + * @size: the size of the PCI address window > > + * @map_size: size of allocated chunk in window > > + * @page_size: size of each page > > + */ > > +struct pci_epc_mem_window { > > + phys_addr_t phys_base; > > + size_t size; > > + size_t map_size; > > mem->bitmap already holds the map_size information. This structure can > be used only for passing info about the mem windows from platform to the > core. So map_size shouldn't be required. > agreed will drop it. Cheers, --Prabhakar From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D41DCC282DD for ; Fri, 10 Jan 2020 18:08:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9DF8C206ED for ; Fri, 10 Jan 2020 18:08:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="dpdMqpGC"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZpCuY6wQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DF8C206ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RjmQuLfqmxirgbrS05gNUBLKDOYh8UzLwQu453p0rYA=; b=dpdMqpGCZwESkf h9XJ4YVYBYHTwzZiWWq3Qf5rpcrmG/SdQ/wqsMzNxpypfIAIBz2dvvFdJL8Jy8SjZEj3iK3F/toRp zaDbgCNe2fWh94P6aMuxXnngCWzc8+bqRTNFIg3qyMm6cz7yv0QbG3ZHs47Uf0aBVdyF5tIb2pUeX X3kJI4Q1TkpAG3MdA6EKtJ3xGK53JbuFFEI2RSOjAhjuumYIUToYiGsZo1OCtB/fYdfgl9i0Am/C2 FX8qMZG9K6pmsfsLOg3b6L7EDJPZkHJRgfPyn2GAeWk/FuBhYhU7N6F5OX1no2jHK6BxkMl7206Zu IPoNCTsKTci+VOaUez3g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ipyiL-0008KS-3l; Fri, 10 Jan 2020 18:08:53 +0000 Received: from mail-ot1-x341.google.com ([2607:f8b0:4864:20::341]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ipyiH-0008Jb-It; Fri, 10 Jan 2020 18:08:51 +0000 Received: by mail-ot1-x341.google.com with SMTP id w21so2797085otj.7; Fri, 10 Jan 2020 10:08:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=N0RmXH+9qj+QX8bAPpoUzZU5ql7ZmqOdGZKEvug39CU=; b=ZpCuY6wQE6lP+dHrYBkWaOk37LWQ92e9RTLQZxCkLE1bfWOyODU4NnVSqcCYGdgE05 NocwzU33V4qvXRXJFWg3PpP04z4Wm1Abphiqn7eI2b2zOL67LJbr77/qLPip1orP3LT6 zWcr/WzRZQs6CFi52+gt27CEA26DYHeLcozAJeFfD5Ey/Zoibwukit/MA1H2D94CfA/h 7tobQjySRq1tqis39rKpiFQcSxudEuSIb96jNJhqnJwi/SbT9i/sgDyJlq4nwwuHZHwB Qt4WSSJHrjN3KivUpy7s4BbF0XOrLvRdvWzB6WBGI5lncQMHA1j9akDhkdKbYPqnYWJa Ypmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=N0RmXH+9qj+QX8bAPpoUzZU5ql7ZmqOdGZKEvug39CU=; b=oYzNCK/1s6hgyF0w+YU799bNf1D2WXp1Qh+j66oittp0OwZxfy0a37JcGsdDRMfZI2 MCZCdPh5zUgGyQ6HFMh3A82v6f0vaLlLXFB6axehlvkBT8cOEIjbTO4fJlGt5zOqPlJt bS14ctP8/oXpS2/0FWVlY7xuXxOMSLyOtGw8G8Bljtlp7qPwMcpZrk3va3Bb6Oy9GM23 MPa46dCbJsp0E3HQZy8V0cjLAe/diUvY5WPrjidV9aAU1CHB2aQIIawlhthfDYE/pYk9 /K/Ik1hHiQ6Ae5G6vCkNqFDKesHoPwy2HFsWzogiLQq3EX8R1Qi47sPZTLVr+xS/53K2 awSQ== X-Gm-Message-State: APjAAAUUceOuxDviWJwUICmJjdiyv2gPCPyoA8aCcifBgI/AJHqJwJOM fh+YZc4jgkXY9UigqAbp5NLNOuEduUkgw9/+2wE= X-Google-Smtp-Source: APXvYqx+6ZH/MWfoY/KTTSTSa7OKZSs98UA8BjDQzaK3ui04JhpUHnfYeMvGxsVFN11dFt8w+c0JffQRXh5xJLGatmM= X-Received: by 2002:a05:6830:1f95:: with SMTP id v21mr3714800otr.325.1578679728104; Fri, 10 Jan 2020 10:08:48 -0800 (PST) MIME-Version: 1.0 References: <20200108162211.22358-1-prabhakar.mahadev-lad.rj@bp.renesas.com> <20200108162211.22358-4-prabhakar.mahadev-lad.rj@bp.renesas.com> <53d74632-34ee-f7f7-656f-a93a6c10e7ba@ti.com> In-Reply-To: <53d74632-34ee-f7f7-656f-a93a6c10e7ba@ti.com> From: "Lad, Prabhakar" Date: Fri, 10 Jan 2020 18:08:21 +0000 Message-ID: Subject: Re: [v3 3/6] PCI: endpoint: Add support to handle multiple base for mapping outbound memory To: Kishon Vijay Abraham I X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200110_100849_649542_7040EB0F X-CRM114-Status: GOOD ( 33.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Heiko Stuebner , Geert Uytterhoeven , linux-pci , Shawn Lin , Frank Rowand , Marek Vasut , Lorenzo Pieralisi , Will Deacon , Magnus Damm , "open list:ARM/Rockchip SoC..." , Catalin Marinas , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , Chris Paterson , Arnd Bergmann , Lad Prabhakar , Rob Herring , Bjorn Helgaas , LAK , Greg Kroah-Hartman , Yoshihiro Shimoda , LKML , Linux-Renesas , Tom Joseph , Simon Horman , Jingoo Han , Andrew Murray , Gustavo Pimentel Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Kishon, Thank you for the review. On Thu, Jan 9, 2020 at 6:25 AM Kishon Vijay Abraham I wrote: > > Hi Prabhakar, > > On 08/01/20 9:52 PM, Lad Prabhakar wrote: > > R-Car PCIe controller has support to map multiple memory regions for > > mapping the outbound memory in local system also the controller limits > > single allocation for each region (that is, once a chunk is used from the > > region it cannot be used to allocate a new one). This features inspires to > > add support for handling multiple memory bases in endpoint framework. > > > > With this patch pci_epc_mem_init() now accepts multiple regions, also > > page_size for each memory region is passed during initialization so as > > to handle single allocation for each region by setting the page_size to > > window_size. > > > > Signed-off-by: Lad Prabhakar > > --- > > .../pci/controller/cadence/pcie-cadence-ep.c | 12 +- > > .../pci/controller/dwc/pcie-designware-ep.c | 31 ++- > > drivers/pci/controller/pcie-rockchip-ep.c | 14 +- > > drivers/pci/endpoint/functions/pci-epf-test.c | 29 +-- > > drivers/pci/endpoint/pci-epc-core.c | 7 +- > > drivers/pci/endpoint/pci-epc-mem.c | 199 ++++++++++++++---- > > include/linux/pci-epc.h | 46 ++-- > > 7 files changed, 245 insertions(+), 93 deletions(-) > > > . > . > > . > . > > diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c > > index 2091508c1620..289c266c2d90 100644 > > --- a/drivers/pci/endpoint/pci-epc-core.c > > +++ b/drivers/pci/endpoint/pci-epc-core.c > > @@ -358,13 +358,15 @@ EXPORT_SYMBOL_GPL(pci_epc_unmap_addr); > > * @epc: the EPC device on which address is allocated > > * @func_no: the endpoint function number in the EPC device > > * @phys_addr: physical address of the local system > > + * @window: index to the window region where PCI address will be mapped > > * @pci_addr: PCI address to which the physical address should be mapped > > * @size: the size of the allocation > > * > > * Invoke to map CPU address with PCI address. > > */ > > int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, > > - phys_addr_t phys_addr, u64 pci_addr, size_t size) > > + phys_addr_t phys_addr, int window, > > + u64 pci_addr, size_t size) > > { > > int ret; > > unsigned long flags; > > @@ -376,7 +378,8 @@ int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, > > return 0; > > > > spin_lock_irqsave(&epc->lock, flags); > > - ret = epc->ops->map_addr(epc, func_no, phys_addr, pci_addr, size); > > + ret = epc->ops->map_addr(epc, func_no, phys_addr, > > + window, pci_addr, size); > > spin_unlock_irqrestore(&epc->lock, flags); > > > > return ret; > > diff --git a/drivers/pci/endpoint/pci-epc-mem.c b/drivers/pci/endpoint/pci-epc-mem.c > > index d2b174ce15de..f205f7819292 100644 > > --- a/drivers/pci/endpoint/pci-epc-mem.c > > +++ b/drivers/pci/endpoint/pci-epc-mem.c > > @@ -38,57 +38,77 @@ static int pci_epc_mem_get_order(struct pci_epc_mem *mem, size_t size) > > /** > > * __pci_epc_mem_init() - initialize the pci_epc_mem structure > > * @epc: the EPC device that invoked pci_epc_mem_init > > - * @phys_base: the physical address of the base > > - * @size: the size of the address space > > - * @page_size: size of each page > > + * @windows: pointer to windows supported by the device > > + * @num_windows: number of windows device supports > > * > > * Invoke to initialize the pci_epc_mem structure used by the > > * endpoint functions to allocate mapped PCI address. > > */ > > -int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, > > - size_t page_size) > > +int __pci_epc_mem_init(struct pci_epc *epc, struct pci_epc_mem_window *windows, > > + int num_windows) > > { > > - int ret; > > - struct pci_epc_mem *mem; > > - unsigned long *bitmap; > > + struct pci_epc_mem *mem = NULL; > > + unsigned long *bitmap = NULL; > > unsigned int page_shift; > > - int pages; > > + size_t page_size; > > int bitmap_size; > > + int pages; > > + int ret; > > + int i; > > > > - if (page_size < PAGE_SIZE) > > - page_size = PAGE_SIZE; > > + epc->mem_windows = 0; > > > > - page_shift = ilog2(page_size); > > - pages = size >> page_shift; > > - bitmap_size = BITS_TO_LONGS(pages) * sizeof(long); > > + if (!windows) > > + return -EINVAL; > > > > - mem = kzalloc(sizeof(*mem), GFP_KERNEL); > > - if (!mem) { > > - ret = -ENOMEM; > > - goto err; > > - } > > + if (num_windows <= 0) > > + return -EINVAL; > > > > - bitmap = kzalloc(bitmap_size, GFP_KERNEL); > > - if (!bitmap) { > > - ret = -ENOMEM; > > - goto err_mem; > > - } > > + epc->mem = kcalloc(num_windows, sizeof(*mem), GFP_KERNEL); > > + if (!epc->mem) > > + return -EINVAL; > > + > > + for (i = 0; i < num_windows; i++) { > > + page_size = windows[i].page_size; > > + if (page_size < PAGE_SIZE) > > + page_size = PAGE_SIZE; > > + page_shift = ilog2(page_size); > > + pages = windows[i].size >> page_shift; > > + bitmap_size = BITS_TO_LONGS(pages) * sizeof(long); > > + > > + mem = kzalloc(sizeof(*mem), GFP_KERNEL); > > + if (!mem) { > > + ret = -ENOMEM; > > + goto err_mem; > > + } > > > > - mem->bitmap = bitmap; > > - mem->phys_base = phys_base; > > - mem->page_size = page_size; > > - mem->pages = pages; > > - mem->size = size; > > + bitmap = kzalloc(bitmap_size, GFP_KERNEL); > > + if (!bitmap) { > > + ret = -ENOMEM; > > + goto err_mem; > > + } > > > > - epc->mem = mem; > > + mem->bitmap = bitmap; > > + mem->window.phys_base = windows[i].phys_base; > > + mem->page_size = page_size; > > + mem->pages = pages; > > + mem->window.size = windows[i].size; > > + mem->window.map_size = 0; > > + > > + epc->mem[i] = mem; > > + } > > + epc->mem_windows = num_windows; > > > > return 0; > > > > err_mem: > > - kfree(mem); > > + for (; i >= 0; i--) { > > mem has to be reinitialized for every iteration of the loop. not sure what exactly you mean here, could you please elaborate. > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > -err: > > -return ret; > > + return ret; > > } > > EXPORT_SYMBOL_GPL(__pci_epc_mem_init); > > > > @@ -101,48 +121,127 @@ EXPORT_SYMBOL_GPL(__pci_epc_mem_init); > > */ > > void pci_epc_mem_exit(struct pci_epc *epc) > > { > > - struct pci_epc_mem *mem = epc->mem; > > + struct pci_epc_mem *mem; > > + int i; > > + > > + if (!epc->mem_windows) > > + return; > > + > > + for (i = 0; i <= epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + kfree(mem->bitmap); > > + kfree(epc->mem[i]); > > + } > > + kfree(epc->mem); > > > > epc->mem = NULL; > > - kfree(mem->bitmap); > > - kfree(mem); > > + epc->mem_windows = 0; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_exit); > > > > +static int pci_epc_find_best_fit_window(struct pci_epc *epc, size_t size) > > +{ > > + size_t window_least_size = 0; > > + int best_fit_window = -1; > > + struct pci_epc_mem *mem; > > + size_t actual_size; > > + size_t avail_size; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + actual_size = ALIGN(size, mem->page_size); > > + avail_size = mem->window.size - mem->window.map_size; > > + > > + if (best_fit_window == -1) { > > + if (actual_size <= avail_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } else { > > + if (actual_size <= avail_size && > > + mem->window.size < window_least_size) { > > + best_fit_window = i; > > + window_least_size = mem->window.size; > > + } > > + } > > + } > > + > > + return best_fit_window; > > +} > > This function shouldn't be required at all. Just loop over all the > windows invoking bitmap_find_free_region(), breaking the loop if it > succeeds and continuing if there is no free region. sure will do. > > + > > /** > > * pci_epc_mem_alloc_addr() - allocate memory address from EPC addr space > > * @epc: the EPC device on which memory has to be allocated > > * @phys_addr: populate the allocated physical address here > > + * @window: populate the window here which will be used to map PCI address > > * @size: the size of the address space that has to be allocated > > * > > * Invoke to allocate memory address from the EPC address space. This > > * is usually done to map the remote RC address into the local system. > > */ > > void __iomem *pci_epc_mem_alloc_addr(struct pci_epc *epc, > > - phys_addr_t *phys_addr, size_t size) > > + phys_addr_t *phys_addr, > > + int *window, size_t size) > > { > > + int best_fit = PCI_EPC_DEFAULT_WINDOW; > > + void __iomem *virt_addr = NULL; > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > int pageno; > > - void __iomem *virt_addr; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows <= 0) > > + return NULL; > > + > > + if (epc->mem_windows > 1) { > > + best_fit = pci_epc_find_best_fit_window(epc, size); > > + if (best_fit < 0) > > + return NULL; > > + } > > + > > + mem = epc->mem[best_fit]; > > size = ALIGN(size, mem->page_size); > > + if (size > (mem->window.size - mem->window.map_size)) > > + return NULL; > > + page_shift = ilog2(mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > > > pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order); > > if (pageno < 0) > > return NULL; > > This has to be invoked now in a loop for the number of windows. If > pageno < 0, continue the loop or else break. > > will fix that. > > - *phys_addr = mem->phys_base + ((phys_addr_t)pageno << page_shift); > > + *phys_addr = mem->window.phys_base + > > + ((phys_addr_t)pageno << page_shift); > > virt_addr = ioremap(*phys_addr, size); > > - if (!virt_addr) > > + if (!virt_addr) { > > bitmap_release_region(mem->bitmap, pageno, order); > > + } else { > > + mem->window.map_size += size; > > The map_size shouldn't be required. > > + *window = best_fit; > > Can't the platform driver get the window based on the address? Let's try > to avoid modifying API's for platform specific requirements. should be possible, will do that. > > + } > > > > return virt_addr; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > > > +static int pci_epc_get_matching_window(struct pci_epc *epc, > > + phys_addr_t phys_addr) > > +{ > > + struct pci_epc_mem *mem; > > + int i; > > + > > + for (i = 0; i < epc->mem_windows; i++) { > > + mem = epc->mem[i]; > > + > > + if (mem->window.phys_base == phys_addr) > > + return i; > > + } > > + > > + return -EINVAL; > > +} > > + > > /** > > * pci_epc_mem_free_addr() - free the allocated memory address > > * @epc: the EPC device on which memory was allocated > > @@ -155,16 +254,26 @@ EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr); > > void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, > > void __iomem *virt_addr, size_t size) > > { > > + struct pci_epc_mem *mem; > > + unsigned int page_shift; > > + int window = 0; > > int pageno; > > - struct pci_epc_mem *mem = epc->mem; > > - unsigned int page_shift = ilog2(mem->page_size); > > int order; > > > > + if (epc->mem_windows > 1) { > > + window = pci_epc_get_matching_window(epc, phys_addr); > > + if (window < 0) > > + return; > > + } > > + > > + mem = epc->mem[window]; > > + page_shift = ilog2(mem->page_size); > > iounmap(virt_addr); > > - pageno = (phys_addr - mem->phys_base) >> page_shift; > > + pageno = (phys_addr - mem->window.phys_base) >> page_shift; > > size = ALIGN(size, mem->page_size); > > order = pci_epc_mem_get_order(mem, size); > > bitmap_release_region(mem->bitmap, pageno, order); > > + mem->window.map_size -= size; > > } > > EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr); > > > > diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h > > index 56f1846b9d39..89daafa0d19f 100644 > > --- a/include/linux/pci-epc.h > > +++ b/include/linux/pci-epc.h > > @@ -48,7 +48,8 @@ struct pci_epc_ops { > > void (*clear_bar)(struct pci_epc *epc, u8 func_no, > > struct pci_epf_bar *epf_bar); > > int (*map_addr)(struct pci_epc *epc, u8 func_no, > > - phys_addr_t addr, u64 pci_addr, size_t size); > > + phys_addr_t addr, int window, > > + u64 pci_addr, size_t size); > > void (*unmap_addr)(struct pci_epc *epc, u8 func_no, > > phys_addr_t addr); > > int (*set_msi)(struct pci_epc *epc, u8 func_no, u8 interrupts); > > @@ -64,17 +65,31 @@ struct pci_epc_ops { > > struct module *owner; > > }; > > > > +#define PCI_EPC_DEFAULT_WINDOW 0 > > + > > +/** > > + * struct pci_epc_mem_window - address window of the endpoint controller > > + * @phys_base: physical base address of the PCI address window > > + * @size: the size of the PCI address window > > + * @map_size: size of allocated chunk in window > > + * @page_size: size of each page > > + */ > > +struct pci_epc_mem_window { > > + phys_addr_t phys_base; > > + size_t size; > > + size_t map_size; > > mem->bitmap already holds the map_size information. This structure can > be used only for passing info about the mem windows from platform to the > core. So map_size shouldn't be required. > agreed will drop it. Cheers, --Prabhakar _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel