Re: [PATCH v10 08/22] IB/hns: Add icm support

["Wei Hu (Xavier)" <xavier.huwei@huawei.com>]

On 2016/6/20 14:06, Leon Romanovsky wrote:
> On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
>>
>> On 2016/6/17 17:58, Leon Romanovsky wrote:
>>> On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
>>>> This patch mainly added icm support for RoCE. It initializes icm
>>>> which managers the relative memory blocks for RoCE. The data
>>>> structures of RoCE will be located in it. For example, CQ table,
>>>> QP table and MTPT table so on.
>>>>
>>>> Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
>>>> Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
>>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>>> ---
>>> <...>
>>>
>>>> +
>>>> +static int hns_roce_alloc_icm_pages(struct scatterlist *mem, int order,
>>>> +				    gfp_t gfp_mask)
>>>> +{
>>>> +	struct page *page;
>>>> +
>>>> +	page = alloc_pages(gfp_mask, order);
>>>> +	if (!page)
>>>> +		return -ENOMEM;
>>>> +
>>>> +	sg_set_page(mem, page, PAGE_SIZE << order, 0);
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>> +static int hns_roce_alloc_icm_coherent(struct device *dev,
>>>> +				       struct scatterlist *mem, int order,
>>>> +				       gfp_t gfp_mask)
>>>> +{
>>>> +	void *buf = dma_alloc_coherent(dev, PAGE_SIZE << order,
>>>> +				       &sg_dma_address(mem), gfp_mask);
>>>> +	if (!buf)
>>>> +		return -ENOMEM;
>>>> +
>>>> +	sg_set_buf(mem, buf, PAGE_SIZE << order);
>>>> +	WARN_ON(mem->offset);
>>>> +	sg_dma_len(mem) = PAGE_SIZE << order;
>>>> +	return 0;
>>>> +}
>>>> +
>>> <...>
>>>
>>>> +
>>>> +static void hns_roce_free_icm_pages(struct hns_roce_dev *hr_dev,
>>>> +				    struct hns_roce_icm_chunk *chunk)
>>>> +{
>>>> +	int i;
>>>> +
>>>> +	if (chunk->nsg > 0)
>>>> +		dma_unmap_sg(&hr_dev->pdev->dev, chunk->mem, chunk->npages,
>>>> +			     DMA_BIDIRECTIONAL);
>>>> +
>>>> +	for (i = 0; i < chunk->npages; ++i)
>>>> +		__free_pages(sg_page(&chunk->mem[i]),
>>>> +			     get_order(chunk->mem[i].length));
>>> You used alloc_pages for this allocation, so why are you using
>>> __free_pages instead of free_pages?
>> Hi, Leon
>>      The function prototype of these functions as below：
>>          static inline struct page * alloc_pages(gfp_t gfp_mask, unsigned int
>> order);
>>          void free_pages(unsigned long addr, unsigned int order);
>>          void __free_pages(struct page *page, unsigned int order);
>>
>>      The type of the first parameter of free_pages is same with the type of
>> return value of alloc_pages.
>>      Maybe it is better to call __free_pages to release memory that allocated
>> by calling alloc_pages.
> OK, I see.
>
> Another question which you didn't answer [1].
>
> "I wonder if you have the same needs for ICM as it is in mlx4 device.
> Do you have firmware?"
>
> [1] http://marc.info/?l=linux-rdma&m=146545553104913&w=2
Hi, Leon
     Now we haven't firmware.
     But hardware still need memory for QPC\CQC\MTPT\mtt etc.

Thanks
Wei Hu
>> Regards
>> Wei Hu
>>