[PATCH 1/4] test-crypto-perf: use virt2iova to get op IOVA address

[PATCH 1/4] test-crypto-perf: use virt2iova to get op IOVA address

[Anatoly Burakov]

Fixes: 2eb6a1a3e5fc ("app/crypto-perf: fix crypto op init")
Cc: pablo.de.lara.guarch@intel.com
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 app/test-crypto-perf/cperf_test_common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-crypto-perf/cperf_test_common.c b/app/test-crypto-perf/cperf_test_common.c
index 21cb1c2..bd71ac3 100644
--- a/app/test-crypto-perf/cperf_test_common.c
+++ b/app/test-crypto-perf/cperf_test_common.c
@@ -91,7 +91,7 @@ mempool_obj_init(struct rte_mempool *mp,
 	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
 	op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
 	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
-	op->phys_addr = rte_mem_virt2phy(obj);
+	op->phys_addr = rte_mem_virt2iova(obj);
 	op->mempool = mp;
 
 	/* Set source buffer */
-- 
2.7.4

[PATCH 2/4] eal: do not use physical addresses in IOVA as VA mode

[Anatoly Burakov]

We already use VA addresses for IOVA purposes everywhere if we're in
RTE_IOVA_VA mode:
 1) rte_malloc_virt2phy()/rte_malloc_virt2iova() always return VA addresses
 2) Because of 1), memzone's IOVA is set to VA address on reserve
 3) Because of 2), mempool's IOVA addresses are set to VA addresses

The only place where actual physical addresses are stored is in memsegs at
init time, but we're not using them anywhere, and there is no external API
to get those addresses (aside from manually iterating through memsegs), nor
should anyone care about them in RTE_IOVA_VA mode.

So, fix EAL initialization to allocate VA-contiguous segments at the start
without regard for physical addresses (as if they weren't available), and
use VA to set final IOVA addresses for all pages.

Fixes: 62196f4e0941 ("mem: rename address mapping function to IOVA")
Cc: thomas@monjalon.net
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_memory.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index 38853b7..ecf375b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -473,6 +473,9 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
 			hugepg_tbl[i].orig_va = virtaddr;
 		}
 		else {
+			/* rewrite physical addresses in IOVA as VA mode */
+			if (rte_eal_iova_mode() == RTE_IOVA_VA)
+				hugepg_tbl[i].physaddr = (uintptr_t)virtaddr;
 			hugepg_tbl[i].final_va = virtaddr;
 		}
 
@@ -1091,7 +1094,8 @@ rte_eal_hugepage_init(void)
 				continue;
 		}
 
-		if (phys_addrs_available) {
+		if (phys_addrs_available &&
+				rte_eal_iova_mode() != RTE_IOVA_VA) {
 			/* find physical addresses for each hugepage */
 			if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0) {
 				RTE_LOG(DEBUG, EAL, "Failed to find phys addr "
-- 
2.7.4

Re: [PATCH 2/4] eal: do not use physical addresses in IOVA as VA mode

[santosh]

On Sunday 01 April 2018 05:56 PM, Anatoly Burakov wrote:
> We already use VA addresses for IOVA purposes everywhere if we're in
> RTE_IOVA_VA mode:
>  1) rte_malloc_virt2phy()/rte_malloc_virt2iova() always return VA addresses
>  2) Because of 1), memzone's IOVA is set to VA address on reserve
>  3) Because of 2), mempool's IOVA addresses are set to VA addresses
>
> The only place where actual physical addresses are stored is in memsegs at
> init time, but we're not using them anywhere, and there is no external API
> to get those addresses (aside from manually iterating through memsegs), nor
> should anyone care about them in RTE_IOVA_VA mode.
>
> So, fix EAL initialization to allocate VA-contiguous segments at the start
> without regard for physical addresses (as if they weren't available), and
> use VA to set final IOVA addresses for all pages.
>
> Fixes: 62196f4e0941 ("mem: rename address mapping function to IOVA")
> Cc: thomas@monjalon.net
> Cc: stable@dpdk.org
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>  lib/librte_eal/linuxapp/eal/eal_memory.c | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
> index 38853b7..ecf375b 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
> @@ -473,6 +473,9 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
>  			hugepg_tbl[i].orig_va = virtaddr;
>  		}
>  		else {
> +			/* rewrite physical addresses in IOVA as VA mode */
> +			if (rte_eal_iova_mode() == RTE_IOVA_VA)
> +				hugepg_tbl[i].physaddr = (uintptr_t)virtaddr;
>  			hugepg_tbl[i].final_va = virtaddr;
>  		}
>  
> @@ -1091,7 +1094,8 @@ rte_eal_hugepage_init(void)
>  				continue;
>  		}
>  
> -		if (phys_addrs_available) {
> +		if (phys_addrs_available &&
> +				rte_eal_iova_mode() != RTE_IOVA_VA) {

Also can be done like below:
if (phys_addrs_available)
	/* find physical addresses for each hugepage */
	if (find_iovaaddrs(&tmp_hp[hp_offset], hpi) < 0) {

such that;
find_iovaaddrs() --> rte_mem_virt2iova().

That way avoid iova check in above if loop.
does that make sense?
Thanks.
[...]

Re: [PATCH 2/4] eal: do not use physical addresses in IOVA as VA mode

[Burakov, Anatoly]

On 02-Apr-18 6:35 AM, santosh wrote:
> 
> On Sunday 01 April 2018 05:56 PM, Anatoly Burakov wrote:
>> We already use VA addresses for IOVA purposes everywhere if we're in
>> RTE_IOVA_VA mode:
>>   1) rte_malloc_virt2phy()/rte_malloc_virt2iova() always return VA addresses
>>   2) Because of 1), memzone's IOVA is set to VA address on reserve
>>   3) Because of 2), mempool's IOVA addresses are set to VA addresses
>>
>> The only place where actual physical addresses are stored is in memsegs at
>> init time, but we're not using them anywhere, and there is no external API
>> to get those addresses (aside from manually iterating through memsegs), nor
>> should anyone care about them in RTE_IOVA_VA mode.
>>
>> So, fix EAL initialization to allocate VA-contiguous segments at the start
>> without regard for physical addresses (as if they weren't available), and
>> use VA to set final IOVA addresses for all pages.
>>
>> Fixes: 62196f4e0941 ("mem: rename address mapping function to IOVA")
>> Cc: thomas@monjalon.net
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>>   lib/librte_eal/linuxapp/eal/eal_memory.c | 6 +++++-
>>   1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
>> index 38853b7..ecf375b 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
>> @@ -473,6 +473,9 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
>>   			hugepg_tbl[i].orig_va = virtaddr;
>>   		}
>>   		else {
>> +			/* rewrite physical addresses in IOVA as VA mode */
>> +			if (rte_eal_iova_mode() == RTE_IOVA_VA)
>> +				hugepg_tbl[i].physaddr = (uintptr_t)virtaddr;
>>   			hugepg_tbl[i].final_va = virtaddr;
>>   		}
>>   
>> @@ -1091,7 +1094,8 @@ rte_eal_hugepage_init(void)
>>   				continue;
>>   		}
>>   
>> -		if (phys_addrs_available) {
>> +		if (phys_addrs_available &&
>> +				rte_eal_iova_mode() != RTE_IOVA_VA) {
> 
> Also can be done like below:
> if (phys_addrs_available)
> 	/* find physical addresses for each hugepage */
> 	if (find_iovaaddrs(&tmp_hp[hp_offset], hpi) < 0) {
> 
> such that;
> find_iovaaddrs() --> rte_mem_virt2iova().
> 
> That way avoid iova check in above if loop.
> does that make sense?
> Thanks.
> [...]
> 

Hi,

That was the initial implementation, however it doesn't work because we 
do two mappings, original and final, and physical addresses are found 
during original mappings (meaning, their VA's will be all over the 
place). We are interested in final VA as IOVA (after all of the sorting 
and figuring out which segments are contiguous), hence the current 
implementation.

-- 
Thanks,
Anatoly

Re: [PATCH 2/4] eal: do not use physical addresses in IOVA as VA mode

[santosh]

On Monday 02 April 2018 03:32 PM, Burakov, Anatoly wrote:
> On 02-Apr-18 6:35 AM, santosh wrote:
>>
>> On Sunday 01 April 2018 05:56 PM, Anatoly Burakov wrote:
>>> We already use VA addresses for IOVA purposes everywhere if we're in
>>> RTE_IOVA_VA mode:
>>>   1) rte_malloc_virt2phy()/rte_malloc_virt2iova() always return VA addresses
>>>   2) Because of 1), memzone's IOVA is set to VA address on reserve
>>>   3) Because of 2), mempool's IOVA addresses are set to VA addresses
>>>
>>> The only place where actual physical addresses are stored is in memsegs at
>>> init time, but we're not using them anywhere, and there is no external API
>>> to get those addresses (aside from manually iterating through memsegs), nor
>>> should anyone care about them in RTE_IOVA_VA mode.
>>>
>>> So, fix EAL initialization to allocate VA-contiguous segments at the start
>>> without regard for physical addresses (as if they weren't available), and
>>> use VA to set final IOVA addresses for all pages.
>>>
>>> Fixes: 62196f4e0941 ("mem: rename address mapping function to IOVA")
>>> Cc: thomas@monjalon.net
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>> ---
>>>   lib/librte_eal/linuxapp/eal/eal_memory.c | 6 +++++-
>>>   1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
>>> index 38853b7..ecf375b 100644
>>> --- a/lib/librte_eal/linuxapp/eal/eal_memory.c
>>> +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
>>> @@ -473,6 +473,9 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
>>>               hugepg_tbl[i].orig_va = virtaddr;
>>>           }
>>>           else {
>>> +            /* rewrite physical addresses in IOVA as VA mode */
>>> +            if (rte_eal_iova_mode() == RTE_IOVA_VA)
>>> +                hugepg_tbl[i].physaddr = (uintptr_t)virtaddr;
>>>               hugepg_tbl[i].final_va = virtaddr;
>>>           }
>>>   @@ -1091,7 +1094,8 @@ rte_eal_hugepage_init(void)
>>>                   continue;
>>>           }
>>>   -        if (phys_addrs_available) {
>>> +        if (phys_addrs_available &&
>>> +                rte_eal_iova_mode() != RTE_IOVA_VA) {
>>
>> Also can be done like below:
>> if (phys_addrs_available)
>>     /* find physical addresses for each hugepage */
>>     if (find_iovaaddrs(&tmp_hp[hp_offset], hpi) < 0) {
>>
>> such that;
>> find_iovaaddrs() --> rte_mem_virt2iova().
>>
>> That way avoid iova check in above if loop.
>> does that make sense?
>> Thanks.
>> [...]
>>
>
> Hi,
>
> That was the initial implementation, however it doesn't work because we do two mappings, original and final, and physical addresses are found during original mappings (meaning, their VA's will be all over the place). We are interested in final VA as IOVA (after all of the sorting and figuring out which segments are contiguous), hence the current implementation.
>
Ok.

Whole series,
Acked-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>

[PATCH 3/4] fslmc: do not needlessly check for IOVA mode

[Anatoly Burakov]

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 drivers/bus/fslmc/fslmc_vfio.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c
index e840ad6..1310190 100644
--- a/drivers/bus/fslmc/fslmc_vfio.c
+++ b/drivers/bus/fslmc/fslmc_vfio.c
@@ -223,10 +223,7 @@ int rte_fslmc_vfio_dmamap(void)
 		dma_map.size = memseg[i].len;
 		dma_map.vaddr = memseg[i].addr_64;
 #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = memseg[i].iova;
+		dma_map.iova = memseg[i].iova;
 #else
 		dma_map.iova = dma_map.vaddr;
 #endif
-- 
2.7.4

Re: [PATCH 3/4] fslmc: do not needlessly check for IOVA mode

[Hemant Agrawal]

On 4/1/2018 5:56 PM, Anatoly Burakov wrote:
> Cc: stable@dpdk.org
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>   drivers/bus/fslmc/fslmc_vfio.c | 5 +----
>   1 file changed, 1 insertion(+), 4 deletions(-)
>
> diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c
> index e840ad6..1310190 100644
> --- a/drivers/bus/fslmc/fslmc_vfio.c
> +++ b/drivers/bus/fslmc/fslmc_vfio.c
> @@ -223,10 +223,7 @@ int rte_fslmc_vfio_dmamap(void)
>   		dma_map.size = memseg[i].len;
>   		dma_map.vaddr = memseg[i].addr_64;
>   #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA
> -		if (rte_eal_iova_mode() == RTE_IOVA_VA)
> -			dma_map.iova = dma_map.vaddr;
> -		else
> -			dma_map.iova = memseg[i].iova;
> +		dma_map.iova = memseg[i].iova;
>   #else
>   		dma_map.iova = dma_map.vaddr;
>   #endif
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

[PATCH 4/4] vfio: do not needlessly check for IOVA mode

[Anatoly Burakov]

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c
index e44ae4d..2421d51 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c
@@ -681,10 +681,7 @@ vfio_type1_dma_map(int vfio_container_fd)
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = ms[i].addr_64;
 		dma_map.size = ms[i].len;
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = ms[i].iova;
+		dma_map.iova = ms[i].iova;
 		dma_map.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE;
 
 		ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA, &dma_map);
@@ -784,10 +781,7 @@ vfio_spapr_dma_map(int vfio_container_fd)
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = ms[i].addr_64;
 		dma_map.size = ms[i].len;
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = ms[i].iova;
+		dma_map.iova = ms[i].iova;
 		dma_map.flags = VFIO_DMA_MAP_FLAG_READ |
 				 VFIO_DMA_MAP_FLAG_WRITE;
 
-- 
2.7.4

Re: [dpdk-stable] [PATCH 4/4] vfio: do not needlessly check for IOVA mode

[Thomas Monjalon]

01/04/2018 14:26, Anatoly Burakov:
> Cc: stable@dpdk.org
> 
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

It deserves an explanation.

Please Anatoly, think about commit explanations more often.

Re: [dpdk-stable] [PATCH 4/4] vfio: do not needlessly check for IOVA mode

[Burakov, Anatoly]

On 02-Apr-18 5:29 PM, Thomas Monjalon wrote:
> 01/04/2018 14:26, Anatoly Burakov:
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
> It deserves an explanation.
> 
> Please Anatoly, think about commit explanations more often.
> 

I was under the impression that from context of previous patches, it 
would be obvious?

But OK, i'll respin.

-- 
Thanks,
Anatoly

Re: [dpdk-stable] [PATCH 4/4] vfio: do not needlessly check for IOVA mode

[Thomas Monjalon]

02/04/2018 19:12, Burakov, Anatoly:
> On 02-Apr-18 5:29 PM, Thomas Monjalon wrote:
> > 01/04/2018 14:26, Anatoly Burakov:
> >> Cc: stable@dpdk.org
> >>
> >> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> > 
> > It deserves an explanation.
> > 
> > Please Anatoly, think about commit explanations more often.
> > 
> 
> I was under the impression that from context of previous patches, it 
> would be obvious?

When we'll check this file in the git history, the context
of previous patch won't be so useful.

> But OK, i'll respin.

Thanks, it's important to consider each patch individually.

Re: [PATCH 1/4] test-crypto-perf: use virt2iova to get op IOVA address

[De Lara Guarch, Pablo]

> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Sunday, April 1, 2018 1:26 PM
> To: dev@dpdk.org
> Cc: Doherty, Declan <declan.doherty@intel.com>; thomas@monjalon.net;
> santosh.shukla@caviumnetworks.com; De Lara Guarch, Pablo
> <pablo.de.lara.guarch@intel.com>; stable@dpdk.org
> Subject: [PATCH 1/4] test-crypto-perf: use virt2iova to get op IOVA address
> 

Title should be "app/crypto-perf", rest is fine.

Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>

[PATCH v2 1/4] app/crypto-perf: use virt2iova to get op IOVA address

[Anatoly Burakov]

IOVA addresses should be found by calling rte_virt2iova() as
opposed to rte_virt2phy(), as physical address may not be
equal to IOVA address.

Fixes: 2eb6a1a3e5fc ("app/crypto-perf: fix crypto op init")
Cc: pablo.de.lara.guarch@intel.com
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
 app/test-crypto-perf/cperf_test_common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-crypto-perf/cperf_test_common.c b/app/test-crypto-perf/cperf_test_common.c
index 21cb1c2..bd71ac3 100644
--- a/app/test-crypto-perf/cperf_test_common.c
+++ b/app/test-crypto-perf/cperf_test_common.c
@@ -91,7 +91,7 @@ mempool_obj_init(struct rte_mempool *mp,
 	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
 	op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
 	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
-	op->phys_addr = rte_mem_virt2phy(obj);
+	op->phys_addr = rte_mem_virt2iova(obj);
 	op->mempool = mp;
 
 	/* Set source buffer */
-- 
2.7.4

[PATCH v3 1/4] app/crypto-perf: use virt2iova to get op IOVA address

[Anatoly Burakov]

IOVA addresses should be found by calling rte_virt2iova() as
opposed to rte_virt2phy(), as physical address may not be
equal to IOVA address.

Fixes: 2eb6a1a3e5fc ("app/crypto-perf: fix crypto op init")
Cc: pablo.de.lara.guarch@intel.com
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
---

Notes:
    v3:
    - Added missing ack from Santosh
    v2:
    - Add more detailed commit message

 app/test-crypto-perf/cperf_test_common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test-crypto-perf/cperf_test_common.c b/app/test-crypto-perf/cperf_test_common.c
index 21cb1c2..bd71ac3 100644
--- a/app/test-crypto-perf/cperf_test_common.c
+++ b/app/test-crypto-perf/cperf_test_common.c
@@ -91,7 +91,7 @@ mempool_obj_init(struct rte_mempool *mp,
 	op->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
 	op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
 	op->sess_type = RTE_CRYPTO_OP_WITH_SESSION;
-	op->phys_addr = rte_mem_virt2phy(obj);
+	op->phys_addr = rte_mem_virt2iova(obj);
 	op->mempool = mp;
 
 	/* Set source buffer */
-- 
2.7.4

[PATCH v3 2/4] eal: do not use physical addresses in IOVA as VA mode

[Anatoly Burakov]

We already use VA addresses for IOVA purposes everywhere if we're in
RTE_IOVA_VA mode:
 1) rte_malloc_virt2phy()/rte_malloc_virt2iova() always return VA addresses
 2) Because of 1), memzone's IOVA is set to VA address on reserve
 3) Because of 2), mempool's IOVA addresses are set to VA addresses

The only place where actual physical addresses are stored is in memsegs at
init time, but we're not using them anywhere, and there is no external API
to get those addresses (aside from manually iterating through memsegs), nor
should anyone care about them in RTE_IOVA_VA mode.

So, fix EAL initialization to allocate VA-contiguous segments at the start
without regard for physical addresses (as if they weren't available), and
use VA to set final IOVA addresses for all pages.

Fixes: 62196f4e0941 ("mem: rename address mapping function to IOVA")
Cc: thomas@monjalon.net
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
---

Notes:
    v3:
    - Added missing ack from Santosh

 lib/librte_eal/linuxapp/eal/eal_memory.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index 38853b7..ecf375b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -473,6 +473,9 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
 			hugepg_tbl[i].orig_va = virtaddr;
 		}
 		else {
+			/* rewrite physical addresses in IOVA as VA mode */
+			if (rte_eal_iova_mode() == RTE_IOVA_VA)
+				hugepg_tbl[i].physaddr = (uintptr_t)virtaddr;
 			hugepg_tbl[i].final_va = virtaddr;
 		}
 
@@ -1091,7 +1094,8 @@ rte_eal_hugepage_init(void)
 				continue;
 		}
 
-		if (phys_addrs_available) {
+		if (phys_addrs_available &&
+				rte_eal_iova_mode() != RTE_IOVA_VA) {
 			/* find physical addresses for each hugepage */
 			if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0) {
 				RTE_LOG(DEBUG, EAL, "Failed to find phys addr "
-- 
2.7.4

[PATCH v3 3/4] fslmc: do not needlessly check for IOVA mode

[Anatoly Burakov]

We already set IOVA addresses of memsegs and memzones to VA
address during initialization, so we don't need to check
whether we're in RTE_IOVA_VA mode anywhere else.

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
---

Notes:
    v3:
    - Added missing ack from Santosh
    v2:
    - Added more detailed explanation for commit

 drivers/bus/fslmc/fslmc_vfio.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c
index e840ad6..1310190 100644
--- a/drivers/bus/fslmc/fslmc_vfio.c
+++ b/drivers/bus/fslmc/fslmc_vfio.c
@@ -223,10 +223,7 @@ int rte_fslmc_vfio_dmamap(void)
 		dma_map.size = memseg[i].len;
 		dma_map.vaddr = memseg[i].addr_64;
 #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = memseg[i].iova;
+		dma_map.iova = memseg[i].iova;
 #else
 		dma_map.iova = dma_map.vaddr;
 #endif
-- 
2.7.4

[PATCH v3 4/4] vfio: do not needlessly check for IOVA mode

[Anatoly Burakov]

We already set IOVA addresses of memsegs and memzones to VA
address during initialization, so we don't need to check
whether we're in RTE_IOVA_VA mode anywhere else.

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
---

Notes:
    v3:
    - Added missing ack by Santosh
    
    v2:
    - Added more detailed commit message

 lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c
index e44ae4d..2421d51 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c
@@ -681,10 +681,7 @@ vfio_type1_dma_map(int vfio_container_fd)
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = ms[i].addr_64;
 		dma_map.size = ms[i].len;
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = ms[i].iova;
+		dma_map.iova = ms[i].iova;
 		dma_map.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE;
 
 		ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA, &dma_map);
@@ -784,10 +781,7 @@ vfio_spapr_dma_map(int vfio_container_fd)
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = ms[i].addr_64;
 		dma_map.size = ms[i].len;
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = ms[i].iova;
+		dma_map.iova = ms[i].iova;
 		dma_map.flags = VFIO_DMA_MAP_FLAG_READ |
 				 VFIO_DMA_MAP_FLAG_WRITE;
 
-- 
2.7.4

Re: [dpdk-stable] [PATCH v3 4/4] vfio: do not needlessly check for IOVA mode

[Thomas Monjalon]

04/04/2018 16:40, Anatoly Burakov:
> We already set IOVA addresses of memsegs and memzones to VA
> address during initialization, so we don't need to check
> whether we're in RTE_IOVA_VA mode anywhere else.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> Acked-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>

Series applied, thanks

[PATCH v2 2/4] eal: do not use physical addresses in IOVA as VA mode

[Anatoly Burakov]

We already use VA addresses for IOVA purposes everywhere if we're in
RTE_IOVA_VA mode:
 1) rte_malloc_virt2phy()/rte_malloc_virt2iova() always return VA addresses
 2) Because of 1), memzone's IOVA is set to VA address on reserve
 3) Because of 2), mempool's IOVA addresses are set to VA addresses

The only place where actual physical addresses are stored is in memsegs at
init time, but we're not using them anywhere, and there is no external API
to get those addresses (aside from manually iterating through memsegs), nor
should anyone care about them in RTE_IOVA_VA mode.

So, fix EAL initialization to allocate VA-contiguous segments at the start
without regard for physical addresses (as if they weren't available), and
use VA to set final IOVA addresses for all pages.

Fixes: 62196f4e0941 ("mem: rename address mapping function to IOVA")
Cc: thomas@monjalon.net
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_memory.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index 38853b7..ecf375b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -473,6 +473,9 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi,
 			hugepg_tbl[i].orig_va = virtaddr;
 		}
 		else {
+			/* rewrite physical addresses in IOVA as VA mode */
+			if (rte_eal_iova_mode() == RTE_IOVA_VA)
+				hugepg_tbl[i].physaddr = (uintptr_t)virtaddr;
 			hugepg_tbl[i].final_va = virtaddr;
 		}
 
@@ -1091,7 +1094,8 @@ rte_eal_hugepage_init(void)
 				continue;
 		}
 
-		if (phys_addrs_available) {
+		if (phys_addrs_available &&
+				rte_eal_iova_mode() != RTE_IOVA_VA) {
 			/* find physical addresses for each hugepage */
 			if (find_physaddrs(&tmp_hp[hp_offset], hpi) < 0) {
 				RTE_LOG(DEBUG, EAL, "Failed to find phys addr "
-- 
2.7.4

[PATCH v2 3/4] fslmc: do not needlessly check for IOVA mode

[Anatoly Burakov]

We already set IOVA addresses of memsegs and memzones to VA
address during initialization, so we don't need to check
whether we're in RTE_IOVA_VA mode anywhere else.

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
---
 drivers/bus/fslmc/fslmc_vfio.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c
index e840ad6..1310190 100644
--- a/drivers/bus/fslmc/fslmc_vfio.c
+++ b/drivers/bus/fslmc/fslmc_vfio.c
@@ -223,10 +223,7 @@ int rte_fslmc_vfio_dmamap(void)
 		dma_map.size = memseg[i].len;
 		dma_map.vaddr = memseg[i].addr_64;
 #ifdef RTE_LIBRTE_DPAA2_USE_PHYS_IOVA
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = memseg[i].iova;
+		dma_map.iova = memseg[i].iova;
 #else
 		dma_map.iova = dma_map.vaddr;
 #endif
-- 
2.7.4

[PATCH v2 4/4] vfio: do not needlessly check for IOVA mode

[Anatoly Burakov]

We already set IOVA addresses of memsegs and memzones to VA
address during initialization, so we don't need to check
whether we're in RTE_IOVA_VA mode anywhere else.

Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_vfio.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c
index e44ae4d..2421d51 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.c
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c
@@ -681,10 +681,7 @@ vfio_type1_dma_map(int vfio_container_fd)
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = ms[i].addr_64;
 		dma_map.size = ms[i].len;
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = ms[i].iova;
+		dma_map.iova = ms[i].iova;
 		dma_map.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE;
 
 		ret = ioctl(vfio_container_fd, VFIO_IOMMU_MAP_DMA, &dma_map);
@@ -784,10 +781,7 @@ vfio_spapr_dma_map(int vfio_container_fd)
 		dma_map.argsz = sizeof(struct vfio_iommu_type1_dma_map);
 		dma_map.vaddr = ms[i].addr_64;
 		dma_map.size = ms[i].len;
-		if (rte_eal_iova_mode() == RTE_IOVA_VA)
-			dma_map.iova = dma_map.vaddr;
-		else
-			dma_map.iova = ms[i].iova;
+		dma_map.iova = ms[i].iova;
 		dma_map.flags = VFIO_DMA_MAP_FLAG_READ |
 				 VFIO_DMA_MAP_FLAG_WRITE;
 
-- 
2.7.4