[PATCH] doc: add driver limitation for vhost dequeue zero copy

[PATCH] doc: add driver limitation for vhost dequeue zero copy

[Junjie Chen]

In vhost-switch example, when binding nic to vfio-pci, dequeue zero
copy cannot work in VM2NIC mode due to no iommu dma mapping is setup
for guest memory currently.

Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
---
 doc/guides/sample_app_ug/vhost.rst | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
index a4bdc6a..1591a31 100644
--- a/doc/guides/sample_app_ug/vhost.rst
+++ b/doc/guides/sample_app_ug/vhost.rst
@@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
 default value is 15.
 
 **--dequeue-zero-copy**
-Dequeue zero copy will be enabled when this option is given.
+Dequeue zero copy will be enabled when this option is given, it is worth to
+note that if NIC is binded to vfio-pci driver, dequeue zero copy cannot work 
+at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu dma mapping for
+guest memory.
 
 **--vlan-strip 0|1**
 VLAN strip option is removed, because different NICs have different behaviors
-- 
2.0.1

Re: [PATCH] doc: add driver limitation for vhost dequeue zero copy

[Tan, Jianfeng]

Hi Junjie,

> -----Original Message-----
> From: Chen, Junjie J
> Sent: Saturday, February 24, 2018 10:44 PM
> To: yliu@fridaylinux.org; maxime.coquelin@redhat.com; Tan, Jianfeng
> Cc: dev@dpdk.org; Chen, Junjie J
> Subject: [PATCH] doc: add driver limitation for vhost dequeue zero copy
> 
> In vhost-switch example, when binding nic to vfio-pci, dequeue zero
> copy cannot work in VM2NIC mode due to no iommu dma mapping is setup
> for guest memory currently.

IMO, it's a limitation that can be fixed. Do you want to try fixing it?

> 
> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> ---
>  doc/guides/sample_app_ug/vhost.rst | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/sample_app_ug/vhost.rst
> b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..1591a31 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst


Besides, we might also need to put such limitation at http://dpdk.org/doc/guides/prog_guide/vhost_lib.html

Thanks,
Jianfeng


> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx
> retry is enabled. The
>  default value is 15.
> 
>  **--dequeue-zero-copy**
> -Dequeue zero copy will be enabled when this option is given.
> +Dequeue zero copy will be enabled when this option is given, it is worth to
> +note that if NIC is binded to vfio-pci driver, dequeue zero copy cannot work
> +at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu dma
> mapping for
> +guest memory.
> 
>  **--vlan-strip 0|1**
>  VLAN strip option is removed, because different NICs have different
> behaviors
> --
> 2.0.1

[PATCH v2] doc: add driver limitation for vhost dequeue zero copy

[Junjie Chen]

In vhost-switch example, when binding nic to vfio-pci, dequeue zero
copy cannot work in VM2NIC mode due to no iommu dma mapping is setup
for guest memory currently.

Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
---
Changes in V2:
 - add doc in vhost lib

 doc/guides/prog_guide/vhost_lib.rst | 3 +++
 doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 18227b6..bdf77d6 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -83,6 +83,9 @@ The following is an overview of some key Vhost API functions:
       of those segments, thus the fewer the segments, the quicker we will get
       the mapping. NOTE: we may speed it by using tree searching in future.
 
+    * zero copy does not work when using vfio-pci driver currently, this is
+      because we don't setup iommu dma mapping for guest memory.
+
   - ``RTE_VHOST_USER_IOMMU_SUPPORT``
 
     IOMMU support will be enabled when this flag is set. It is disabled by
diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
index a4bdc6a..840c1fd 100644
--- a/doc/guides/sample_app_ug/vhost.rst
+++ b/doc/guides/sample_app_ug/vhost.rst
@@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
 default value is 15.
 
 **--dequeue-zero-copy**
-Dequeue zero copy will be enabled when this option is given.
+Dequeue zero copy will be enabled when this option is given, it is worth to
+note that if NIC is binded to vfio-pci driver, dequeue zero copy cannot work
+at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu dma mapping for
+guest memory.
 
 **--vlan-strip 0|1**
 VLAN strip option is removed, because different NICs have different behaviors
-- 
2.0.1

Re: [PATCH v2] doc: add driver limitation for vhost dequeue zero copy

[Maxime Coquelin]

On 02/27/2018 10:21 AM, Junjie Chen wrote:
> In vhost-switch example, when binding nic to vfio-pci, dequeue zero
> copy cannot work in VM2NIC mode due to no iommu dma mapping is setup
> for guest memory currently.
> 
> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> ---
> Changes in V2:
>   - add doc in vhost lib
> 
>   doc/guides/prog_guide/vhost_lib.rst | 3 +++
>   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
>   2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
> index 18227b6..bdf77d6 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -83,6 +83,9 @@ The following is an overview of some key Vhost API functions:
>         of those segments, thus the fewer the segments, the quicker we will get
>         the mapping. NOTE: we may speed it by using tree searching in future.
>   
> +    * zero copy does not work when using vfio-pci driver currently, this is
> +      because we don't setup iommu dma mapping for guest memory.
> +

I guess that it should work with vfio-pci in noiommu mode? Maybe worth
to clarify.

>     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
>   
>       IOMMU support will be enabled when this flag is set. It is disabled by
> diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..840c1fd 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst
> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
>   default value is 15.
>   
>   **--dequeue-zero-copy**
> -Dequeue zero copy will be enabled when this option is given.
> +Dequeue zero copy will be enabled when this option is given, it is worth to
> +note that if NIC is binded to vfio-pci driver, dequeue zero copy cannot work
> +at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu dma mapping for
> +guest memory.
>   
>   **--vlan-strip 0|1**
>   VLAN strip option is removed, because different NICs have different behaviors
> 

Re: [PATCH v2] doc: add driver limitation for vhost dequeue zero copy

[Chen, Junjie J]

> -----Original Message-----
> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Tuesday, March 6, 2018 4:57 PM
> To: Chen, Junjie J <junjie.j.chen@intel.com>; yliu@fridaylinux.org; Tan,
> Jianfeng <jianfeng.tan@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH v2] doc: add driver limitation for vhost dequeue zero copy
> 
> 
> 
> On 02/27/2018 10:21 AM, Junjie Chen wrote:
> > In vhost-switch example, when binding nic to vfio-pci, dequeue zero
> > copy cannot work in VM2NIC mode due to no iommu dma mapping is setup
> > for guest memory currently.
> >
> > Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> > ---
> > Changes in V2:
> >   - add doc in vhost lib
> >
> >   doc/guides/prog_guide/vhost_lib.rst | 3 +++
> >   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
> >   2 files changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/guides/prog_guide/vhost_lib.rst
> > b/doc/guides/prog_guide/vhost_lib.rst
> > index 18227b6..bdf77d6 100644
> > --- a/doc/guides/prog_guide/vhost_lib.rst
> > +++ b/doc/guides/prog_guide/vhost_lib.rst
> > @@ -83,6 +83,9 @@ The following is an overview of some key Vhost API
> functions:
> >         of those segments, thus the fewer the segments, the quicker we
> will get
> >         the mapping. NOTE: we may speed it by using tree searching in
> future.
> >
> > +    * zero copy does not work when using vfio-pci driver currently, this is
> > +      because we don't setup iommu dma mapping for guest memory.
> > +
> 
> I guess that it should work with vfio-pci in noiommu mode? Maybe worth to
> clarify.
You are right, it works on noiommu mode. Updated patch accordingly. 
> 
> >     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
> >
> >       IOMMU support will be enabled when this flag is set. It is
> > disabled by diff --git a/doc/guides/sample_app_ug/vhost.rst
> > b/doc/guides/sample_app_ug/vhost.rst
> > index a4bdc6a..840c1fd 100644
> > --- a/doc/guides/sample_app_ug/vhost.rst
> > +++ b/doc/guides/sample_app_ug/vhost.rst
> > @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx
> retry is enabled. The
> >   default value is 15.
> >
> >   **--dequeue-zero-copy**
> > -Dequeue zero copy will be enabled when this option is given.
> > +Dequeue zero copy will be enabled when this option is given, it is
> > +worth to note that if NIC is binded to vfio-pci driver, dequeue zero
> > +copy cannot work at VM2NIC mode (vm2vm=0) due to currently we don't
> > +setup iommu dma mapping for guest memory.
> >
> >   **--vlan-strip 0|1**
> >   VLAN strip option is removed, because different NICs have different
> > behaviors
> >

[PATCH v3] doc: add driver limitation for vhost dequeue zero copy

[Junjie Chen]

In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
is setup for guest memory currently.

Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
---
Changes in V3:
- update limitation to iommu
Changes in V2:
- add doc in vhost lib

 doc/guides/prog_guide/vhost_lib.rst | 5 +++++
 doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 18227b6..06d214f 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -83,6 +83,11 @@ The following is an overview of some key Vhost API functions:
       of those segments, thus the fewer the segments, the quicker we will get
       the mapping. NOTE: we may speed it by using tree searching in future.
 
+    * zero copy does not work when using driver with iommu mode currently, this
+      is because we don't setup iommu dma mapping for guest memory. For example,
+      when you bind device to vfio-pci driver, you need to set driver to work
+      in noiommu mode.
+
   - ``RTE_VHOST_USER_IOMMU_SUPPORT``
 
     IOMMU support will be enabled when this flag is set. It is disabled by
diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
index a4bdc6a..f0bb169 100644
--- a/doc/guides/sample_app_ug/vhost.rst
+++ b/doc/guides/sample_app_ug/vhost.rst
@@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
 default value is 15.
 
 **--dequeue-zero-copy**
-Dequeue zero copy will be enabled when this option is given.
+Dequeue zero copy will be enabled when this option is given. it is worth to
+note that if NIC is binded to driver with iommu enabled, dequeue zero copy
+cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu
+dma mapping for guest memory.
 
 **--vlan-strip 0|1**
 VLAN strip option is removed, because different NICs have different behaviors
-- 
2.0.1

Re: [PATCH v3] doc: add driver limitation for vhost dequeue zero copy

[Tan, Jianfeng]

On 3/9/2018 6:07 PM, Junjie Chen wrote:
> In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
> dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
> is setup for guest memory currently.
>
> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> ---
> Changes in V3:
> - update limitation to iommu
> Changes in V2:
> - add doc in vhost lib
>
>   doc/guides/prog_guide/vhost_lib.rst | 5 +++++
>   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
>   2 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
> index 18227b6..06d214f 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API functions:
>         of those segments, thus the fewer the segments, the quicker we will get
>         the mapping. NOTE: we may speed it by using tree searching in future.
>   
> +    * zero copy does not work when using driver with iommu mode currently, this

Considering FreeBSD driver nic_uio does not support iommu, we can make 
"driver" more explicitly. For example,

"driver with iommu mode" -> "vfio-pci with iommu mode"

> +      is because we don't setup iommu dma mapping for guest memory.

> For example, when you bind device to vfio-pci driver, you need to set driver to work in noiommu mode.

Reword the above sentence a little bit:
"If vfio-pci is a must in your case, insert vfio-pci kernel module in 
noiommu mode."

> +
>     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
>   
>       IOMMU support will be enabled when this flag is set. It is disabled by
> diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..f0bb169 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst
> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
>   default value is 15.
>   
>   **--dequeue-zero-copy**
> -Dequeue zero copy will be enabled when this option is given.
> +Dequeue zero copy will be enabled when this option is given. it is worth to
> +note that if NIC is binded to driver with iommu enabled, dequeue zero copy
> +cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu
> +dma mapping for guest memory.
>   
>   **--vlan-strip 0|1**
>   VLAN strip option is removed, because different NICs have different behaviors

Re: [PATCH] doc: add driver limitation for vhost dequeue zero copy

[Kovacevic, Marko]

> > Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> > ---
> >  doc/guides/sample_app_ug/vhost.rst | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/guides/sample_app_ug/vhost.rst
> > b/doc/guides/sample_app_ug/vhost.rst
> > index a4bdc6a..1591a31 100644
> > --- a/doc/guides/sample_app_ug/vhost.rst
> > +++ b/doc/guides/sample_app_ug/vhost.rst


Can you please supersede this patch and the V2 if you have sent a V3 to the mailing list.

Thanks

Re: [PATCH] doc: add driver limitation for vhost dequeue zero copy

[Chen, Junjie J]

Done, Thanks for reminder. 

Cheers
JJ

> -----Original Message-----
> From: Kovacevic, Marko
> Sent: Tuesday, March 13, 2018 5:13 PM
> To: Tan, Jianfeng <jianfeng.tan@intel.com>; Chen, Junjie J
> <junjie.j.chen@intel.com>; dev@dpdk.org
> Cc: yliu@fridaylinux.org; maxime.coquelin@redhat.com
> Subject: RE: [dpdk-dev] [PATCH] doc: add driver limitation for vhost dequeue
> zero copy
> 
> > > Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> > > ---
> > >  doc/guides/sample_app_ug/vhost.rst | 5 ++++-
> > >  1 file changed, 4 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/doc/guides/sample_app_ug/vhost.rst
> > > b/doc/guides/sample_app_ug/vhost.rst
> > > index a4bdc6a..1591a31 100644
> > > --- a/doc/guides/sample_app_ug/vhost.rst
> > > +++ b/doc/guides/sample_app_ug/vhost.rst
> 
> 
> Can you please supersede this patch and the V2 if you have sent a V3 to the
> mailing list.
> 
> Thanks

Re: [PATCH v3] doc: add driver limitation for vhost dequeue zero copy

[Chen, Junjie J]

Hi Jianfeng.

> -----Original Message-----
> From: Tan, Jianfeng
> Sent: Monday, March 12, 2018 11:15 AM
> To: Chen, Junjie J <junjie.j.chen@intel.com>; yliu@fridaylinux.org;
> maxime.coquein@redhat.com
> Cc: dev@dpdk.org
> Subject: Re: [PATCH v3] doc: add driver limitation for vhost dequeue zero copy
> 
> 
> 
> On 3/9/2018 6:07 PM, Junjie Chen wrote:
> > In vhost-switch example, when binding nic to vfio-pci with iommu
> > enabled, dequeue zero copy cannot work in VM2NIC mode due to no iommu
> > dma mapping is setup for guest memory currently.
> >
> > Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> > ---
> > Changes in V3:
> > - update limitation to iommu
> > Changes in V2:
> > - add doc in vhost lib
> >
> >   doc/guides/prog_guide/vhost_lib.rst | 5 +++++
> >   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
> >   2 files changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/guides/prog_guide/vhost_lib.rst
> > b/doc/guides/prog_guide/vhost_lib.rst
> > index 18227b6..06d214f 100644
> > --- a/doc/guides/prog_guide/vhost_lib.rst
> > +++ b/doc/guides/prog_guide/vhost_lib.rst
> > @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API
> functions:
> >         of those segments, thus the fewer the segments, the quicker we will
> get
> >         the mapping. NOTE: we may speed it by using tree searching in
> future.
> >
> > +    * zero copy does not work when using driver with iommu mode
> > + currently, this
> 
> Considering FreeBSD driver nic_uio does not support iommu, we can make
> "driver" more explicitly. For example,
> 
> "driver with iommu mode" -> "vfio-pci with iommu mode"
Will update.
> 
> > +      is because we don't setup iommu dma mapping for guest memory.
> 
> > For example, when you bind device to vfio-pci driver, you need to set driver to
> work in noiommu mode.
> 
> Reword the above sentence a little bit:
> "If vfio-pci is a must in your case, insert vfio-pci kernel module in noiommu
> mode."
Will update, thanks.

> > +
> >     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
> >
> >       IOMMU support will be enabled when this flag is set. It is
> > disabled by diff --git a/doc/guides/sample_app_ug/vhost.rst
> > b/doc/guides/sample_app_ug/vhost.rst
> > index a4bdc6a..f0bb169 100644
> > --- a/doc/guides/sample_app_ug/vhost.rst
> > +++ b/doc/guides/sample_app_ug/vhost.rst
> > @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx
> retry is enabled. The
> >   default value is 15.
> >
> >   **--dequeue-zero-copy**
> > -Dequeue zero copy will be enabled when this option is given.
> > +Dequeue zero copy will be enabled when this option is given. it is
> > +worth to note that if NIC is binded to driver with iommu enabled,
> > +dequeue zero copy cannot work at VM2NIC mode (vm2vm=0) due to
> > +currently we don't setup iommu dma mapping for guest memory.
> >
> >   **--vlan-strip 0|1**
> >   VLAN strip option is removed, because different NICs have different
> > behaviors

[PATCH v4] doc: add driver limitation for vhost dequeue zero copy

[Junjie Chen]

In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
is setup for guest memory currently.

Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
---
Changes in V4:
- make words more accurate
Changes in V3:
- update limitation to iommu
Changes in V2:
- add doc in vhost lib
 doc/guides/prog_guide/vhost_lib.rst | 5 +++++
 doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
index 18227b6..b49e36e 100644
--- a/doc/guides/prog_guide/vhost_lib.rst
+++ b/doc/guides/prog_guide/vhost_lib.rst
@@ -83,6 +83,11 @@ The following is an overview of some key Vhost API functions:
       of those segments, thus the fewer the segments, the quicker we will get
       the mapping. NOTE: we may speed it by using tree searching in future.
 
+    * zero copy can not work when using vfio-pci with iommu mode currently, this
+      is because we don't setup iommu dma mapping for guest memory. If you have
+      to use vfio-pci driver, please insert vfio-pci kernel module in noiommu
+      mode.
+
   - ``RTE_VHOST_USER_IOMMU_SUPPORT``
 
     IOMMU support will be enabled when this flag is set. It is disabled by
diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
index a4bdc6a..f0bb169 100644
--- a/doc/guides/sample_app_ug/vhost.rst
+++ b/doc/guides/sample_app_ug/vhost.rst
@@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
 default value is 15.
 
 **--dequeue-zero-copy**
-Dequeue zero copy will be enabled when this option is given.
+Dequeue zero copy will be enabled when this option is given. it is worth to
+note that if NIC is binded to driver with iommu enabled, dequeue zero copy
+cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu
+dma mapping for guest memory.
 
 **--vlan-strip 0|1**
 VLAN strip option is removed, because different NICs have different behaviors
-- 
2.0.1

Re: [PATCH v4] doc: add driver limitation for vhost dequeue zero copy

[Tan, Jianfeng]

> -----Original Message-----
> From: Chen, Junjie J
> Sent: Thursday, March 15, 2018 12:24 AM
> To: Tan, Jianfeng; maxime.coquelin@redhat.com; yliu@fridaylinux.org
> Cc: dev@dpdk.org; Chen, Junjie J
> Subject: [PATCH v4] doc: add driver limitation for vhost dequeue zero copy
> 
> In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
> dequeue zero copy cannot work in VM2NIC mode due to no iommu dma
> mapping
> is setup for guest memory currently.
> 
> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>

Reviewed-by: Jianfeng Tan <jianfeng.tan@intel.com>

Thanks!

> ---
> Changes in V4:
> - make words more accurate
> Changes in V3:
> - update limitation to iommu
> Changes in V2:
> - add doc in vhost lib
>  doc/guides/prog_guide/vhost_lib.rst | 5 +++++
>  doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
>  2 files changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/prog_guide/vhost_lib.rst
> b/doc/guides/prog_guide/vhost_lib.rst
> index 18227b6..b49e36e 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API
> functions:
>        of those segments, thus the fewer the segments, the quicker we will get
>        the mapping. NOTE: we may speed it by using tree searching in future.
> 
> +    * zero copy can not work when using vfio-pci with iommu mode currently,
> this
> +      is because we don't setup iommu dma mapping for guest memory. If
> you have
> +      to use vfio-pci driver, please insert vfio-pci kernel module in noiommu
> +      mode.
> +
>    - ``RTE_VHOST_USER_IOMMU_SUPPORT``
> 
>      IOMMU support will be enabled when this flag is set. It is disabled by
> diff --git a/doc/guides/sample_app_ug/vhost.rst
> b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..f0bb169 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst
> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx
> retry is enabled. The
>  default value is 15.
> 
>  **--dequeue-zero-copy**
> -Dequeue zero copy will be enabled when this option is given.
> +Dequeue zero copy will be enabled when this option is given. it is worth to
> +note that if NIC is binded to driver with iommu enabled, dequeue zero copy
> +cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup
> iommu
> +dma mapping for guest memory.
> 
>  **--vlan-strip 0|1**
>  VLAN strip option is removed, because different NICs have different
> behaviors
> --
> 2.0.1

Re: [PATCH v4] doc: add driver limitation for vhost dequeue zero copy

[Maxime Coquelin]

On 03/14/2018 05:24 PM, Junjie Chen wrote:
> In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
> dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
> is setup for guest memory currently.
> 
> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

John, as this is a doc patch for Vhost, does it go through the doc tree 
or through Virtio one?

Thanks,
Maxime
> ---
> Changes in V4:
> - make words more accurate
> Changes in V3:
> - update limitation to iommu
> Changes in V2:
> - add doc in vhost lib
>   doc/guides/prog_guide/vhost_lib.rst | 5 +++++
>   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
>   2 files changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/prog_guide/vhost_lib.rst b/doc/guides/prog_guide/vhost_lib.rst
> index 18227b6..b49e36e 100644
> --- a/doc/guides/prog_guide/vhost_lib.rst
> +++ b/doc/guides/prog_guide/vhost_lib.rst
> @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API functions:
>         of those segments, thus the fewer the segments, the quicker we will get
>         the mapping. NOTE: we may speed it by using tree searching in future.
>   
> +    * zero copy can not work when using vfio-pci with iommu mode currently, this
> +      is because we don't setup iommu dma mapping for guest memory. If you have
> +      to use vfio-pci driver, please insert vfio-pci kernel module in noiommu
> +      mode.
> +
>     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
>   
>       IOMMU support will be enabled when this flag is set. It is disabled by
> diff --git a/doc/guides/sample_app_ug/vhost.rst b/doc/guides/sample_app_ug/vhost.rst
> index a4bdc6a..f0bb169 100644
> --- a/doc/guides/sample_app_ug/vhost.rst
> +++ b/doc/guides/sample_app_ug/vhost.rst
> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when rx retry is enabled. The
>   default value is 15.
>   
>   **--dequeue-zero-copy**
> -Dequeue zero copy will be enabled when this option is given.
> +Dequeue zero copy will be enabled when this option is given. it is worth to
> +note that if NIC is binded to driver with iommu enabled, dequeue zero copy
> +cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu
> +dma mapping for guest memory.
>   
>   **--vlan-strip 0|1**
>   VLAN strip option is removed, because different NICs have different behaviors
> 

Re: [PATCH v4] doc: add driver limitation for vhost dequeue zero copy

[Maxime Coquelin]

On 03/29/2018 02:14 PM, Maxime Coquelin wrote:
> 
> 
> On 03/14/2018 05:24 PM, Junjie Chen wrote:
>> In vhost-switch example, when binding nic to vfio-pci with iommu enabled,
>> dequeue zero copy cannot work in VM2NIC mode due to no iommu dma mapping
>> is setup for guest memory currently.
>>
>> Signed-off-by: Junjie Chen <junjie.j.chen@intel.com>
> 
> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> 
> John, as this is a doc patch for Vhost, does it go through the doc tree 
> or through Virtio one?


Applied to dpdk-next-virtio/master

Thanks,
Maxime

> Thanks,
> Maxime
>> ---
>> Changes in V4:
>> - make words more accurate
>> Changes in V3:
>> - update limitation to iommu
>> Changes in V2:
>> - add doc in vhost lib
>>   doc/guides/prog_guide/vhost_lib.rst | 5 +++++
>>   doc/guides/sample_app_ug/vhost.rst  | 5 ++++-
>>   2 files changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/doc/guides/prog_guide/vhost_lib.rst 
>> b/doc/guides/prog_guide/vhost_lib.rst
>> index 18227b6..b49e36e 100644
>> --- a/doc/guides/prog_guide/vhost_lib.rst
>> +++ b/doc/guides/prog_guide/vhost_lib.rst
>> @@ -83,6 +83,11 @@ The following is an overview of some key Vhost API 
>> functions:
>>         of those segments, thus the fewer the segments, the quicker we 
>> will get
>>         the mapping. NOTE: we may speed it by using tree searching in 
>> future.
>> +    * zero copy can not work when using vfio-pci with iommu mode 
>> currently, this
>> +      is because we don't setup iommu dma mapping for guest memory. 
>> If you have
>> +      to use vfio-pci driver, please insert vfio-pci kernel module in 
>> noiommu
>> +      mode.
>> +
>>     - ``RTE_VHOST_USER_IOMMU_SUPPORT``
>>       IOMMU support will be enabled when this flag is set. It is 
>> disabled by
>> diff --git a/doc/guides/sample_app_ug/vhost.rst 
>> b/doc/guides/sample_app_ug/vhost.rst
>> index a4bdc6a..f0bb169 100644
>> --- a/doc/guides/sample_app_ug/vhost.rst
>> +++ b/doc/guides/sample_app_ug/vhost.rst
>> @@ -147,7 +147,10 @@ retries on an RX burst, it takes effect only when 
>> rx retry is enabled. The
>>   default value is 15.
>>   **--dequeue-zero-copy**
>> -Dequeue zero copy will be enabled when this option is given.
>> +Dequeue zero copy will be enabled when this option is given. it is 
>> worth to
>> +note that if NIC is binded to driver with iommu enabled, dequeue zero 
>> copy
>> +cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup 
>> iommu
>> +dma mapping for guest memory.
>>   **--vlan-strip 0|1**
>>   VLAN strip option is removed, because different NICs have different 
>> behaviors
>>