[PATCH v2] doc: update mlx5 flow count limitations

[PATCH v2] doc: update mlx5 flow count limitations

[Ori Kam]

Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
 doc/guides/nics/mlx5.rst |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index d24941a..7da5c87 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -104,7 +104,8 @@ Limitations
 -----------
 
 - Inner RSS for VXLAN frames is not supported yet.
-- Port statistics through software counters only.
+- Port statistics through software counters only. Flow statistics are
+  supported by hardware counters.
 - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
 - Forked secondary process not supported.
 - Flow pattern without any specific vlan will match for vlan packets as well:
@@ -127,6 +128,8 @@ Limitations
 - A multi segment packet must have less than 6 segments in case the Tx burst function
   is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
   less than 50 segments.
+- Count action for RTE flow is only supported in OFED 4.2.
+- Two identical rules with count action are not supported.
 
 Configuration
 -------------
-- 
1.7.1

Re: [PATCH v2] doc: update mlx5 flow count limitations

[Nélio Laranjeiro]

On Mon, Oct 16, 2017 at 11:32:39AM +0300, Ori Kam wrote:
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> ---
>  doc/guides/nics/mlx5.rst |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
> 
> diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> index d24941a..7da5c87 100644
> --- a/doc/guides/nics/mlx5.rst
> +++ b/doc/guides/nics/mlx5.rst
> @@ -104,7 +104,8 @@ Limitations
>  -----------
>  
>  - Inner RSS for VXLAN frames is not supported yet.
> -- Port statistics through software counters only.
> +- Port statistics through software counters only. Flow statistics are
> +  supported by hardware counters.
>  - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
>  - Forked secondary process not supported.
>  - Flow pattern without any specific vlan will match for vlan packets as well:
> @@ -127,6 +128,8 @@ Limitations
>  - A multi segment packet must have less than 6 segments in case the Tx burst function
>    is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
>    less than 50 segments.
> +- Count action for RTE flow is only supported in OFED 4.2.
> +- Two identical rules with count action are not supported.

This last line does not bring anything, having two identical rules
already create a collision which is not defined by the flow API.
In PMD situation is is even worse, as it will match both rules and
thus it will duplicate a packet.

There is no need to add a limit on an undefined behavior by the
API itself.

Thanks,

PS: when you send a new revision, please use the --thread and
--in-reply-to to link it with the previous one.

-- 
Nélio Laranjeiro
6WIND

Re: [PATCH v2] doc: update mlx5 flow count limitations

[Ori Kam]

Please see my comments,

> -----Original Message-----
> From: Nélio Laranjeiro [mailto:nelio.laranjeiro@6wind.com]
> Sent: Monday, October 16, 2017 2:18 PM
> To: Ori Kam <orika@mellanox.com>
> Cc: Adrien Mazarguil <adrien.mazarguil@6wind.com>; Yongseok Koh
> <yskoh@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH v2] doc: update mlx5 flow count limitations
> 
> On Mon, Oct 16, 2017 at 11:32:39AM +0300, Ori Kam wrote:
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> > ---
> >  doc/guides/nics/mlx5.rst |    5 ++++-
> >  1 files changed, 4 insertions(+), 1 deletions(-)
> >
> > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index
> > d24941a..7da5c87 100644
> > --- a/doc/guides/nics/mlx5.rst
> > +++ b/doc/guides/nics/mlx5.rst
> > @@ -104,7 +104,8 @@ Limitations
> >  -----------
> >
> >  - Inner RSS for VXLAN frames is not supported yet.
> > -- Port statistics through software counters only.
> > +- Port statistics through software counters only. Flow statistics are
> > +  supported by hardware counters.
> >  - Hardware checksum RX offloads for VXLAN inner header are not
> supported yet.
> >  - Forked secondary process not supported.
> >  - Flow pattern without any specific vlan will match for vlan packets as well:
> > @@ -127,6 +128,8 @@ Limitations
> >  - A multi segment packet must have less than 6 segments in case the Tx
> burst function
> >    is set to multi-packet send or Enhanced multi-packet send. Otherwise it
> must have
> >    less than 50 segments.
> > +- Count action for RTE flow is only supported in OFED 4.2.
> > +- Two identical rules with count action are not supported.
> 
> This last line does not bring anything, having two identical rules already
> create a collision which is not defined by the flow API.
> In PMD situation is is even worse, as it will match both rules and thus it will
> duplicate a packet.
> 
> There is no need to add a limit on an undefined behavior by the API itself.
> 

The thing is that when you try adding duplicate rules without the count it works
without any error while if you add the count action then for the second rule you get an error.
This line address this inconsistency.

> Thanks,
> 
> PS: when you send a new revision, please use the --thread and --in-reply-to
> to link it with the previous one.
> 


My bad,

> --
> Nélio Laranjeiro
> 6WIND

Regards,
Ori Kam

Re: [PATCH v2] doc: update mlx5 flow count limitations

[Nélio Laranjeiro]

On Mon, Oct 16, 2017 at 11:31:32AM +0000, Ori Kam wrote:
> Please see my comments,
> 
> > -----Original Message-----
> > From: Nélio Laranjeiro [mailto:nelio.laranjeiro@6wind.com]
> > Sent: Monday, October 16, 2017 2:18 PM
> > To: Ori Kam <orika@mellanox.com>
> > Cc: Adrien Mazarguil <adrien.mazarguil@6wind.com>; Yongseok Koh
> > <yskoh@mellanox.com>; dev@dpdk.org
> > Subject: Re: [PATCH v2] doc: update mlx5 flow count limitations
> > 
> > On Mon, Oct 16, 2017 at 11:32:39AM +0300, Ori Kam wrote:
> > > Signed-off-by: Ori Kam <orika@mellanox.com>
> > > Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> > > ---
> > >  doc/guides/nics/mlx5.rst |    5 ++++-
> > >  1 files changed, 4 insertions(+), 1 deletions(-)
> > >
> > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index
> > > d24941a..7da5c87 100644
> > > --- a/doc/guides/nics/mlx5.rst
> > > +++ b/doc/guides/nics/mlx5.rst
> > > @@ -104,7 +104,8 @@ Limitations
> > >  -----------
> > >
> > >  - Inner RSS for VXLAN frames is not supported yet.
> > > -- Port statistics through software counters only.
> > > +- Port statistics through software counters only. Flow statistics are
> > > +  supported by hardware counters.
> > >  - Hardware checksum RX offloads for VXLAN inner header are not
> > supported yet.
> > >  - Forked secondary process not supported.
> > >  - Flow pattern without any specific vlan will match for vlan packets as well:
> > > @@ -127,6 +128,8 @@ Limitations
> > >  - A multi segment packet must have less than 6 segments in case the Tx
> > burst function
> > >    is set to multi-packet send or Enhanced multi-packet send. Otherwise it
> > must have
> > >    less than 50 segments.
> > > +- Count action for RTE flow is only supported in OFED 4.2.
> > > +- Two identical rules with count action are not supported.
> > 
> > This last line does not bring anything, having two identical rules already
> > create a collision which is not defined by the flow API.
> > In PMD situation is is even worse, as it will match both rules and thus it will
> > duplicate a packet.
> > 
> > There is no need to add a limit on an undefined behavior by the API itself.
> > 
> 
> The thing is that when you try adding duplicate rules without the count it works
> without any error while if you add the count action then for the second rule you get an error.
> This line address this inconsistency.

According to Flow API documentation [1]

"If a packet is matched by several rules of a given group for a given
priority level, the outcome is undefined. It can take any path, may be
duplicated or even cause unrecoverable errors."

This is already documented and application makers should avoid it.
There is no need to duplicate this information.

> > Thanks,
> > 
> > PS: when you send a new revision, please use the --thread and --in-reply-to
> > to link it with the previous one.
> > 
> 
> 
> My bad,
> 
> > --
> > Nélio Laranjeiro
> > 6WIND
> 
> Regards,
> Ori Kam

Thanks,

[1] http://dpdk.org/doc/guides/prog_guide/rte_flow.html

-- 
Nélio Laranjeiro
6WIND

Re: [PATCH v2] doc: update mlx5 flow count limitations

[Ori Kam]

> -----Original Message-----
> From: Nélio Laranjeiro [mailto:nelio.laranjeiro@6wind.com]
> Sent: Monday, October 16, 2017 3:36 PM
> To: Ori Kam <orika@mellanox.com>
> Cc: Adrien Mazarguil <adrien.mazarguil@6wind.com>; Yongseok Koh
> <yskoh@mellanox.com>; dev@dpdk.org
> Subject: Re: [PATCH v2] doc: update mlx5 flow count limitations
> 
> On Mon, Oct 16, 2017 at 11:31:32AM +0000, Ori Kam wrote:
> > Please see my comments,
> >
> > > -----Original Message-----
> > > From: Nélio Laranjeiro [mailto:nelio.laranjeiro@6wind.com]
> > > Sent: Monday, October 16, 2017 2:18 PM
> > > To: Ori Kam <orika@mellanox.com>
> > > Cc: Adrien Mazarguil <adrien.mazarguil@6wind.com>; Yongseok Koh
> > > <yskoh@mellanox.com>; dev@dpdk.org
> > > Subject: Re: [PATCH v2] doc: update mlx5 flow count limitations
> > >
> > > On Mon, Oct 16, 2017 at 11:32:39AM +0300, Ori Kam wrote:
> > > > Signed-off-by: Ori Kam <orika@mellanox.com>
> > > > Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> > > > ---
> > > >  doc/guides/nics/mlx5.rst |    5 ++++-
> > > >  1 files changed, 4 insertions(+), 1 deletions(-)
> > > >
> > > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> > > > index
> > > > d24941a..7da5c87 100644
> > > > --- a/doc/guides/nics/mlx5.rst
> > > > +++ b/doc/guides/nics/mlx5.rst
> > > > @@ -104,7 +104,8 @@ Limitations
> > > >  -----------
> > > >
> > > >  - Inner RSS for VXLAN frames is not supported yet.
> > > > -- Port statistics through software counters only.
> > > > +- Port statistics through software counters only. Flow statistics
> > > > +are
> > > > +  supported by hardware counters.
> > > >  - Hardware checksum RX offloads for VXLAN inner header are not
> > > supported yet.
> > > >  - Forked secondary process not supported.
> > > >  - Flow pattern without any specific vlan will match for vlan packets as
> well:
> > > > @@ -127,6 +128,8 @@ Limitations
> > > >  - A multi segment packet must have less than 6 segments in case
> > > > the Tx
> > > burst function
> > > >    is set to multi-packet send or Enhanced multi-packet send.
> > > > Otherwise it
> > > must have
> > > >    less than 50 segments.
> > > > +- Count action for RTE flow is only supported in OFED 4.2.
> > > > +- Two identical rules with count action are not supported.
> > >
> > > This last line does not bring anything, having two identical rules
> > > already create a collision which is not defined by the flow API.
> > > In PMD situation is is even worse, as it will match both rules and
> > > thus it will duplicate a packet.
> > >
> > > There is no need to add a limit on an undefined behavior by the API itself.
> > >
> >
> > The thing is that when you try adding duplicate rules without the
> > count it works without any error while if you add the count action then for
> the second rule you get an error.
> > This line address this inconsistency.
> 
> According to Flow API documentation [1]
> 
> "If a packet is matched by several rules of a given group for a given priority
> level, the outcome is undefined. It can take any path, may be duplicated or
> even cause unrecoverable errors."
> 
> This is already documented and application makers should avoid it.
> There is no need to duplicate this information.
> 

Will fix in version 3.

> > > Thanks,
> > >
> > > PS: when you send a new revision, please use the --thread and
> > > --in-reply-to to link it with the previous one.
> > >
> >
> >
> > My bad,
> >
> > > --
> > > Nélio Laranjeiro
> > > 6WIND
> >
> > Regards,
> > Ori Kam
> 
> Thanks,
> 
> [1]
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdpd
> k.org%2Fdoc%2Fguides%2Fprog_guide%2Frte_flow.html&data=02%7C01%7
> Corika%40mellanox.com%7Ccc884d2a17c641d6388b08d51492886c%7Ca65297
> 1c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636437541948539609&sdata=fh
> 67Sys0ZaabU6hmpv5uo%2BgfXlRe2eDjTn65MUVO7IU%3D&reserved=0
> 
> --
> Nélio Laranjeiro
> 6WIND

Regards,
Ori Kam

[PATCH v3] doc: update mlx5 flow count limitations

[Ori Kam]

Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>

---

v3:
* Remove unnecessary line.

 doc/guides/nics/mlx5.rst | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index d24941a..7ef9b0d 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -104,7 +104,8 @@ Limitations
 -----------
 
 - Inner RSS for VXLAN frames is not supported yet.
-- Port statistics through software counters only.
+- Port statistics through software counters only. Flow statistics are
+  supported by hardware counters.
 - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
 - Forked secondary process not supported.
 - Flow pattern without any specific vlan will match for vlan packets as well:
@@ -127,6 +128,7 @@ Limitations
 - A multi segment packet must have less than 6 segments in case the Tx burst function
   is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
   less than 50 segments.
+- Count action for RTE flow is only supported in OFED 4.2.
 
 Configuration
 -------------
-- 
1.8.3.1

Re: [PATCH v3] doc: update mlx5 flow count limitations

[Nélio Laranjeiro]

On Mon, Oct 16, 2017 at 05:11:26PM +0300, Ori Kam wrote:
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> 
> ---
> 
> v3:
> * Remove unnecessary line.
> 
>  doc/guides/nics/mlx5.rst | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> index d24941a..7ef9b0d 100644
> --- a/doc/guides/nics/mlx5.rst
> +++ b/doc/guides/nics/mlx5.rst
> @@ -104,7 +104,8 @@ Limitations
>  -----------
>  
>  - Inner RSS for VXLAN frames is not supported yet.
> -- Port statistics through software counters only.
> +- Port statistics through software counters only. Flow statistics are
> +  supported by hardware counters.
>  - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
>  - Forked secondary process not supported.
>  - Flow pattern without any specific vlan will match for vlan packets as well:
> @@ -127,6 +128,7 @@ Limitations
>  - A multi segment packet must have less than 6 segments in case the Tx burst function
>    is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
>    less than 50 segments.
> +- Count action for RTE flow is only supported in OFED 4.2.

OFED or Mellanox OFED?

Thanks,

-- 
Nélio Laranjeiro
6WIND

[PATCH v4] doc: update mlx5 flow count limitations

[Ori Kam]

Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
---
v4:
* Clarify which OFED versionis required.

v3:
* Remove unnecessary line.

 doc/guides/nics/mlx5.rst | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
index d24941a..bcc39f6 100644
--- a/doc/guides/nics/mlx5.rst
+++ b/doc/guides/nics/mlx5.rst
@@ -104,7 +104,8 @@ Limitations
 -----------
 
 - Inner RSS for VXLAN frames is not supported yet.
-- Port statistics through software counters only.
+- Port statistics through software counters only. Flow statistics are
+  supported by hardware counters.
 - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
 - Forked secondary process not supported.
 - Flow pattern without any specific vlan will match for vlan packets as well:
@@ -127,6 +128,7 @@ Limitations
 - A multi segment packet must have less than 6 segments in case the Tx burst function
   is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
   less than 50 segments.
+- Count action for RTE flow is only supported in Mellanox OFED 4.2.
 
 Configuration
 -------------
-- 
1.8.3.1

Re: [PATCH v4] doc: update mlx5 flow count limitations

[Ferruh Yigit]

On 10/18/2017 11:09 PM, Ori Kam wrote:
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> ---
> v4:
> * Clarify which OFED versionis required.
> 
> v3:
> * Remove unnecessary line.
> 
>  doc/guides/nics/mlx5.rst | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> index d24941a..bcc39f6 100644
> --- a/doc/guides/nics/mlx5.rst
> +++ b/doc/guides/nics/mlx5.rst
> @@ -104,7 +104,8 @@ Limitations
>  -----------
>  
>  - Inner RSS for VXLAN frames is not supported yet.
> -- Port statistics through software counters only.
> +- Port statistics through software counters only. Flow statistics are
> +  supported by hardware counters.
>  - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
>  - Forked secondary process not supported.
>  - Flow pattern without any specific vlan will match for vlan packets as well:
> @@ -127,6 +128,7 @@ Limitations
>  - A multi segment packet must have less than 6 segments in case the Tx burst function
>    is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
>    less than 50 segments.
> +- Count action for RTE flow is only supported in Mellanox OFED 4.2.

Is 4.2 become public?

>  
>  Configuration
>  -------------
> 

Re: [PATCH v4] doc: update mlx5 flow count limitations

[Olga Shern]

Hi Ferruh,

MLNX_OFED 4.2 will be released by the end of October

Best Regards,
Olga


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit
> Sent: Thursday, October 19, 2017 9:19 AM
> To: Ori Kam <orika@mellanox.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; Nélio Laranjeiro
> <nelio.laranjeiro@6wind.com>; Yongseok Koh <yskoh@mellanox.com>;
> john.mcnamara@intel.com
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4] doc: update mlx5 flow count limitations
> 
> On 10/18/2017 11:09 PM, Ori Kam wrote:
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> > ---
> > v4:
> > * Clarify which OFED versionis required.
> >
> > v3:
> > * Remove unnecessary line.
> >
> >  doc/guides/nics/mlx5.rst | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index
> > d24941a..bcc39f6 100644
> > --- a/doc/guides/nics/mlx5.rst
> > +++ b/doc/guides/nics/mlx5.rst
> > @@ -104,7 +104,8 @@ Limitations
> >  -----------
> >
> >  - Inner RSS for VXLAN frames is not supported yet.
> > -- Port statistics through software counters only.
> > +- Port statistics through software counters only. Flow statistics are
> > +  supported by hardware counters.
> >  - Hardware checksum RX offloads for VXLAN inner header are not
> supported yet.
> >  - Forked secondary process not supported.
> >  - Flow pattern without any specific vlan will match for vlan packets as well:
> > @@ -127,6 +128,7 @@ Limitations
> >  - A multi segment packet must have less than 6 segments in case the Tx
> burst function
> >    is set to multi-packet send or Enhanced multi-packet send. Otherwise it
> must have
> >    less than 50 segments.
> > +- Count action for RTE flow is only supported in Mellanox OFED 4.2.
> 
> Is 4.2 become public?
> 
> >
> >  Configuration
> >  -------------
> >

Re: [PATCH v4] doc: update mlx5 flow count limitations

[Nélio Laranjeiro]

On Thu, Oct 19, 2017 at 09:09:26AM +0300, Ori Kam wrote:
> Signed-off-by: Ori Kam <orika@mellanox.com>
> Acked-by: Shahaf Shuler <shahafs@mellanox.com>
> ---
> v4:
> * Clarify which OFED versionis required.
> 
> v3:
> * Remove unnecessary line.
> 
>  doc/guides/nics/mlx5.rst | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> index d24941a..bcc39f6 100644
> --- a/doc/guides/nics/mlx5.rst
> +++ b/doc/guides/nics/mlx5.rst
> @@ -104,7 +104,8 @@ Limitations
>  -----------
>  
>  - Inner RSS for VXLAN frames is not supported yet.
> -- Port statistics through software counters only.
> +- Port statistics through software counters only. Flow statistics are
> +  supported by hardware counters.
>  - Hardware checksum RX offloads for VXLAN inner header are not supported yet.
>  - Forked secondary process not supported.
>  - Flow pattern without any specific vlan will match for vlan packets as well:
> @@ -127,6 +128,7 @@ Limitations
>  - A multi segment packet must have less than 6 segments in case the Tx burst function
>    is set to multi-packet send or Enhanced multi-packet send. Otherwise it must have
>    less than 50 segments.
> +- Count action for RTE flow is only supported in Mellanox OFED 4.2.
>  
>  Configuration
>  -------------
> -- 
> 1.8.3.1
 

Acked-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>

-- 
Nélio Laranjeiro
6WIND

Re: [PATCH v4] doc: update mlx5 flow count limitations

[Ferruh Yigit]

On 10/19/2017 12:05 AM, Nélio Laranjeiro wrote:
> On Thu, Oct 19, 2017 at 09:09:26AM +0300, Ori Kam wrote:
>> Signed-off-by: Ori Kam <orika@mellanox.com>
>> Acked-by: Shahaf Shuler <shahafs@mellanox.com>


> Acked-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>

Applied to dpdk-next-net/master, thanks.