[PATCH] ixgbe 82599: Query assignment of queues to Virtual Function

[PATCH] ixgbe 82599: Query assignment of queues to Virtual Function

[Qinglai Xiao]

Physical Function assignes Tx/Rx queues to each VF according to
different schemes[1]. By querying through mailbox, VF is able to
get number of Tx/Rx queues assigned to it.

Note that current Intel ixgbe driver ixgbe-3.18.7 does not fully
support mailbox message IXGBE_VF_GET_QUEUES. The service routine
for IXGBE_VF_GET_QUEUES must be fixed, otherwise PF always return
1 as Tx/Rx queue number.

[1] See section 7.2.1.2.1, 7.1.2.2 and 7.10.2.7.2 of Intel 82599 10
    Gbe Controller Datasheet.

Signed-off-by: Qinglai Xiao <jigsaw-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c |   19 +++++++++++++++++++
 1 files changed, 19 insertions(+), 0 deletions(-)

diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
index f217564..e269ad7 100644
--- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
+++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
@@ -750,6 +750,22 @@ eth_ixgbe_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,
 	return 0;
 }
 
+static void ixgbevf_get_queue_num(struct ixgbe_hw *hw)
+{
+	/* Traffic classes are not supported by now */
+	unsigned int tcs, tc;
+
+	/*
+	 * Must let PF know we are at mailbox API 11 level.
+	 * Otherwise PF won't answer properly.
+	 * In case that PF fails to provide Rx/Tx queue number,
+	 * max_tx_queues and max_rx_queues remain to be 1.
+	 */
+	if (!ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_11))
+		ixgbevf_get_queues(hw, &tcs, &tc);
+}
+
+
 /*
  * Virtual Function device init
  */
@@ -818,6 +834,9 @@ eth_ixgbevf_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,
 		return (diag);
 	}
 
+	/* Get Rx/Tx queue count via mailbox, which is ready after reset_hw */
+	ixgbevf_get_queue_num(hw);
+
 	/* Allocate memory for storing MAC addresses */
 	eth_dev->data->mac_addrs = rte_zmalloc("ixgbevf", ETHER_ADDR_LEN *
 			hw->mac.num_rar_entries, 0);
-- 
1.7.1

Re: [PATCH] ixgbe: query assignment of queues to VF

[Thomas Monjalon]

21/10/2013 16:11, Qinglai Xiao :
> Physical Function assignes Tx/Rx queues to each VF according to
> different schemes[1]. By querying through mailbox, VF is able to
> get number of Tx/Rx queues assigned to it.
> 
> Note that current Intel ixgbe driver ixgbe-3.18.7 does not fully
> support mailbox message IXGBE_VF_GET_QUEUES. The service routine
> for IXGBE_VF_GET_QUEUES must be fixed, otherwise PF always return
> 1 as Tx/Rx queue number.
> 
> [1] See section 7.2.1.2.1, 7.1.2.2 and 7.10.2.7.2 of Intel 82599 10
>     Gbe Controller Datasheet.
> 
> Signed-off-by: Qinglai Xiao <jigsaw-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

Acked and applied with light modifications:

-	 * Must let PF know we are at mailbox API 11 level.
+	 * Must let PF know we are at mailbox API version 1.1.

> +}
> +
> +
>  /*
>   * Virtual Function device init

Only 1 empty line is sufficient here.

Thanks for your patch.
Please, keep us informed about the needed PF patch.
-- 
Thomas

Re: [PATCH] ixgbe: query assignment of queues to VF

[jigsaw]

Hi Thomas,

>>Please, keep us informed about the needed PF patch.

First of all, where and how should I post the patch to PF driver?
And secondly I have only 82599, so I cannot run test for 82598 and
X540. Is it possible that you could test on different NIC?

thx &
rgds,
-ql

On Wed, Oct 23, 2013 at 12:19 PM, Thomas Monjalon
<thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org> wrote:
> 21/10/2013 16:11, Qinglai Xiao :
>> Physical Function assignes Tx/Rx queues to each VF according to
>> different schemes[1]. By querying through mailbox, VF is able to
>> get number of Tx/Rx queues assigned to it.
>>
>> Note that current Intel ixgbe driver ixgbe-3.18.7 does not fully
>> support mailbox message IXGBE_VF_GET_QUEUES. The service routine
>> for IXGBE_VF_GET_QUEUES must be fixed, otherwise PF always return
>> 1 as Tx/Rx queue number.
>>
>> [1] See section 7.2.1.2.1, 7.1.2.2 and 7.10.2.7.2 of Intel 82599 10
>>     Gbe Controller Datasheet.
>>
>> Signed-off-by: Qinglai Xiao <jigsaw-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>
> Acked and applied with light modifications:
>
> -        * Must let PF know we are at mailbox API 11 level.
> +        * Must let PF know we are at mailbox API version 1.1.
>
>> +}
>> +
>> +
>>  /*
>>   * Virtual Function device init
>
> Only 1 empty line is sufficient here.
>
> Thanks for your patch.
> Please, keep us informed about the needed PF patch.
> --
> Thomas

Re: [PATCH] ixgbe: query assignment of queues to VF

[Thomas Monjalon]

23/10/2013 14:26, jigsaw :
> First of all, where and how should I post the patch to PF driver?

You should send your patch to kernel.org via the netdev mailing list:
	http://vger.kernel.org/vger-lists.html#netdev

> And secondly I have only 82599, so I cannot run test for 82598 and
> X540. Is it possible that you could test on different NIC?

I don't know if I'll have access to 82598 and X540 in the next weeks.
Is there anybody else who would like to help with tests ?

-- 
Thomas

Re: [PATCH] ixgbe: query assignment of queues to VF

[jigsaw]

Hi Thomas,

Thanks for your answer. I'll try to send the patch to netdev list and
let you know the result.
BTW, I double checked 82598 and find that it doesn't have mailbox at
all, meaning this patch is for 82599 and X540 only.
So the test is only needed for X540.

thx &
rgds,
-ql

On Thu, Oct 24, 2013 at 11:48 AM, Thomas Monjalon
<thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org> wrote:
> 23/10/2013 14:26, jigsaw :
>> First of all, where and how should I post the patch to PF driver?
>
> You should send your patch to kernel.org via the netdev mailing list:
>         http://vger.kernel.org/vger-lists.html#netdev
>
>> And secondly I have only 82599, so I cannot run test for 82598 and
>> X540. Is it possible that you could test on different NIC?
>
> I don't know if I'll have access to 82598 and X540 in the next weeks.
> Is there anybody else who would like to help with tests ?
>
> --
> Thomas

Re: [PATCH] ixgbe 82599: Query assignment of queues to Virtual Function.

[jigsaw]

Sorry it was not on my mind. Next patch will not have any trace to GPL code. -ql

On Mon, Oct 21, 2013 at 4:04 PM, Thomas Monjalon
<thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org> wrote:
> 21/10/2013 13:28, jigsaw :
>> The ixgbevf_negotiate_api is copied from Intel's VF driver.
> [..]
>> I will make another attempt with elimination of  ixgbevf_negotiate_api.
>
> Please, when sending a patch, be sure that there is no licensing issue.
> We cannot integrate GPL code in BSD files.
> By adding your "signed-off-by" line, you are certifying that the code is in the appropriate license. See "Certificate of Origin" in Linux guidelines:
>         https://www.kernel.org/doc/Documentation/SubmittingPatches
>
> --
> Thomas

Re: [PATCH] ixgbe 82599: Query assignment of queues to Virtual Function.

[Thomas Monjalon]

21/10/2013 13:28, jigsaw :
> The ixgbevf_negotiate_api is copied from Intel's VF driver.
[..]
> I will make another attempt with elimination of  ixgbevf_negotiate_api.

Please, when sending a patch, be sure that there is no licensing issue.
We cannot integrate GPL code in BSD files.
By adding your "signed-off-by" line, you are certifying that the code is in the appropriate license. See "Certificate of Origin" in Linux guidelines:
	https://www.kernel.org/doc/Documentation/SubmittingPatches

-- 
Thomas

Re: [PATCH] ixgbe 82599: Query assignment of queues to Virtual Function.

[jigsaw]

Hi Thomas,

Thanks for comment.
The ixgbevf_negotiate_api is copied from Intel's VF driver. The while
loop is not quite usual, indeed. A direct call to
ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_11) is sufficient, I
think. Coz PF is supposed to be satisfied with ver 1.1.
I will make another attempt with elimination of  ixgbevf_negotiate_api.

Any other comment?

thx &
rgds,
-ql

On Mon, Oct 21, 2013 at 11:36 AM, Thomas Monjalon
<thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org> wrote:
> Hi,
>
> 21/10/2013 09:09, Qinglai Xiao :
>> Physical Function assignes Tx/Rx queues to each VF according to
>> different schemes[1]. By querying through mailbox, VF is able to
>> get number of Tx/Rx queues assigned to it.
>>
>> Note that current Intel ixgbe driver ixgbe-3.18.7 does not fully
>> support mailbox message IXGBE_VF_GET_QUEUES. The service routine
>> for IXGBE_VF_GET_QUEUES must be fixed, otherwise PF always return
>> 1 as Tx/Rx queue number.
>>
>> [1] See section 7.2.1.2.1, 7.1.2.2 and 7.10.2.7.2 of Intel 82599 10
>>     Gbe Controller Datasheet.
>>
>> Signed-off-by: Qinglai Xiao <jigsaw-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>> ---
>
> Thanks for the patch, your comments are clear.
>
>> +static void ixgbevf_negotiate_api(struct ixgbe_hw *hw)
>> +{
>> +     int api[] = { ixgbe_mbox_api_11,
>> +                   ixgbe_mbox_api_10,
>> +                   ixgbe_mbox_api_unknown };
>> +     int err = 0, idx = 0;
>> +
>> +     while (api[idx] != ixgbe_mbox_api_unknown) {
>> +             err = ixgbevf_negotiate_api_version(hw, api[idx]);
>> +             if (!err)
>> +                     break;
>> +             idx++;
>> +     }
>> +}
>
> I have comments about the mbox API:
> 1) Why not call directly ixgbevf_negotiate_api_version for ixgbe_mbox_api_11 only ?
> It seems that only version 1.1 supports ixgbevf_get_queues.
> 2) If a loop is still needed, why not use a simple "for loop" ?
> 3) If ixgbevf_negotiate_api is still needed, I'd prefer to have "_mbx" in its name.
>
> Thank you
> --
> Thomas

Re: [PATCH] ixgbe 82599: Query assignment of queues to Virtual Function.

[Thomas Monjalon]

Hi,

21/10/2013 09:09, Qinglai Xiao :
> Physical Function assignes Tx/Rx queues to each VF according to
> different schemes[1]. By querying through mailbox, VF is able to
> get number of Tx/Rx queues assigned to it.
> 
> Note that current Intel ixgbe driver ixgbe-3.18.7 does not fully
> support mailbox message IXGBE_VF_GET_QUEUES. The service routine
> for IXGBE_VF_GET_QUEUES must be fixed, otherwise PF always return
> 1 as Tx/Rx queue number.
> 
> [1] See section 7.2.1.2.1, 7.1.2.2 and 7.10.2.7.2 of Intel 82599 10
>     Gbe Controller Datasheet.
> 
> Signed-off-by: Qinglai Xiao <jigsaw-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> ---

Thanks for the patch, your comments are clear.

> +static void ixgbevf_negotiate_api(struct ixgbe_hw *hw)
> +{
> +	int api[] = { ixgbe_mbox_api_11,
> +		      ixgbe_mbox_api_10,
> +		      ixgbe_mbox_api_unknown };
> +	int err = 0, idx = 0;
> +
> +	while (api[idx] != ixgbe_mbox_api_unknown) {
> +		err = ixgbevf_negotiate_api_version(hw, api[idx]);
> +		if (!err)
> +			break;
> +		idx++;
> +	}
> +}

I have comments about the mbox API:
1) Why not call directly ixgbevf_negotiate_api_version for ixgbe_mbox_api_11 only ?
It seems that only version 1.1 supports ixgbevf_get_queues.
2) If a loop is still needed, why not use a simple "for loop" ?
3) If ixgbevf_negotiate_api is still needed, I'd prefer to have "_mbx" in its name.

Thank you
-- 
Thomas

[PATCH] ixgbe 82599: Query assignment of queues to Virtual Function.

[Qinglai Xiao]

Physical Function assignes Tx/Rx queues to each VF according to
different schemes[1]. By querying through mailbox, VF is able to
get number of Tx/Rx queues assigned to it.

Note that current Intel ixgbe driver ixgbe-3.18.7 does not fully
support mailbox message IXGBE_VF_GET_QUEUES. The service routine
for IXGBE_VF_GET_QUEUES must be fixed, otherwise PF always return
1 as Tx/Rx queue number.

[1] See section 7.2.1.2.1, 7.1.2.2 and 7.10.2.7.2 of Intel 82599 10
    Gbe Controller Datasheet.

Signed-off-by: Qinglai Xiao <jigsaw-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
 lib/librte_pmd_ixgbe/ixgbe_ethdev.c |   32 ++++++++++++++++++++++++++++++++
 1 files changed, 32 insertions(+), 0 deletions(-)

diff --git a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
index f217564..991e1b3 100644
--- a/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
+++ b/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
@@ -750,6 +750,34 @@ eth_ixgbe_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,
 	return 0;
 }
 
+static void ixgbevf_negotiate_api(struct ixgbe_hw *hw)
+{
+	int api[] = { ixgbe_mbox_api_11,
+		      ixgbe_mbox_api_10,
+		      ixgbe_mbox_api_unknown };
+	int err = 0, idx = 0;
+
+	while (api[idx] != ixgbe_mbox_api_unknown) {
+		err = ixgbevf_negotiate_api_version(hw, api[idx]);
+		if (!err)
+			break;
+		idx++;
+	}
+}
+
+static void ixgbevf_get_queue_num(struct ixgbe_hw *hw)
+{
+	/* Traffic classes are not supported by now */
+	unsigned int tcs, tc;
+
+	/*
+	 * In case that PF fails to provide Rx/Tx queue number,
+	 * max_tx_queues and max_rx_queues remain to be 1.
+	 */
+	ixgbevf_get_queues(hw, &tcs, &tc);
+}
+
+
 /*
  * Virtual Function device init
  */
@@ -818,6 +846,10 @@ eth_ixgbevf_dev_init(__attribute__((unused)) struct eth_driver *eth_drv,
 		return (diag);
 	}
 
+	/* Get Rx/Tx queue count via mailbox, which is ready after reset_hw */
+	ixgbevf_negotiate_api(hw);
+	ixgbevf_get_queue_num(hw);
+
 	/* Allocate memory for storing MAC addresses */
 	eth_dev->data->mac_addrs = rte_zmalloc("ixgbevf", ETHER_ADDR_LEN *
 			hw->mac.num_rar_entries, 0);
-- 
1.7.1