[PATCH net v2] net: cbs: Fix software cbs to consider packet sending time

[PATCH net v2] net: cbs: Fix software cbs to consider packet sending time

[Zh-yuan Ye]

Currently the software CBS does not consider the packet sending time
when depleting the credits. It caused the throughput to be
Idleslope[kbps] * (Port transmit rate[kbps] / |Sendslope[kbps]|) where
Idleslope * (Port transmit rate / (Idleslope + |Sendslope|)) = Idleslope
is expected. In order to fix the issue above, this patch takes the time
when the packet sending completes into account by moving the anchor time
variable "last" ahead to the send completion time upon transmission and
adding wait when the next dequeue request comes before the send
completion time of the previous packet.

Signed-off-by: Zh-yuan Ye <ye.zh-yuan@socionext.com>

---
changes in v2:
 - combine variable "send_completed" into "last"
 - add the comment for estimate of the packet sending

---
 net/sched/sch_cbs.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/net/sched/sch_cbs.c b/net/sched/sch_cbs.c
index b2905b03a432..20f95f0b9d5b 100644
--- a/net/sched/sch_cbs.c
+++ b/net/sched/sch_cbs.c
@@ -181,6 +181,11 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
 	s64 credits;
 	int len;
 
+	/* The previous packet is still being sent */
+	if (now < q->last) {
+		qdisc_watchdog_schedule_ns(&q->watchdog, q->last);
+		return NULL;
+	}
 	if (q->credits < 0) {
 		credits = timediff_to_credits(now - q->last, q->idleslope);
 
@@ -192,7 +197,6 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
 
 			delay = delay_from_credits(q->credits, q->idleslope);
 			qdisc_watchdog_schedule_ns(&q->watchdog, now + delay);
-
 			q->last = now;
 
 			return NULL;
@@ -212,7 +216,9 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
 	credits += q->credits;
 
 	q->credits = max_t(s64, credits, q->locredit);
-	q->last = now;
+	/* Estimate of the transmission of the last byte of the packet in ns */
+	q->last = now + div64_s64(len * NSEC_PER_SEC,
+				  atomic64_read(&q->port_rate));
 
 	return skb;
 }
-- 
2.20.1

Re: [PATCH net v2] net: cbs: Fix software cbs to consider packet sending time

[Jakub Kicinski]

Minor comments, while waiting for more in-depth review from Vinicius :)

On Mon, 23 Mar 2020 15:17:09 +0900 Zh-yuan Ye wrote:
> Currently the software CBS does not consider the packet sending time
> when depleting the credits. It caused the throughput to be
> Idleslope[kbps] * (Port transmit rate[kbps] / |Sendslope[kbps]|) where
> Idleslope * (Port transmit rate / (Idleslope + |Sendslope|)) = Idleslope
> is expected. In order to fix the issue above, this patch takes the time
> when the packet sending completes into account by moving the anchor time
> variable "last" ahead to the send completion time upon transmission and
> adding wait when the next dequeue request comes before the send
> completion time of the previous packet.

Please add a Fixes tag.

> Signed-off-by: Zh-yuan Ye <ye.zh-yuan@socionext.com>
> ---
> changes in v2:
>  - combine variable "send_completed" into "last"
>  - add the comment for estimate of the packet sending

You can keep those inside the commit message for networking patches

> diff --git a/net/sched/sch_cbs.c b/net/sched/sch_cbs.c
> index b2905b03a432..20f95f0b9d5b 100644
> --- a/net/sched/sch_cbs.c
> +++ b/net/sched/sch_cbs.c
> @@ -181,6 +181,11 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
>  	s64 credits;
>  	int len;
>  
> +	/* The previous packet is still being sent */
> +	if (now < q->last) {
> +		qdisc_watchdog_schedule_ns(&q->watchdog, q->last);
> +		return NULL;
> +	}
>  	if (q->credits < 0) {
>  		credits = timediff_to_credits(now - q->last, q->idleslope);
>  
> @@ -192,7 +197,6 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
>  
>  			delay = delay_from_credits(q->credits, q->idleslope);
>  			qdisc_watchdog_schedule_ns(&q->watchdog, now + delay);
> -

Please don't do whitespace cleanup like this in a bug fix. Bug fixes
should be minimal to avoid conflicts.

>  			q->last = now;
>  
>  			return NULL;
> @@ -212,7 +216,9 @@ static struct sk_buff *cbs_dequeue_soft(struct Qdisc *sch)
>  	credits += q->credits;
>  
>  	q->credits = max_t(s64, credits, q->locredit);
> -	q->last = now;
> +	/* Estimate of the transmission of the last byte of the packet in ns */
> +	q->last = now + div64_s64(len * NSEC_PER_SEC,
> +				  atomic64_read(&q->port_rate));

credits_from_len() checks if port_rate is 0 before division. Can you double check 
it's unnecessary?

>  
>  	return skb;
>  }

Re: [PATCH net v2] net: cbs: Fix software cbs to consider packet sending time

[Vinicius Costa Gomes]

Hi,

Zh-yuan Ye <ye.zh-yuan@socionext.com> writes:

> Currently the software CBS does not consider the packet sending time
> when depleting the credits. It caused the throughput to be
> Idleslope[kbps] * (Port transmit rate[kbps] / |Sendslope[kbps]|) where
> Idleslope * (Port transmit rate / (Idleslope + |Sendslope|)) = Idleslope
> is expected. In order to fix the issue above, this patch takes the time
> when the packet sending completes into account by moving the anchor time
> variable "last" ahead to the send completion time upon transmission and
> adding wait when the next dequeue request comes before the send
> completion time of the previous packet.

Yeah, this improves (a lot) the accuracy of the software mode when
dealing with larger packets:

| Idleslope (packet size) | Old (bps) | New (bps) |
|-------------------------+-----------+-----------|
| 500mbps (100bytes)      | 528M      | 492M      |
| 500mbps (1500bytes)     | 622M      | 499M      |
| 1mbps (100bytes)        | 1007k     | 1006k     |
| 1mbps (1500bytes)       | 1010k     | 1010k     |

(Sorry for the mess of units)

So, when the comments from Jakub are addressed:

Reviewed-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>


Cheers,
-- 
Vinicius