From: "Patel, Vedang" <vedang.patel@intel.com>
To: Murali Karicheri <m-karicheri2@ti.com>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"Kirsher, Jeffrey T" <jeffrey.t.kirsher@intel.com>,
"David S . Miller" <davem@davemloft.net>,
Jamal Hadi Salim <jhs@mojatatu.com>,
Cong Wang <xiyou.wangcong@gmail.com>,
Jiri Pirko <jiri@resnulli.us>,
"intel-wired-lan@lists.osuosl.org"
<intel-wired-lan@lists.osuosl.org>,
"Gomes, Vinicius" <vinicius.gomes@intel.com>,
"l@dorileo.org" <l@dorileo.org>
Subject: Re: [PATCH net-next v1 0/7] net/sched: Add txtime assist support for taprio
Date: Tue, 4 Jun 2019 19:53:17 +0000 [thread overview]
Message-ID: <28C662B0-6B93-453B-B664-F3D8FFA8EFBB@intel.com> (raw)
In-Reply-To: <4c4ec746-4282-ad3a-351c-1c7a355d093d@ti.com>
> On Jun 3, 2019, at 6:54 AM, Murali Karicheri <m-karicheri2@ti.com> wrote:
>
> On 06/03/2019 09:50 AM, Murali Karicheri wrote:
>> Hi Vedang,
>> On 05/28/2019 01:46 PM, Vedang Patel wrote:
>>> Currently, we are seeing packets being transmitted outside their timeslices. We
>>> can confirm that the packets are being dequeued at the right time. So, the
>>> delay is induced after the packet is dequeued, because taprio, without any
>>> offloading, has no control of when a packet is actually transmitted.
>>>
>>> In order to solve this, we are making use of the txtime feature provided by ETF
>>> qdisc. Hardware offloading needs to be supported by the ETF qdisc in order to
>>> take advantage of this feature. The taprio qdisc will assign txtime (in
>>> skb->tstamp) for all the packets which do not have the txtime allocated via the
>>> SO_TXTIME socket option. For the packets which already have SO_TXTIME set,
>>> taprio will validate whether the packet will be transmitted in the correct
>>> interval.
>>>
>>> In order to support this, the following parameters have been added:
>>> - offload (taprio): This is added in order to support different offloading
>>> modes which will be added in the future.
>>> - txtime-delay (taprio): this is the time the packet takes to traverse through
>>> the kernel to adapter card.
>> I am very new to this TC and QoS handling of Linux kernel and TSN. So sorry some of the questions below are very basic in nature. I would soon would be working to add this support in our platform based on this.
>> So please answer.
>> Is txtime-delay from the instance qdisc de-queue the packet to the time
>> packets get onto the wire? I am wondering if this time is deterministic
>> or we have some way to ensure this can be tuned to meet a max value?
>> Also how would one calculate this value or have to measure it?
>>> - skip_sock_check (etf): etf qdisc currently drops any packet which does not
>>> have the SO_TXTIME socket option set. This check can be skipped by specifying
>>> this option.
>>>
>>> Following is an example configuration:
>>>
>>> $ tc qdisc replace dev $IFACE parent root handle 100 taprio \\
>>> num_tc 3 \\
>>> map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \\
>>> queues 1@0 1@0 1@0 \\
>>> base-time $BASE_TIME \\
>>> sched-entry S 01 300000 \\
>>> sched-entry S 02 300000 \\
>>> sched-entry S 04 400000 \\
>>> offload 2 \\
>>> txtime-delay 40000 \\
>>> clockid CLOCK_TAI
>>>
>> Could you tell me what is CLOCK_TAI?? I see below in the source code for
>> the definition in include/uapi/linux/time.h
>> /*
>> * The driver implementing this got removed. The clock ID is kept as a
>> * place holder. Do not reuse!
>> */
>> #define CLOCK_SGI_CYCLE 10
>> #define CLOCK_TAI 11
>> So why do I see such defines being used in the example that is being
>> removed?
>> AFAIK, From the usage point of view, TSN requires the network being
>> synchronized through linux PTP. For synchronization phc2sys is used to
>> synchronize system time to the PHC. So why don't one use system time for
>> this?
>> So application will use clock_gettime() to know current time and
>> schedule the packet for transmission as well as user would use scripts
>> or such to check current system time and issue tc command above. So the
>> command should use CLOCK_REALTIME in that case. So all h/w and software
>> are aligned to the same time source. Or Have I understood it wrong? I am
>> assuming that for the offloaded case, h/w schedule Gate open, send
>> packet etc are synchronized to the PHC or use a translated time w.r.t the common time (network time. Assuming PHC tracks this).
>> Thanks in advance for clarifying this.
>>> $ tc qdisc replace dev $IFACE parent 100:1 etf \\
>>> offload delta 200000 clockid CLOCK_TAI skip_sock_check
>>>
>>> Here, the "offload" parameter is indicating that the TXTIME_OFFLOAD mode is
>>> enabled. Also, that all the traffic classes have been assigned the same queue.
>
> Actually the tc class is mapped to the same queue in the previous
> command, not this one, right?
>
Yes, you are right. It is done while setting up taprio. The way it’s written looks confusing. I will modify and clarify this in the next version of the series.
Thanks,
Vedang
> Murali
>>> This is to prevent the traffic classes in the lower priority queues from
>>> getting starved. Note that this configuration is specific to the i210 ethernet
>>> card. Other network cards where the hardware queues are given the same
>>> priority, might be able to utilize more than one queue.
>>>
>>> Following are some of the other highlights of the series:
>>> - Fix a bug where hardware timestamping and SO_TXTIME options cannot be used
>>> together. (Patch 1)
>>> - Introduce hardware offloading. This patch introduces offload parameter which
>>> can be used to enable the txtime offload mode. It will also support enabling
>>> full offload when the support is available in drivers. (Patch 3)
>>> - Make TxTime assist mode work with TCP packets. (Patch 6 & 7)
>>>
>>>
>>> The following changes are recommended to be done in order to get the best
>>> performance from taprio in this mode:
>>> # ip link set dev enp1s0 mtu 1514
>> May I know why MTU is changed to 1514 to include the Ethernet header?
>>> # ethtool -K eth0 gso off
>>> # ethtool -K eth0 tso off
>>> # ethtool --set-eee eth0 eee off
>> Could you please explain why these are needed?
>> Thanks
>> Murali
>>>
>>> Thanks,
>>> Vedang Patel
>>>
>>> Vedang Patel (6):
>>> igb: clear out tstamp after sending the packet.
>>> etf: Add skip_sock_check
>>> taprio: calculate cycle_time when schedule is installed
>>> taprio: Add support for txtime offload mode.
>>> taprio: make clock reference conversions easier
>>> taprio: Adjust timestamps for TCP packets.
>>>
>>> Vinicius Costa Gomes (1):
>>> taprio: Add the skeleton to enable hardware offloading
>>>
>>> drivers/net/ethernet/intel/igb/igb_main.c | 1 +
>>> include/uapi/linux/pkt_sched.h | 6 +
>>> net/sched/sch_etf.c | 10 +
>>> net/sched/sch_taprio.c | 548 ++++++++++++++++++++--
>>> 4 files changed, 532 insertions(+), 33 deletions(-)
>>>
>
next prev parent reply other threads:[~2019-06-04 19:53 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-28 17:46 [PATCH net-next v1 0/7] net/sched: Add txtime assist support for taprio Vedang Patel
2019-05-28 17:46 ` [PATCH net-next v1 1/7] igb: clear out tstamp after sending the packet Vedang Patel
2019-05-28 17:46 ` [PATCH net-next v1 2/7] etf: Add skip_sock_check Vedang Patel
2019-05-28 17:46 ` [PATCH net-next v1 3/7] taprio: Add the skeleton to enable hardware offloading Vedang Patel
2019-05-28 22:45 ` Jakub Kicinski
2019-05-29 17:06 ` Patel, Vedang
2019-05-29 19:14 ` Jakub Kicinski
2019-05-29 20:05 ` Patel, Vedang
2019-05-29 21:06 ` Jakub Kicinski
2019-05-30 0:21 ` Patel, Vedang
2019-05-30 20:41 ` Jakub Kicinski
2019-05-28 17:46 ` [PATCH net-next v1 4/7] taprio: calculate cycle_time when schedule is installed Vedang Patel
2019-05-28 17:46 ` [PATCH net-next v1 5/7] taprio: Add support for txtime offload mode Vedang Patel
2019-06-03 14:15 ` Murali Karicheri
2019-06-04 20:06 ` Patel, Vedang
2019-06-07 18:52 ` Murali Karicheri
2019-06-07 21:12 ` Patel, Vedang
2019-06-10 14:27 ` Murali Karicheri
2019-06-11 21:03 ` Patel, Vedang
2019-05-28 17:46 ` [PATCH net-next v1 6/7] taprio: make clock reference conversions easier Vedang Patel
2019-05-28 17:46 ` [PATCH net-next v1 7/7] taprio: Adjust timestamps for TCP packets Vedang Patel
2019-06-03 13:50 ` [PATCH net-next v1 0/7] net/sched: Add txtime assist support for taprio Murali Karicheri
2019-06-03 13:54 ` Murali Karicheri
2019-06-04 19:53 ` Patel, Vedang [this message]
2019-06-04 19:47 ` Patel, Vedang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=28C662B0-6B93-453B-B664-F3D8FFA8EFBB@intel.com \
--to=vedang.patel@intel.com \
--cc=davem@davemloft.net \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jeffrey.t.kirsher@intel.com \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=l@dorileo.org \
--cc=m-karicheri2@ti.com \
--cc=netdev@vger.kernel.org \
--cc=vinicius.gomes@intel.com \
--cc=xiyou.wangcong@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).