Re: [PATCH net-next 3/3] docs: networking: timestamping: add a set of frequently asked questions

From: Vladimir Oltean <olteanv@gmail.com>
To: Jacob Keller <jacob.e.keller@intel.com>
Cc: kuba@kernel.org, davem@davemloft.net, netdev@vger.kernel.org,
	richardcochran@gmail.com, sorganov@gmail.com,
	linux-doc@vger.kernel.org
Subject: Re: [PATCH net-next 3/3] docs: networking: timestamping: add a set of frequently asked questions
Date: Tue, 21 Jul 2020 00:05:18 +0300	[thread overview]
Message-ID: <20200720210518.5uddqqbjuci5wxki@skbuf> (raw)
In-Reply-To: <887fcc0d-4f3d-3cb8-bdea-8144b62c5d85@intel.com>

On Mon, Jul 20, 2020 at 11:54:30AM -0700, Jacob Keller wrote:
> On 7/18/2020 4:35 AM, Vladimir Oltean wrote:
> > On Fri, Jul 17, 2020 at 04:12:07PM -0700, Jacob Keller wrote:
> >> On 7/17/2020 9:10 AM, Vladimir Oltean wrote:
> >>> +When the interface they represent offers both ``SOF_TIMESTAMPING_TX_HARDWARE``
> >>> +and ``SOF_TIMESTAMPING_TX_SOFTWARE``.
> >>> +Originally, the network stack could deliver either a hardware or a software
> >>> +time stamp, but not both. This flag prevents software timestamp delivery.
> >>> +This restriction was eventually lifted via the ``SOF_TIMESTAMPING_OPT_TX_SWHW``
> >>> +option, but still the original behavior is preserved as the default.
> >>> +
> >>
> >> So, this implies that we set this only if both are supported? I thought
> >> the intention was to set this flag whenever we start a HW timestamp.
> >>
> > 
> > It's only _required_ when SOF_TIMESTAMPING_TX_SOFTWARE is used, it
> > seems. I had also thought of setting 'SKBTX_IN_PROGRESS' as good
> > practice, but there are many situations where it can do more harm than
> > good.
> > 
> 
> I guess I've only ever implemented a driver with software timestamping
> enabled as an option. What sort of issues arise when you have this set?
> I'm guessing that it's some configuration of stacked devices as in the
> other cases? If the issue can't be fixed I'd at least like more
> explanation here, since the prevailing convention is that we set this
> flag, so understanding when and why it's problematic would be useful.
> 
> Thanks,
> Jake

Yes, the problematic cases have to do with stacked PHCs (DSA, PHY). The
pattern is that:
- DSA sets SKBTX_IN_PROGRESS
- calls dev_queue_xmit towards the MAC driver
- MAC driver sees SKBTX_IN_PROGRESS, thinks it's the one who set it
- MAC driver delivers TX timestamp
- DSA ends poll or receives TX interrupt, collects its timestamp, and
  delivers a second TX timestamp
In fact this is explained in a bit more detail in the current
timestamping.rst file.
Not only are there existing in-tree drivers that do that (and various
subtle variations of it), but new code also has this tendency to take
shortcuts and interpret any SKBTX_IN_PROGRESS flag set as being set
locally. Good thing it's caught during review most of the time these
days. It's an error-prone design.
On the DSA front, 1 driver sets this flag (sja1105) and 3 don't (felix,
mv88e6xxx, hellcreek). The driver who had trouble because of this flag?
sja1105.
On the PHY front, 2 drivers set this flag (mscc_phy, dp83640) and 1
doesn't (ptp_ines). The driver who had trouble? dp83640.
So it's very far from obvious that setting this flag is 'the prevailing
convention'. For a MAC driver, that might well be, but for DSA/PHY,
there seem to be risks associated with doing that, and driver writers
should know what they're signing up for.

-Vladimir