Re: [RFC PATCH net-next 0/6] devlink: Add device metric support

[Florian Fainelli <f.fainelli@gmail.com>]

On 8/19/20 9:18 AM, Jakub Kicinski wrote:
> On Tue, 18 Aug 2020 21:30:16 -0700 Florian Fainelli wrote:
>>>>> I spend way too much time patrolling ethtool -S outputs already.  
>>>>
>>>> But that's the nature of detailed stats which are often essential to
>>>> ensuring the system is operating as expected or debugging some problem.
>>>> Commonality is certainly desired in names when relevant to be able to
>>>> build tooling around the stats.  
>>>
>>> There are stats which are clearly detailed and device specific,
>>> but what ends up happening is that people expose very much not
>>> implementation specific stats through the free form interfaces,
>>> because it's the easiest.
>>>
>>> And users are left picking up the pieces, having to ask vendors what
>>> each stat means, and trying to create abstractions in their user space
>>> glue.  
>>
>> Should we require vendors to either provide a Documentation/ entry for 
>> each statistics they have (and be guaranteed that it will be outdated 
>> unless someone notices), or would you rather have the statistics 
>> description be part of the devlink interface itself? Should we define 
>> namespaces such that standard metrics should be under the standard 
>> namespace and the vendor standard is the wild west?
> 
> I'm trying to find a solution which will not require a policeman to
> constantly monitor the compliance. Please see my effort to ensure
> drivers document and use the same ethtool -S stats in the TLS offload
> implementations. I've been trying to improve this situation for a long
> time, and it's getting old.

Which is why I am asking genuinely what do you think should be done
besides doing more code reviews? It does not seem to me that there is an
easy way to catch new stats being added with tools/scripts/whatever and
then determine what they are about, right?

> 
> Please focus on the stats this set adds, instead of fantasizing of what
> could be. These are absolutely not implementation specific!

Not sure if fantasizing is quite what I would use. I am just pointing
out that given the inability to standardize on statistics maybe we
should have namespaces and try our best to have everything fit into the
standard namespace along with a standard set of names, and push back
whenever we see vendor stats being added (or more pragmatically, ask
what they are). But maybe this very idea is moot.

> 
>>> If I have to download vendor documentation and tooling, or adapt my own
>>> scripts for every new vendor, I could have as well downloaded an SDK.  
>>
>> Are not you being a bit over dramatic here with your example? 
> 
> I hope not. It's very hard/impossible today to run a fleet of Linux
> machines without resorting to vendor tooling.

Your argument was putting on the same level resorting to vendor tooling
to extract meaningful statistics/counters versus using a SDK to operate
the hardware (this is how I understood it), and I do not believe this is
fair.

> 
>> At least  you can run the same command to obtain the stats regardless
>> of the driver and vendor, so from that perspective Linux continues to
>> be the abstraction and that is not broken.
> 
> Format of the data is no abstraction.
> 
-- 
Florian