All of lore.kernel.org
 help / color / mirror / Atom feed
* HW communication debugging interface - ideas?
@ 2015-09-30 13:51 Jiri Pirko
  2015-10-05  8:54 ` Jiri Pirko
                   ` (4 more replies)
  0 siblings, 5 replies; 23+ messages in thread
From: Jiri Pirko @ 2015-09-30 13:51 UTC (permalink / raw)
  To: davem; +Cc: netdev, eladr, idosch

Hi Dave/all.

There is quite common for drivers to speak with HW using "packet-like"
messages. In our case, we put message into skb and push that out as
an ordinary packet. HW then sends us reply in a packet, similar to other
rx-ed packets.

It makes sense to monitor this communication. Not only for debugging
during development, but mainly for collecting data for error reports
from users.

Very cool usecase is to teach wireshark to parse this communication so
one can actually see online what is going on.

Now the question is, what is the correct way to expose this
communication to userspace. Naturally, this interface should be read-only.
Also, the interface should be generic enough so it can be used for all
drivers, including non-net drivers.

1) Special monitoring netdev.
Might make sense to use a separate netdev. Is is easy to tap the
communication. Driver would just create new netdev and push incoming and
outgoing messages there. Downside is that there would be needed to wrap the
messages with some metadata header, which seems quite odd.

2) generic Netlink (genl) interface. Easy to put metadata in, including the
device identificator (pci address). User then can use nlmon in order to
be able to use wireshark to see the netlink messages.

Looks like 2) might be viable, well-defined, generic interface to carry
this info. What do you think? Does this make sense?

Thanks!

Jiri

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-09-30 13:51 HW communication debugging interface - ideas? Jiri Pirko
@ 2015-10-05  8:54 ` Jiri Pirko
  2015-10-05  9:38   ` David Miller
  2015-10-05  9:54 ` Rosen, Rami
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05  8:54 UTC (permalink / raw)
  To: davem; +Cc: netdev, eladr, idosch

Wed, Sep 30, 2015 at 03:51:41PM CEST, jiri@resnulli.us wrote:
>Hi Dave/all.
>
>There is quite common for drivers to speak with HW using "packet-like"
>messages. In our case, we put message into skb and push that out as
>an ordinary packet. HW then sends us reply in a packet, similar to other
>rx-ed packets.
>
>It makes sense to monitor this communication. Not only for debugging
>during development, but mainly for collecting data for error reports
>from users.
>
>Very cool usecase is to teach wireshark to parse this communication so
>one can actually see online what is going on.
>
>Now the question is, what is the correct way to expose this
>communication to userspace. Naturally, this interface should be read-only.
>Also, the interface should be generic enough so it can be used for all
>drivers, including non-net drivers.
>
>1) Special monitoring netdev.
>Might make sense to use a separate netdev. Is is easy to tap the
>communication. Driver would just create new netdev and push incoming and
>outgoing messages there. Downside is that there would be needed to wrap the
>messages with some metadata header, which seems quite odd.
>
>2) generic Netlink (genl) interface. Easy to put metadata in, including the
>device identificator (pci address). User then can use nlmon in order to
>be able to use wireshark to see the netlink messages.
>
>Looks like 2) might be viable, well-defined, generic interface to carry
>this info. What do you think? Does this make sense?
>
>Thanks!
>
>Jiri

Any ideas, anyone, Dave, please?

Thanks!

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05  8:54 ` Jiri Pirko
@ 2015-10-05  9:38   ` David Miller
  2015-10-05  9:45     ` Jiri Pirko
  0 siblings, 1 reply; 23+ messages in thread
From: David Miller @ 2015-10-05  9:38 UTC (permalink / raw)
  To: jiri; +Cc: netdev, eladr, idosch

From: Jiri Pirko <jiri@resnulli.us>
Date: Mon, 5 Oct 2015 10:54:06 +0200

> Any ideas, anyone, Dave, please?

I am very busy over the next few weeks and therefore it will take
me some time to look into this very deeply.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05  9:38   ` David Miller
@ 2015-10-05  9:45     ` Jiri Pirko
  0 siblings, 0 replies; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05  9:45 UTC (permalink / raw)
  To: David Miller; +Cc: netdev, eladr, idosch

Mon, Oct 05, 2015 at 11:38:53AM CEST, davem@davemloft.net wrote:
>From: Jiri Pirko <jiri@resnulli.us>
>Date: Mon, 5 Oct 2015 10:54:06 +0200
>
>> Any ideas, anyone, Dave, please?
>
>I am very busy over the next few weeks and therefore it will take
>me some time to look into this very deeply.

If you could spend like 5-10 minutes on this, that would be great. We would
like to start to implement this, but we are not sure which direction to
take. I'll owe you a beer then :)

Thanks!

^ permalink raw reply	[flat|nested] 23+ messages in thread

* RE: HW communication debugging interface - ideas?
  2015-09-30 13:51 HW communication debugging interface - ideas? Jiri Pirko
  2015-10-05  8:54 ` Jiri Pirko
@ 2015-10-05  9:54 ` Rosen, Rami
  2015-10-05  9:56   ` Jiri Pirko
  2015-10-05 12:28 ` Thomas Graf
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 23+ messages in thread
From: Rosen, Rami @ 2015-10-05  9:54 UTC (permalink / raw)
  To: Jiri Pirko, davem; +Cc: netdev, eladr, idosch

Hi, Jiri,

>In our case, we put message into skb and  ...

Just to be sure we are on the same page:

By "In our case" - I assume you are referring to the mlxsw Ethernet switch driver, right ? 

>we put message into skb and push that out as an ordinary packet. >HW then > sends us reply in a packet, similar to other rx-ed packets.

Are you referring here to messages of the EMAD protocol ?

Rami Rosen
Intel Corporation

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05  9:54 ` Rosen, Rami
@ 2015-10-05  9:56   ` Jiri Pirko
  2015-10-05 14:49     ` Andrew Lunn
  0 siblings, 1 reply; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05  9:56 UTC (permalink / raw)
  To: Rosen, Rami; +Cc: davem, netdev, eladr, idosch

Mon, Oct 05, 2015 at 11:54:38AM CEST, rami.rosen@intel.com wrote:
>Hi, Jiri,
>
>>In our case, we put message into skb and  ...
>
>Just to be sure we are on the same page:
>
>By "In our case" - I assume you are referring to the mlxsw Ethernet switch driver, right ? 

Yes.

>
>>we put message into skb and push that out as an ordinary packet. >HW then > sends us reply in a packet, similar to other rx-ed packets.
>
>Are you referring here to messages of the EMAD protocol ?

Yes.

>
>Rami Rosen
>Intel Corporation
>--
>To unsubscribe from this list: send the line "unsubscribe netdev" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-09-30 13:51 HW communication debugging interface - ideas? Jiri Pirko
  2015-10-05  8:54 ` Jiri Pirko
  2015-10-05  9:54 ` Rosen, Rami
@ 2015-10-05 12:28 ` Thomas Graf
  2015-10-05 18:32 ` Marcelo Ricardo Leitner
  2015-11-01 16:51 ` David Miller
  4 siblings, 0 replies; 23+ messages in thread
From: Thomas Graf @ 2015-10-05 12:28 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: davem, netdev, eladr, idosch

On 09/30/15 at 03:51pm, Jiri Pirko wrote:
> 2) generic Netlink (genl) interface. Easy to put metadata in, including the
> device identificator (pci address). User then can use nlmon in order to
> be able to use wireshark to see the netlink messages.
> 
> Looks like 2) might be viable, well-defined, generic interface to carry
> this info. What do you think? Does this make sense?

+1 since we already have nlmon and Netlink is our standard
control plane protocol.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05  9:56   ` Jiri Pirko
@ 2015-10-05 14:49     ` Andrew Lunn
  2015-10-05 14:55       ` Jiri Pirko
  0 siblings, 1 reply; 23+ messages in thread
From: Andrew Lunn @ 2015-10-05 14:49 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: Rosen, Rami, davem, netdev, eladr, idosch

> >Are you referring here to messages of the EMAD protocol ?

I know nothing about this protocol.....

Does it at least use standard Ethernet framing? Source and Destination
header and an EtherType which mean EMAD?

Marvell has a way to access registers of their switches using Ethernet
Frames, and they are valid frames with a specific EtherType. Linux
does not use them, but if it did, i would of thought passing them
along using the normal pcap interface would be trivial.

      Andrew

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 14:49     ` Andrew Lunn
@ 2015-10-05 14:55       ` Jiri Pirko
  2015-10-05 14:58         ` Andrew Lunn
  0 siblings, 1 reply; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05 14:55 UTC (permalink / raw)
  To: Andrew Lunn; +Cc: Rosen, Rami, davem, netdev, eladr, idosch

Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>> >Are you referring here to messages of the EMAD protocol ?
>
>I know nothing about this protocol.....
>
>Does it at least use standard Ethernet framing? Source and Destination
>header and an EtherType which mean EMAD?

Yep, but that does not really matter. I believe we should find debugging
interface which is protocol agnostic. Just arbitrary messages
monitoring.


>
>Marvell has a way to access registers of their switches using Ethernet
>Frames, and they are valid frames with a specific EtherType. Linux
>does not use them, but if it did, i would of thought passing them
>along using the normal pcap interface would be trivial.
>
>      Andrew

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 14:55       ` Jiri Pirko
@ 2015-10-05 14:58         ` Andrew Lunn
  2015-10-05 15:18           ` Jiri Pirko
  0 siblings, 1 reply; 23+ messages in thread
From: Andrew Lunn @ 2015-10-05 14:58 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: Rosen, Rami, davem, netdev, eladr, idosch

On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
> >> >Are you referring here to messages of the EMAD protocol ?
> >
> >I know nothing about this protocol.....
> >
> >Does it at least use standard Ethernet framing? Source and Destination
> >header and an EtherType which mean EMAD?
> 
> Yep, but that does not really matter. I believe we should find debugging
> interface which is protocol agnostic. Just arbitrary messages
> monitoring.

Hi Jiri

O.K, it is just that you mentioned wireshark. Passing the frames to
network interface taps would make this trivial.

	Andrew

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 14:58         ` Andrew Lunn
@ 2015-10-05 15:18           ` Jiri Pirko
  2015-10-05 15:29             ` John Fastabend
  0 siblings, 1 reply; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05 15:18 UTC (permalink / raw)
  To: Andrew Lunn; +Cc: Rosen, Rami, davem, netdev, eladr, idosch

Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>> >> >Are you referring here to messages of the EMAD protocol ?
>> >
>> >I know nothing about this protocol.....
>> >
>> >Does it at least use standard Ethernet framing? Source and Destination
>> >header and an EtherType which mean EMAD?
>> 
>> Yep, but that does not really matter. I believe we should find debugging
>> interface which is protocol agnostic. Just arbitrary messages
>> monitoring.
>
>Hi Jiri
>
>O.K, it is just that you mentioned wireshark. Passing the frames to
>network interface taps would make this trivial.

That is true. But using netlink+nlmon would do the same.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 15:18           ` Jiri Pirko
@ 2015-10-05 15:29             ` John Fastabend
  2015-10-05 15:35               ` Jiri Pirko
  0 siblings, 1 reply; 23+ messages in thread
From: John Fastabend @ 2015-10-05 15:29 UTC (permalink / raw)
  To: Jiri Pirko, Andrew Lunn; +Cc: Rosen, Rami, davem, netdev, eladr, idosch

On 15-10-05 08:18 AM, Jiri Pirko wrote:
> Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>> On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>>>>>> Are you referring here to messages of the EMAD protocol ?
>>>>
>>>> I know nothing about this protocol.....
>>>>
>>>> Does it at least use standard Ethernet framing? Source and Destination
>>>> header and an EtherType which mean EMAD?
>>>
>>> Yep, but that does not really matter. I believe we should find debugging
>>> interface which is protocol agnostic. Just arbitrary messages
>>> monitoring.
>>
>> Hi Jiri
>>
>> O.K, it is just that you mentioned wireshark. Passing the frames to
>> network interface taps would make this trivial.
> 
> That is true. But using netlink+nlmon would do the same.

Also I guess if you go this direction you want to make it generic
enough for any drivers to use it to snoop software/firmware msgs. This
is common across many devices.

In the past though I've just used ethtool dump commands and some
"scripts" on top of this to debug devices. And when it got really
bad wrote some throw away code to debug my issue. I guess it might
be nice to have something in the kernel to improve this but have
you considered using the tracing features that already exist?

.John

> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 15:29             ` John Fastabend
@ 2015-10-05 15:35               ` Jiri Pirko
  2015-10-05 15:47                 ` John Fastabend
  0 siblings, 1 reply; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05 15:35 UTC (permalink / raw)
  To: John Fastabend; +Cc: Andrew Lunn, Rosen, Rami, davem, netdev, eladr, idosch

Mon, Oct 05, 2015 at 05:29:09PM CEST, john.fastabend@gmail.com wrote:
>On 15-10-05 08:18 AM, Jiri Pirko wrote:
>> Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>>> On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>>>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>>>>>>> Are you referring here to messages of the EMAD protocol ?
>>>>>
>>>>> I know nothing about this protocol.....
>>>>>
>>>>> Does it at least use standard Ethernet framing? Source and Destination
>>>>> header and an EtherType which mean EMAD?
>>>>
>>>> Yep, but that does not really matter. I believe we should find debugging
>>>> interface which is protocol agnostic. Just arbitrary messages
>>>> monitoring.
>>>
>>> Hi Jiri
>>>
>>> O.K, it is just that you mentioned wireshark. Passing the frames to
>>> network interface taps would make this trivial.
>> 
>> That is true. But using netlink+nlmon would do the same.
>
>Also I guess if you go this direction you want to make it generic
>enough for any drivers to use it to snoop software/firmware msgs. This
>is common across many devices.

Yes, definitelly, this should be something generic to be usable for
every device type.


>
>In the past though I've just used ethtool dump commands and some
>"scripts" on top of this to debug devices. And when it got really
>bad wrote some throw away code to debug my issue. I guess it might
>be nice to have something in the kernel to improve this but have
>you considered using the tracing features that already exist?

Which ones do you have in mind?


>
>.John
>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 15:35               ` Jiri Pirko
@ 2015-10-05 15:47                 ` John Fastabend
  2015-10-06  8:14                   ` Jiri Pirko
  0 siblings, 1 reply; 23+ messages in thread
From: John Fastabend @ 2015-10-05 15:47 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: Andrew Lunn, Rosen, Rami, davem, netdev, eladr, idosch

On 15-10-05 08:35 AM, Jiri Pirko wrote:
> Mon, Oct 05, 2015 at 05:29:09PM CEST, john.fastabend@gmail.com wrote:
>> On 15-10-05 08:18 AM, Jiri Pirko wrote:
>>> Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>>>> On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>>>>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>>>>>>>> Are you referring here to messages of the EMAD protocol ?
>>>>>>
>>>>>> I know nothing about this protocol.....
>>>>>>
>>>>>> Does it at least use standard Ethernet framing? Source and Destination
>>>>>> header and an EtherType which mean EMAD?
>>>>>
>>>>> Yep, but that does not really matter. I believe we should find debugging
>>>>> interface which is protocol agnostic. Just arbitrary messages
>>>>> monitoring.
>>>>
>>>> Hi Jiri
>>>>
>>>> O.K, it is just that you mentioned wireshark. Passing the frames to
>>>> network interface taps would make this trivial.
>>>
>>> That is true. But using netlink+nlmon would do the same.
>>
>> Also I guess if you go this direction you want to make it generic
>> enough for any drivers to use it to snoop software/firmware msgs. This
>> is common across many devices.
> 
> Yes, definitelly, this should be something generic to be usable for
> every device type.
> 
> 
>>
>> In the past though I've just used ethtool dump commands and some
>> "scripts" on top of this to debug devices. And when it got really
>> bad wrote some throw away code to debug my issue. I guess it might
>> be nice to have something in the kernel to improve this but have
>> you considered using the tracing features that already exist?
> 
> Which ones do you have in mind?
> 

I was thinking something like kprobes+bpf to dump a trace and
then a lua script in wireshark to parse the input and pretty
print it for users. This might get you good-enough support without
having to carry it around in the kernel just so we can debug
the devices. We could build some libs/pkgs around it in userspace
and get it published somewhere so we can all work on it together.

I suspect the primary users for this will be developers anyways
and maybe some users who have a real nasty bug.

> 
>>
>> .John
>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-09-30 13:51 HW communication debugging interface - ideas? Jiri Pirko
                   ` (2 preceding siblings ...)
  2015-10-05 12:28 ` Thomas Graf
@ 2015-10-05 18:32 ` Marcelo Ricardo Leitner
  2015-10-05 20:40   ` Jiri Pirko
  2015-11-01 16:51 ` David Miller
  4 siblings, 1 reply; 23+ messages in thread
From: Marcelo Ricardo Leitner @ 2015-10-05 18:32 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: davem, netdev, eladr, idosch

On Wed, Sep 30, 2015 at 03:51:41PM +0200, Jiri Pirko wrote:
> Hi Dave/all.
> 
> There is quite common for drivers to speak with HW using "packet-like"
> messages. In our case, we put message into skb and push that out as
> an ordinary packet. HW then sends us reply in a packet, similar to other
> rx-ed packets.

Better sure than sorry so I'll ask. It's not possible for an application
to inject such skbs somehow, right?  Like, a guest, tied to a bridge,
tied to a port which ends up understanding these skbs.

...
> 2) generic Netlink (genl) interface. Easy to put metadata in, including the
> device identificator (pci address). User then can use nlmon in order to
> be able to use wireshark to see the netlink messages.
> 
> Looks like 2) might be viable, well-defined, generic interface to carry
> this info. What do you think? Does this make sense?

2 seems very interesting, fwiw

  Marcelo

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 18:32 ` Marcelo Ricardo Leitner
@ 2015-10-05 20:40   ` Jiri Pirko
  0 siblings, 0 replies; 23+ messages in thread
From: Jiri Pirko @ 2015-10-05 20:40 UTC (permalink / raw)
  To: Marcelo Ricardo Leitner; +Cc: davem, netdev, eladr, idosch

Mon, Oct 05, 2015 at 08:32:33PM CEST, marcelo.leitner@gmail.com wrote:
>On Wed, Sep 30, 2015 at 03:51:41PM +0200, Jiri Pirko wrote:
>> Hi Dave/all.
>> 
>> There is quite common for drivers to speak with HW using "packet-like"
>> messages. In our case, we put message into skb and push that out as
>> an ordinary packet. HW then sends us reply in a packet, similar to other
>> rx-ed packets.
>
>Better sure than sorry so I'll ask. It's not possible for an application
>to inject such skbs somehow, right?  Like, a guest, tied to a bridge,
>tied to a port which ends up understanding these skbs.

No, it's purely for monitoring of communication with HW.


>
>...
>> 2) generic Netlink (genl) interface. Easy to put metadata in, including the
>> device identificator (pci address). User then can use nlmon in order to
>> be able to use wireshark to see the netlink messages.
>> 
>> Looks like 2) might be viable, well-defined, generic interface to carry
>> this info. What do you think? Does this make sense?
>
>2 seems very interesting, fwiw
>
>  Marcelo

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-05 15:47                 ` John Fastabend
@ 2015-10-06  8:14                   ` Jiri Pirko
  2015-10-06 14:54                     ` John Fastabend
  0 siblings, 1 reply; 23+ messages in thread
From: Jiri Pirko @ 2015-10-06  8:14 UTC (permalink / raw)
  To: John Fastabend; +Cc: Andrew Lunn, Rosen, Rami, davem, netdev, eladr, idosch

Mon, Oct 05, 2015 at 05:47:09PM CEST, john.fastabend@gmail.com wrote:
>On 15-10-05 08:35 AM, Jiri Pirko wrote:
>> Mon, Oct 05, 2015 at 05:29:09PM CEST, john.fastabend@gmail.com wrote:
>>> On 15-10-05 08:18 AM, Jiri Pirko wrote:
>>>> Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>>>>> On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>>>>>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>>>>>>>>> Are you referring here to messages of the EMAD protocol ?
>>>>>>>
>>>>>>> I know nothing about this protocol.....
>>>>>>>
>>>>>>> Does it at least use standard Ethernet framing? Source and Destination
>>>>>>> header and an EtherType which mean EMAD?
>>>>>>
>>>>>> Yep, but that does not really matter. I believe we should find debugging
>>>>>> interface which is protocol agnostic. Just arbitrary messages
>>>>>> monitoring.
>>>>>
>>>>> Hi Jiri
>>>>>
>>>>> O.K, it is just that you mentioned wireshark. Passing the frames to
>>>>> network interface taps would make this trivial.
>>>>
>>>> That is true. But using netlink+nlmon would do the same.
>>>
>>> Also I guess if you go this direction you want to make it generic
>>> enough for any drivers to use it to snoop software/firmware msgs. This
>>> is common across many devices.
>> 
>> Yes, definitelly, this should be something generic to be usable for
>> every device type.
>> 
>> 
>>>
>>> In the past though I've just used ethtool dump commands and some
>>> "scripts" on top of this to debug devices. And when it got really
>>> bad wrote some throw away code to debug my issue. I guess it might
>>> be nice to have something in the kernel to improve this but have
>>> you considered using the tracing features that already exist?
>> 
>> Which ones do you have in mind?
>> 
>
>I was thinking something like kprobes+bpf to dump a trace and
>then a lua script in wireshark to parse the input and pretty
>print it for users. This might get you good-enough support without
>having to carry it around in the kernel just so we can debug
>the devices. We could build some libs/pkgs around it in userspace
>and get it published somewhere so we can all work on it together.

Well, I was thinking rather about some standard interface, not dependent
on actual kernel internals.


>
>I suspect the primary users for this will be developers anyways
>and maybe some users who have a real nasty bug.
>
>> 
>>>
>>> .John
>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-06  8:14                   ` Jiri Pirko
@ 2015-10-06 14:54                     ` John Fastabend
  2015-10-06 15:02                       ` Jiri Pirko
  2015-10-06 15:02                       ` Andrew Lunn
  0 siblings, 2 replies; 23+ messages in thread
From: John Fastabend @ 2015-10-06 14:54 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: Andrew Lunn, Rosen, Rami, davem, netdev, eladr, idosch

On 15-10-06 01:14 AM, Jiri Pirko wrote:
> Mon, Oct 05, 2015 at 05:47:09PM CEST, john.fastabend@gmail.com wrote:
>> On 15-10-05 08:35 AM, Jiri Pirko wrote:
>>> Mon, Oct 05, 2015 at 05:29:09PM CEST, john.fastabend@gmail.com wrote:
>>>> On 15-10-05 08:18 AM, Jiri Pirko wrote:
>>>>> Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>>>>>> On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>>>>>>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>>>>>>>>>> Are you referring here to messages of the EMAD protocol ?
>>>>>>>>
>>>>>>>> I know nothing about this protocol.....
>>>>>>>>
>>>>>>>> Does it at least use standard Ethernet framing? Source and Destination
>>>>>>>> header and an EtherType which mean EMAD?
>>>>>>>
>>>>>>> Yep, but that does not really matter. I believe we should find debugging
>>>>>>> interface which is protocol agnostic. Just arbitrary messages
>>>>>>> monitoring.
>>>>>>
>>>>>> Hi Jiri
>>>>>>
>>>>>> O.K, it is just that you mentioned wireshark. Passing the frames to
>>>>>> network interface taps would make this trivial.
>>>>>
>>>>> That is true. But using netlink+nlmon would do the same.
>>>>
>>>> Also I guess if you go this direction you want to make it generic
>>>> enough for any drivers to use it to snoop software/firmware msgs. This
>>>> is common across many devices.
>>>
>>> Yes, definitelly, this should be something generic to be usable for
>>> every device type.
>>>
>>>
>>>>
>>>> In the past though I've just used ethtool dump commands and some
>>>> "scripts" on top of this to debug devices. And when it got really
>>>> bad wrote some throw away code to debug my issue. I guess it might
>>>> be nice to have something in the kernel to improve this but have
>>>> you considered using the tracing features that already exist?
>>>
>>> Which ones do you have in mind?
>>>
>>
>> I was thinking something like kprobes+bpf to dump a trace and
>> then a lua script in wireshark to parse the input and pretty
>> print it for users. This might get you good-enough support without
>> having to carry it around in the kernel just so we can debug
>> the devices. We could build some libs/pkgs around it in userspace
>> and get it published somewhere so we can all work on it together.
> 
> Well, I was thinking rather about some standard interface, not dependent
> on actual kernel internals.
> 


Sure just throwing out an idea. I suspect whatever interface you have
will include the vendor-id or some other identifier and a set of
parsers in user space to pretty print the msg.

.John

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-06 14:54                     ` John Fastabend
@ 2015-10-06 15:02                       ` Jiri Pirko
  2015-10-06 15:02                       ` Andrew Lunn
  1 sibling, 0 replies; 23+ messages in thread
From: Jiri Pirko @ 2015-10-06 15:02 UTC (permalink / raw)
  To: John Fastabend; +Cc: Andrew Lunn, Rosen, Rami, davem, netdev, eladr, idosch

Tue, Oct 06, 2015 at 04:54:09PM CEST, john.fastabend@gmail.com wrote:
>On 15-10-06 01:14 AM, Jiri Pirko wrote:
>> Mon, Oct 05, 2015 at 05:47:09PM CEST, john.fastabend@gmail.com wrote:
>>> On 15-10-05 08:35 AM, Jiri Pirko wrote:
>>>> Mon, Oct 05, 2015 at 05:29:09PM CEST, john.fastabend@gmail.com wrote:
>>>>> On 15-10-05 08:18 AM, Jiri Pirko wrote:
>>>>>> Mon, Oct 05, 2015 at 04:58:42PM CEST, andrew@lunn.ch wrote:
>>>>>>> On Mon, Oct 05, 2015 at 04:55:42PM +0200, Jiri Pirko wrote:
>>>>>>>> Mon, Oct 05, 2015 at 04:49:41PM CEST, andrew@lunn.ch wrote:
>>>>>>>>>>> Are you referring here to messages of the EMAD protocol ?
>>>>>>>>>
>>>>>>>>> I know nothing about this protocol.....
>>>>>>>>>
>>>>>>>>> Does it at least use standard Ethernet framing? Source and Destination
>>>>>>>>> header and an EtherType which mean EMAD?
>>>>>>>>
>>>>>>>> Yep, but that does not really matter. I believe we should find debugging
>>>>>>>> interface which is protocol agnostic. Just arbitrary messages
>>>>>>>> monitoring.
>>>>>>>
>>>>>>> Hi Jiri
>>>>>>>
>>>>>>> O.K, it is just that you mentioned wireshark. Passing the frames to
>>>>>>> network interface taps would make this trivial.
>>>>>>
>>>>>> That is true. But using netlink+nlmon would do the same.
>>>>>
>>>>> Also I guess if you go this direction you want to make it generic
>>>>> enough for any drivers to use it to snoop software/firmware msgs. This
>>>>> is common across many devices.
>>>>
>>>> Yes, definitelly, this should be something generic to be usable for
>>>> every device type.
>>>>
>>>>
>>>>>
>>>>> In the past though I've just used ethtool dump commands and some
>>>>> "scripts" on top of this to debug devices. And when it got really
>>>>> bad wrote some throw away code to debug my issue. I guess it might
>>>>> be nice to have something in the kernel to improve this but have
>>>>> you considered using the tracing features that already exist?
>>>>
>>>> Which ones do you have in mind?
>>>>
>>>
>>> I was thinking something like kprobes+bpf to dump a trace and
>>> then a lua script in wireshark to parse the input and pretty
>>> print it for users. This might get you good-enough support without
>>> having to carry it around in the kernel just so we can debug
>>> the devices. We could build some libs/pkgs around it in userspace
>>> and get it published somewhere so we can all work on it together.
>> 
>> Well, I was thinking rather about some standard interface, not dependent
>> on actual kernel internals.
>> 
>
>
>Sure just throwing out an idea. I suspect whatever interface you have
>will include the vendor-id or some other identifier and a set of
>parsers in user space to pretty print the msg.

Yes, that is the plan.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-06 14:54                     ` John Fastabend
  2015-10-06 15:02                       ` Jiri Pirko
@ 2015-10-06 15:02                       ` Andrew Lunn
  2015-11-01 17:58                         ` Guy Harris
  1 sibling, 1 reply; 23+ messages in thread
From: Andrew Lunn @ 2015-10-06 15:02 UTC (permalink / raw)
  To: John Fastabend; +Cc: Jiri Pirko, Rosen, Rami, davem, netdev, eladr, idosch

> Sure just throwing out an idea. I suspect whatever interface you have
> will include the vendor-id or some other identifier and a set of
> parsers in user space to pretty print the msg.

If you are going to use wireshark, in this case, all you need to do is
make the stream as being Ethernet frames. The destination and
Ethertype tell you all you need to know to identify the protocol.

	  Andrew

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-09-30 13:51 HW communication debugging interface - ideas? Jiri Pirko
                   ` (3 preceding siblings ...)
  2015-10-05 18:32 ` Marcelo Ricardo Leitner
@ 2015-11-01 16:51 ` David Miller
  2015-11-02  6:13   ` Jiri Pirko
  4 siblings, 1 reply; 23+ messages in thread
From: David Miller @ 2015-11-01 16:51 UTC (permalink / raw)
  To: jiri; +Cc: netdev, eladr, idosch

From: Jiri Pirko <jiri@resnulli.us>
Date: Wed, 30 Sep 2015 15:51:41 +0200

> 2) generic Netlink (genl) interface. Easy to put metadata in, including the
> device identificator (pci address). User then can use nlmon in order to
> be able to use wireshark to see the netlink messages.
> 
> Looks like 2) might be viable, well-defined, generic interface to carry
> this info. What do you think? Does this make sense?

Sorry for the late response.  I definitely prefer this netlink idea,
particularly for the metadata aspect.

Yes, the ethernet frames should "identify" the device, but I much more
like the idea of precisely having the geographic ID of the device (PCI
ID, whatever) available as metadata as well.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-10-06 15:02                       ` Andrew Lunn
@ 2015-11-01 17:58                         ` Guy Harris
  0 siblings, 0 replies; 23+ messages in thread
From: Guy Harris @ 2015-11-01 17:58 UTC (permalink / raw)
  To: Andrew Lunn
  Cc: John Fastabend, Jiri Pirko, Rosen, Rami, davem, netdev, eladr, idosch


On Oct 6, 2015, at 8:02 AM, Andrew Lunn <andrew@lunn.ch> wrote:

>> Sure just throwing out an idea. I suspect whatever interface you have
>> will include the vendor-id or some other identifier and a set of
>> parsers in user space to pretty print the msg.
> 
> If you are going to use wireshark, in this case, all you need to do is
> make the stream as being Ethernet frames. The destination and
> Ethertype tell you all you need to know to identify the protocol.

You will, of course, have registered that Ethertype with the IEEE, to ensure nobody else registers it for use with a different protocol.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: HW communication debugging interface - ideas?
  2015-11-01 16:51 ` David Miller
@ 2015-11-02  6:13   ` Jiri Pirko
  0 siblings, 0 replies; 23+ messages in thread
From: Jiri Pirko @ 2015-11-02  6:13 UTC (permalink / raw)
  To: David Miller; +Cc: netdev, eladr, idosch

Sun, Nov 01, 2015 at 05:51:16PM CET, davem@davemloft.net wrote:
>From: Jiri Pirko <jiri@resnulli.us>
>Date: Wed, 30 Sep 2015 15:51:41 +0200
>
>> 2) generic Netlink (genl) interface. Easy to put metadata in, including the
>> device identificator (pci address). User then can use nlmon in order to
>> be able to use wireshark to see the netlink messages.
>> 
>> Looks like 2) might be viable, well-defined, generic interface to carry
>> this info. What do you think? Does this make sense?
>
>Sorry for the late response.  I definitely prefer this netlink idea,
>particularly for the metadata aspect.
>
>Yes, the ethernet frames should "identify" the device, but I much more
>like the idea of precisely having the geographic ID of the device (PCI
>ID, whatever) available as metadata as well.

Okay. Thanks!

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2015-11-02  6:13 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-30 13:51 HW communication debugging interface - ideas? Jiri Pirko
2015-10-05  8:54 ` Jiri Pirko
2015-10-05  9:38   ` David Miller
2015-10-05  9:45     ` Jiri Pirko
2015-10-05  9:54 ` Rosen, Rami
2015-10-05  9:56   ` Jiri Pirko
2015-10-05 14:49     ` Andrew Lunn
2015-10-05 14:55       ` Jiri Pirko
2015-10-05 14:58         ` Andrew Lunn
2015-10-05 15:18           ` Jiri Pirko
2015-10-05 15:29             ` John Fastabend
2015-10-05 15:35               ` Jiri Pirko
2015-10-05 15:47                 ` John Fastabend
2015-10-06  8:14                   ` Jiri Pirko
2015-10-06 14:54                     ` John Fastabend
2015-10-06 15:02                       ` Jiri Pirko
2015-10-06 15:02                       ` Andrew Lunn
2015-11-01 17:58                         ` Guy Harris
2015-10-05 12:28 ` Thomas Graf
2015-10-05 18:32 ` Marcelo Ricardo Leitner
2015-10-05 20:40   ` Jiri Pirko
2015-11-01 16:51 ` David Miller
2015-11-02  6:13   ` Jiri Pirko

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.