Re: Firmware debugging patches?

From: Ben Greear <greearb@candelatech.com>
To: Emmanuel Grumbach <egrumbach@gmail.com>
Cc: Kalle Valo <kvalo@qca.qualcomm.com>,
	ath10k <ath10k@lists.infradead.org>,
	"linux-wireless@vger.kernel.org" <linux-wireless@vger.kernel.org>
Subject: Re: Firmware debugging patches?
Date: Mon, 02 Jun 2014 12:48:23 -0700	[thread overview]
Message-ID: <538CD507.1050803@candelatech.com> (raw)
In-Reply-To: <CANUX_P2vbKD3zkwE=DvJJWpo=gEMORCBy=XFoC2YSFpmg9sr_A@mail.gmail.com>

On 06/02/2014 12:29 PM, Emmanuel Grumbach wrote:

>>>> a) Normal Fedora/Ubuntu/etc default-installed distribution user
>>>> with ath10k NIC has wifi issues, firmware crashes, they don't
>>>> really know what firmware means or that it crashed, but some automated crash-log
>>>> tool notices and gathers debug info for automated bug reporting.
>>>
>>> I am working on that for our firmware. I recently added such capability relying on udev to notify the userspace that something bad happens. I gather all the data and prepare a binary file that is sent through debugfs (pulled by a script triggered by udev). I remember the first crash only.
>>
>> How is this binary blob encoded?
> 
> Different TLV based binary blobs concatenated. The actual encoding of
> each of them is another story.

Should we try to make the 'Type' in TLV be consistent as convenient
across different drivers?  That might someday help auto-reporting tools?

Do you have a link to your patch that defines the types you used?

>> At least for drivers that can recover from firmware crashes, I think
>> we should continue to report crashes, not just the first.
>>
> 
> I remember the first until udev kicks the script that will empty the
> buffer. Then I take the second crash's log.

Sounds good enough to me.

>> Maybe could store another one after initial crash has been read
>> and 1 minute has elapsed, or if initial crash has not been read
>> in 1 day, or something like that.
>>
>> Also, if we use debugfs then we require upstream kernels to have this
>> compiled in and mounted if we want to handle this class of user.
> 
> Agreed. I rely on debugfs. But this is "just" the way to reach the filesystem.
> Give me another way and I am fine with it.
> FWIW Ubuntu which is not exactly the distribution of the super
> advanced users has it mounted by default.

That sounds promising...Looks like Fedora 20 does as well, so maybe
debugfs will be good enough for crash dumps.

It does not resolve my interest in firmware logs interleaved with
kernel logs and possibly supplicant, however.

Looks like trace-cmds could do that, but it will not be
running for normal users when they experience crashes.

Any suggestions other than printk for this feature?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com