Re: [RFC 00/12] Support for sustained capturing of GuC firmware logs

[Daniel Vetter <daniel@ffwll.ch>]

On Sat, May 28, 2016 at 01:12:51AM +0530, akash.goel@intel.com wrote:
> From: Akash Goel <akash.goel@intel.com>
> 
> GuC firmware log its debug messages into a Host-GuC shared memory buffer
> and when the buffer is half full it sends a Flush interrupt to Host.
> GuC firmware expects that while it is writing to 2nd half of the buffer,
> first half would get consumed by Host and then get a flush completed
> acknowledgement from Host, so that it does not end up doing any overwrite
> causing loss of logs.
> So far Flush interrupt wasn't enabled on Host side & User could capture the
> contents/snapshot of log buffer through 'i915_guc_log_dump' debugfs iface.
> But this couldn't meet couple of key requirements, especially of Validation,
> first is to ensure capturing of all boot time logs even with high verbosity
> level and second is to enable capturing of logs in a sustained manner like
> for the entire duration of a workload.
> Now Driver will enable Flush interrupt and on receving it, would copy the
> contents of log buffer into its local buffer. The size of local buffer would
> be big enough to contain multiple snapshots of the log buffer giving ample
> time to User to pull boot time messages. Also reads from User space will be
> blocked if there isn't any data left in the local buffer until another
> Flush interrupt is received and new logs are copied into the local buffer,
> enabling capturing of logs in a streaming manner.
> 
> Have added 3 interfaces for capturing logs.
> Patch 5 & 6 avails relay framework to store logs and export that to User,
> in which Driver just needs to use a relay API to copy logs into the relay
> buffer and behavior is equivalent to 'dmesg -c'. But there are certain
> requirements with relay which in most likelihood can be met only for a
> debugfs file. But since debugfs may not be always available in production
> kernels, added support for other interfaces as a fall back, where Driver
> only will have to do allocation/mapping of local buffer pages, handle file
> read calls, read/write pointer management, overflow handling etc.
> One is a sysfs interface '/sys/class/drm/card0/guc_log', which is generally
> the preferred location. This mimics '/proc/kmsg' behavior, so meets the
> streaming requirement but at a time only single client can pull the logs
> in a consistent manner.
> Other is a character device file interface '/dev/dri/guc_log', this mimics
> '/dev/kmsg' behavior, so can meet the streaming requirement and also allow
> multiple clients to consume logs at the same time.
> Based on the comments and considering the relative merits, can decide which
> interface to actually use.

I still kinda like relayfs, except that it's not available in non-debug
builds. But so are plenty of other really interesting files we have hidden
in there.

sysfs isn't the solution, I already have a black eye from the sysfs
maintainer for our error state.

char dev feels like it's bad at discoverability, no one would guess you
can just cat it. Also, the relationship between driver instance and
chardev logging is a bit messy.

No idea really where to put stuff. One option might be to have an official
debug directory (like we have power already) in sysfs as the canonical
place where drivers can dump stuff. We're not the only ones with too much
data to get to userspace for debugging driver/hw issues, e.g. wireless
firmware has pretty similar solutions.

Adding Johannes, he's worked on something like this for the intel wireless
drivers iirc.

Cheers, Daniel

> 
> Akash Goel (7):
>   drm/i915: Add a relay backed debugfs interface for capturing GuC logs
>   drm/i915: Store GuC ukernel logs in the relay buffer
>   drm/i915: Allocate local buffer to store GuC ukernel logs
>   drm/i915: Store GuC ukernel logs in the local buffer
>   drm/i915: Add a char device file interface to capture GuC ukernel logs
>   drm/i915: Support to capture GuC logs by multiple clients via device
>     file iface
>   drm/i915: Add sysfs interface to capture the GuC ukernel logs
> 
> Sagar Arun Kamble (5):
>   drm/i915: Decouple GuC log setup from verbosity parameter
>   drm/i915: Add GuC ukernel logging related fields to fw interface file
>   drm/i915: Support for GuC interrupts
>   drm/i915: Handle log buffer flush interrupt event from GuC
>   drm/i915: Forcefully flush GuC log buffer on reset
> 
>  drivers/gpu/drm/i915/i915_drv.h            |   1 +
>  drivers/gpu/drm/i915/i915_guc_submission.c | 483 ++++++++++++++++++++++++++++-
>  drivers/gpu/drm/i915/i915_irq.c            | 120 ++++++-
>  drivers/gpu/drm/i915/i915_reg.h            |  11 +
>  drivers/gpu/drm/i915/i915_sysfs.c          |  40 +++
>  drivers/gpu/drm/i915/intel_drv.h           |   3 +
>  drivers/gpu/drm/i915/intel_guc.h           |  30 ++
>  drivers/gpu/drm/i915/intel_guc_fwif.h      |  42 +++
>  drivers/gpu/drm/i915/intel_guc_loader.c    |   9 +-
>  9 files changed, 730 insertions(+), 9 deletions(-)
> 
> -- 
> 1.9.2
> 
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx