CIP-dev Archive on lore.kernel.org
 help / color / Atom feed
* [cip-dev] FW: [Automated-testing] RFC: dashboards, visualization and analytics for test results
       [not found] <7hk0w2c4i3.fsf@baylibre.com>
@ 2020-10-13 14:16 ` Chris Paterson
  0 siblings, 0 replies; only message in thread
From: Chris Paterson @ 2020-10-13 14:16 UTC (permalink / raw)
  To: cip-dev


[-- Attachment #1: Type: text/plain, Size: 3149 bytes --]

FYI

Please feel free to add any use cases or comments to the mail thread below from KernelCI.

Kind regards, Chris

> From: automated-testing@lists.yoctoproject.org <automated-
> testing@lists.yoctoproject.org> On Behalf Of Kevin Hilman via
> lists.yoctoproject.org
> Sent: 07 October 2020 18:48
> To: automated-testing@lists.yoctoproject.org; kernelci@groups.io
> Cc: tools@linux.kernel.org
> Subject: [Automated-testing] RFC: dashboards, visualization and analytics for
> test results
> 
> Hello folks interested in kernel testing/automation,
> 
> The KernelCI project is starting to look at what's next for dashboards,
> visualization and analytics for the various Linux focused testing
> projects.
> 
> At Linux Plumbers, we launched some discussions[1] around common ways
> to
> collect test results, logs and metadata into a public, shared dataset,
> and we've already started collecting data from several different
> sources.
> 
> So the next question is... how do we best use all of this data?
> 
> We're beginning to brainstorm how to visualize, analyze and learn from
> this data in useful ways.
> 
> To that end, we're starting to collect a set of user stories to help us
> brainstorm a new design for web based dashboard and analytics, and we'd
> like to hear from you.
> 
> Below is the start of a list of user stories[2], but we want to grow
> this list with your ideas, so please share them on this thread.
> 
> We're also very interested in talking with any big data people and data
> scientists who might be willing to look at this growing set of data and
> help us better plan for the future full of lots of test data.
> 
> We appreciate your ideas and feedback,
> 
> Kevin (on behalf of the KernelCI team)
> 
> 
> [1] c.f. Unifying Test Reporting with KernelCI from the testing/fuzzing
>     micro-conference:
>     https://linuxplumbersconf.org/event/7/sessions/80/#20200826
> 
> 
> 
> [2] Example user stories
> 
> A kernel developer has sent a patch which caused a regression
>    - Find the details, how to reproduce, check when it’s fixed
> 
> A maintainer is getting a branch ready for the next merge window
>   - Compare results against mainline, ensure all tests were run correctly
> 
> An OEM or SoC vendor needs to upgrade their kernel or move to upstream
>   - See all results for a particular platform on various stable releases
> 
> Regular visitors who want to know how the kernel is doing
>   - Highlight new regressions, show trends for pass/fail and performance
>   - Does kernel X work on my hardware?
> 
> Distro vendor wants to know
>   - Is the latest kernel stable enough for my distro?
>   - Why does this test fail on my distro kernel but not stable kernel?
> 
> Stable kernel maintainer wants to know
>   - Is this kernel stable enough to release?
> 
> Kernel / subsystem maintainer wants to know
>   - Is my code/subsystem being tested?  How much coverage?
>   - Is the kernel size growing?  Why?  Which subsystems
>   - Is boot time / test execution time increasing/decreasing?  What are the
> historical trends?
> 


[-- Attachment #2: Type: text/plain, Size: 420 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5571): https://lists.cip-project.org/g/cip-dev/message/5571
Mute This Topic: https://lists.cip-project.org/mt/77483847/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-


^ permalink raw reply	[flat|nested] only message in thread

only message in thread, back to index

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <7hk0w2c4i3.fsf@baylibre.com>
2020-10-13 14:16 ` [cip-dev] FW: [Automated-testing] RFC: dashboards, visualization and analytics for test results Chris Paterson

CIP-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/cip-dev/0 cip-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 cip-dev cip-dev/ https://lore.kernel.org/cip-dev \
		cip-dev@lists.cip-project.org
	public-inbox-index cip-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.cip-project.lists.cip-dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git