[Ksummit-discuss] [3a/5] reporting-issues: reference section, main guide

[Thorsten Leemhuis <linux@leemhuis.info>]

On 26.03.21 07:13, Thorsten Leemhuis wrote:
> 
> Lo! Since a few months mainline in
> Documentation/admin-guide/reporting-issues.rst contains a text written
> to obsolete the good old reporting-bugs text. For now, the new document
> still contains a warning at the top that basically says "this is WIP".
> But I'd like to remove that warning and delete reporting-bugs.rst in the
> next merge window to make reporting-issues.rst fully official. With this
> mail I want to give everyone a chance to take a look at the text and
> speak up if you don't want me to move ahead for now.
> 
> For easier review I'll post the text of reporting-issues.rst in reply to
> this mail. I'll do that in a few chunks, as if this was a cover letter
> for a patch-set. 

Part 3 was to big for ksummit-discuss, so here it is the first half of it:

Reference section: Reporting issues to the kernel maintainers

=============================================================



The detailed guides above outline all the major steps in brief fashion, which

should be enough for most people. But sometimes there are situations where even

experienced users might wonder how to actually do one of those steps. That's

what this section is for, as it will provide a lot more details on each of the

above steps. Consider this as reference documentation: it's possible to read it

from top to bottom. But it's mainly meant to skim over and a place to look up

details how to actually perform those steps.



A few words of general advice before digging into the details:



 * The Linux kernel developers are well aware this process is complicated and

   demands more than other FLOSS projects. We'd love to make it simpler. But

   that would require work in various places as well as some infrastructure,

   which would need constant maintenance; nobody has stepped up to do that

   work, so that's just how things are for now.



 * A warranty or support contract with some vendor doesn't entitle you to

   request fixes from developers in the upstream Linux kernel community: such

   contracts are completely outside the scope of the Linux kernel, its

   development community, and this document. That's why you can't demand

   anything such a contract guarantees in this context, not even if the

   developer handling the issue works for the vendor in question. If you want

   to claim your rights, use the vendor's support channel instead. When doing

   so, you might want to mention you'd like to see the issue fixed in the

   upstream Linux kernel; motivate them by saying it's the only way to ensure

   the fix in the end will get incorporated in all Linux distributions.



 * If you never reported an issue to a FLOSS project before you should consider

   reading `How to Report Bugs Effectively

   <https://www.chiark.greenend.org.uk/~sgtatham/bugs.html>`_, `How To Ask

   Questions The Smart Way

   <http://www.catb.org/esr/faqs/smart-questions.html>`_, and `How to ask good

   questions <https://jvns.ca/blog/good-questions/>`_.



With that off the table, find below the details on how to properly report

issues to the Linux kernel developers.





Make sure you're using the upstream Linux kernel

------------------------------------------------



   *Are you facing an issue with a Linux kernel a hardware or software vendor

   provided? Then in almost all cases you are better off to stop reading this

   document and reporting the issue to your vendor instead, unless you are

   willing to install the latest Linux version yourself. Be aware the latter

   will often be needed anyway to hunt down and fix issues.*



Like most programmers, Linux kernel developers don't like to spend time dealing

with reports for issues that don't even happen with their current code. It's

just a waste everybody's time, especially yours. Unfortunately such situations

easily happen when it comes to the kernel and often leads to frustration on both

sides. That's because almost all Linux-based kernels pre-installed on devices

(Computers, Laptops, Smartphones, Routers, …) and most shipped by Linux

distributors are quite distant from the official Linux kernel as distributed by

kernel.org: these kernels from these vendors are often ancient from the point of

Linux development or heavily modified, often both.



Most of these vendor kernels are quite unsuitable for reporting bugs to the

Linux kernel developers: an issue you face with one of them might have been

fixed by the Linux kernel developers months or years ago already; additionally,

the modifications and enhancements by the vendor might be causing the issue you

face, even if they look small or totally unrelated. That's why you should report

issues with these kernels to the vendor. Its developers should look into the

report and, in case it turns out to be an upstream issue, fix it directly

upstream or forward the report there. In practice that often does not work out

or might not what you want. You thus might want to consider circumventing the

vendor by installing the very latest Linux kernel core yourself. If that's an

option for you move ahead in this process, as a later step in this guide will

explain how to do that once it rules out other potential causes for your issue.



Note, the previous paragraph is starting with the word 'most', as sometimes

developers in fact are willing to handle reports about issues occurring with

vendor kernels. If they do in the end highly depends on the developers and the

issue in question. Your chances are quite good if the distributor applied only

small modifications to a kernel based on a recent Linux version; that for

example often holds true for the mainline kernels shipped by Debian GNU/Linux

Sid or Fedora Rawhide. Some developers will also accept reports about issues

with kernels from distributions shipping the latest stable kernel, as long as

its only slightly modified; that for example is often the case for Arch Linux,

regular Fedora releases, and openSUSE Tumbleweed. But keep in mind, you better

want to use a mainline Linux and avoid using a stable kernel for this

process, as outlined in the section 'Install a fresh kernel for testing' in more

detail.



Obviously you are free to ignore all this advice and report problems with an old

or heavily modified vendor kernel to the upstream Linux developers. But note,

those often get rejected or ignored, so consider yourself warned. But it's still

better than not reporting the issue at all: sometimes such reports directly or

indirectly will help to get the issue fixed over time.





Search for existing reports, first run

--------------------------------------



   *Perform a rough search for existing reports with your favorite internet

   search engine; additionally, check the archives of the Linux Kernel Mailing

   List (LKML). If you find matching reports, join the discussion instead of

   sending a new one.*



Reporting an issue that someone else already brought forward is often a waste of

time for everyone involved, especially you as the reporter. So it's in your own

interest to thoroughly check if somebody reported the issue already. At this

step of the process it's okay to just perform a rough search: a later step will

tell you to perform a more detailed search once you know where your issue needs

to be reported to. Nevertheless, do not hurry with this step of the reporting

process, it can save you time and trouble.



Simply search the internet with your favorite search engine first. Afterwards,

search the `Linux Kernel Mailing List (LKML) archives

<https://lore.kernel.org/lkml/>`_.



If you get flooded with results consider telling your search engine to limit

search timeframe to the past month or year. And wherever you search, make sure

to use good search terms; vary them a few times, too. While doing so try to

look at the issue from the perspective of someone else: that will help you to

come up with other words to use as search terms. Also make sure not to use too

many search terms at once. Remember to search with and without information like

the name of the kernel driver or the name of the affected hardware component.

But its exact brand name (say 'ASUS Red Devil Radeon RX 5700 XT Gaming OC')

often is not much helpful, as it is too specific. Instead try search terms like

the model line (Radeon 5700 or Radeon 5000) and the code name of the main chip

('Navi' or 'Navi10') with and without its manufacturer ('AMD').



In case you find an existing report about your issue, join the discussion, as

you might be able to provide valuable additional information. That can be

important even when a fix is prepared or in its final stages already, as

developers might look for people that can provide additional information or

test a proposed fix. Jump to the section 'Duties after the report went out' for

details on how to get properly involved.



Note, searching `bugzilla.kernel.org <https://bugzilla.kernel.org/>`_ might also

be a good idea, as that might provide valuable insights or turn up matching

reports. If you find the latter, just keep in mind: most subsystems expect

reports in different places, as described below in the section "Check where you

need to report your issue". The developers that should take care of the issue

thus might not even be aware of the bugzilla ticket. Hence, check the ticket if

the issue already got reported as outlined in this document and if not consider

doing so.





Issue of high priority?

-----------------------



    *See if the issue you are dealing with qualifies as regression, security

    issue, or a really severe problem: those are 'issues of high priority' that

    need special handling in some steps that are about to follow.*



Linus Torvalds and the leading Linux kernel developers want to see some issues

fixed as soon as possible, hence there are 'issues of high priority' that get

handled slightly differently in the reporting process. Three type of cases

qualify: regressions, security issues, and really severe problems.



You deal with a 'regression' if something that worked with an older version of

the Linux kernel does not work with a newer one or somehow works worse with it.

It thus is a regression when a WiFi driver that did a fine job with Linux 5.7

somehow misbehaves with 5.8 or doesn't work at all. It's also a regression if

an application shows erratic behavior with a newer kernel, which might happen

due to incompatible changes in the interface between the kernel and the

userland (like procfs and sysfs). Significantly reduced performance or

increased power consumption also qualify as regression. But keep in mind: the

new kernel needs to be built with a configuration that is similar to the one

from the old kernel (see below how to achieve that). That's because the kernel

developers sometimes can not avoid incompatibilities when implementing new

features; but to avoid regressions such features have to be enabled explicitly

during build time configuration.



What qualifies as security issue is left to your judgment. Consider reading

'Documentation/admin-guide/security-bugs.rst' before proceeding, as it

provides additional details how to best handle security issues.



An issue is a 'really severe problem' when something totally unacceptably bad

happens. That's for example the case when a Linux kernel corrupts the data it's

handling or damages hardware it's running on. You're also dealing with a severe

issue when the kernel suddenly stops working with an error message ('kernel

panic') or without any farewell note at all. Note: do not confuse a 'panic' (a

fatal error where the kernel stop itself) with a 'Oops' (a recoverable error),

as the kernel remains running after the latter.





Ensure a healthy environment

----------------------------



    *Make sure it's not the kernel's surroundings that are causing the issue

    you face.*



Problems that look a lot like a kernel issue are sometimes caused by build or

runtime environment. It's hard to rule out that problem completely, but you

should minimize it:



 * Use proven tools when building your kernel, as bugs in the compiler or the

   binutils can cause the resulting kernel to misbehave.



 * Ensure your computer components run within their design specifications;

   that's especially important for the main processor, the main memory, and the

   motherboard. Therefore, stop undervolting or overclocking when facing a

   potential kernel issue.



 * Try to make sure it's not faulty hardware that is causing your issue. Bad

   main memory for example can result in a multitude of issues that will

   manifest itself in problems looking like kernel issues.



 * If you're dealing with a filesystem issue, you might want to check the file

   system in question with ``fsck``, as it might be damaged in a way that leads

   to unexpected kernel behavior.



 * When dealing with a regression, make sure it's not something else that

   changed in parallel to updating the kernel. The problem for example might be

   caused by other software that was updated at the same time. It can also

   happen that a hardware component coincidentally just broke when you rebooted

   into a new kernel for the first time. Updating the systems BIOS or changing

   something in the BIOS Setup can also lead to problems that on look a lot

   like a kernel regression.





Prepare for emergencies

-----------------------



    *Create a fresh backup and put system repair and restore tools at hand.*



Reminder, you are dealing with computers, which sometimes do unexpected things,

especially if you fiddle with crucial parts like the kernel of its operating

system. That's what you are about to do in this process. Thus, make sure to

create a fresh backup; also ensure you have all tools at hand to repair or

reinstall the operating system as well as everything you need to restore the

backup.





Make sure your kernel doesn't get enhanced

------------------------------------------



    *Ensure your system does not enhance its kernels by building additional

    kernel modules on-the-fly, which solutions like DKMS might be doing locally

    without your knowledge.*



The risk your issue report gets ignored or rejected dramatically increases if

your kernel gets enhanced in any way. That's why you should remove or disable

mechanisms like akmods and DKMS: those build add-on kernel modules

automatically, for example when you install a new Linux kernel or boot it for

the first time. Also remove any modules they might have installed. Then reboot

before proceeding.



Note, you might not be aware that your system is using one of these solutions:

they often get set up silently when you install Nvidia's proprietary graphics

driver, VirtualBox, or other software that requires a some support from a

module not part of the Linux kernel. That why your might need to uninstall the

packages with such software to get rid of any 3rd party kernel module.





Check 'taint' flag

------------------



    *Check if your kernel was 'tainted' when the issue occurred, as the event

    that made the kernel set this flag might be causing the issue you face.*



The kernel marks itself with a 'taint' flag when something happens that might

lead to follow-up errors that look totally unrelated. The issue you face might

be such an error if your kernel is tainted. That's why it's in your interest to

rule this out early before investing more time into this process. This is the

only reason why this step is here, as this process later will tell you to

install the latest mainline kernel; you will need to check the taint flag again

then, as that's when it matters because it's the kernel the report will focus

on.



On a running system is easy to check if the kernel tainted itself: if ``cat

/proc/sys/kernel/tainted`` returns '0' then the kernel is not tainted and

everything is fine. Checking that file is impossible in some situations; that's

why the kernel also mentions the taint status when it reports an internal

problem (a 'kernel bug'), a recoverable error (a 'kernel Oops') or a

non-recoverable error before halting operation (a 'kernel panic'). Look near

the top of the error messages printed when one of these occurs and search for a

line starting with 'CPU:'. It should end with 'Not tainted' if the kernel was

not tainted when it noticed the problem; it was tainted if you see 'Tainted:'

followed by a few spaces and some letters.



If your kernel is tainted, study 'Documentation/admin-guide/tainted-kernels.rst'

to find out why. Try to eliminate the reason. Often it's caused by one these

three things:



 1. A recoverable error (a 'kernel Oops') occurred and the kernel tainted

    itself, as the kernel knows it might misbehave in strange ways after that

    point. In that case check your kernel or system log and look for a section

    that starts with this::



       Oops: 0000 [#1] SMP



    That's the first Oops since boot-up, as the '#1' between the brackets shows.

    Every Oops and any other problem that happens after that point might be a

    follow-up problem to that first Oops, even if both look totally unrelated.

    Rule this out by getting rid of the cause for the first Oops and reproducing

    the issue afterwards. Sometimes simply restarting will be enough, sometimes

    a change to the configuration followed by a reboot can eliminate the Oops.

    But don't invest too much time into this at this point of the process, as

    the cause for the Oops might already be fixed in the newer Linux kernel

    version you are going to install later in this process.



 2. Your system uses a software that installs its own kernel modules, for

    example Nvidia's proprietary graphics driver or VirtualBox. The kernel

    taints itself when it loads such module from external sources (even if

    they are Open Source): they sometimes cause errors in unrelated kernel

    areas and thus might be causing the issue you face. You therefore have to

    prevent those modules from loading when you want to report an issue to the

    Linux kernel developers. Most of the time the easiest way to do that is:

    temporarily uninstall such software including any modules they might have

    installed. Afterwards reboot.



 3. The kernel also taints itself when it's loading a module that resides in

    the staging tree of the Linux kernel source. That's a special area for

    code (mostly drivers) that does not yet fulfill the normal Linux kernel

    quality standards. When you report an issue with such a module it's

    obviously okay if the kernel is tainted; just make sure the module in

    question is the only reason for the taint. If the issue happens in an

    unrelated area reboot and temporarily block the module from being loaded

    by specifying ``foo.blacklist=1`` as kernel parameter (replace 'foo' with

    the name of the module in question).





Document how to reproduce issue

-------------------------------



    *Write down coarsely how to reproduce the issue. If you deal with multiple

    issues at once, create separate notes for each of them and make sure they

    work independently on a freshly booted system. That's needed, as each issue

    needs to get reported to the kernel developers separately, unless they are

    strongly entangled.*



If you deal with multiple issues at once, you'll have to report each of them

separately, as they might be handled by different developers. Describing

various issues in one report also makes it quite difficult for others to tear

it apart. Hence, only combine issues in one report if they are very strongly

entangled.



Additionally, during the reporting process you will have to test if the issue

happens with other kernel versions. Therefore, it will make your work easier if

you know exactly how to reproduce an issue quickly on a freshly booted system.



Note: it's often fruitless to report issues that only happened once, as they

might be caused by a bit flip due to cosmic radiation. That's why you should

try to rule that out by reproducing the issue before going further. Feel free

to ignore this advice if you are experienced enough to tell a one-time error

due to faulty hardware apart from a kernel issue that rarely happens and thus

is hard to reproduce.





Regression in stable or longterm kernel?

----------------------------------------



    *If you are facing a regression within a stable or longterm version line

    (say something broke when updating from 5.10.4 to 5.10.5), scroll down to

    'Dealing with regressions within a stable and longterm kernel line'.*



Regression within a stable and longterm kernel version line are something the

Linux developers want to fix badly, as such issues are even more unwanted than

regression in the main development branch, as they can quickly affect a lot of

people. The developers thus want to learn about such issues as quickly as

possible, hence there is a streamlined process to report them. Note,

regressions with newer kernel version line (say something broke when switching

from 5.9.15 to 5.10.5) do not qualify.





Check where you need to report your issue

-----------------------------------------



    *Locate the driver or kernel subsystem that seems to be causing the issue.

    Find out how and where its developers expect reports. Note: most of the

    time this won't be bugzilla.kernel.org, as issues typically need to be sent

    by mail to a maintainer and a public mailing list.*



It's crucial to send your report to the right people, as the Linux kernel is a

big project and most of its developers are only familiar with a small subset of

it. Quite a few programmers for example only care for just one driver, for

example one for a WiFi chip; its developer likely will only have small or no

knowledge about the internals of remote or unrelated "subsystems", like the TCP

stack, the PCIe/PCI subsystem, memory management or file systems.



Problem is: the Linux kernel lacks a central bug tracker where you can simply

file your issue and make it reach the developers that need to know about it.

That's why you have to find the right place and way to report issues yourself.

You can do that with the help of a script (see below), but it mainly targets

kernel developers and experts. For everybody else the MAINTAINERS file is the

better place.



How to read the MAINTAINERS file

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To illustrate how to use the :ref:`MAINTAINERS <maintainers>` file, lets assume

the WiFi in your Laptop suddenly misbehaves after updating the kernel. In that

case it's likely an issue in the WiFi driver. Obviously it could also be some

code it builds upon, but unless you suspect something like that stick to the

driver. If it's really something else, the driver's developers will get the

right people involved.



Sadly, there is no way to check which code is driving a particular hardware

component that is both universal and easy.



In case of a problem with the WiFi driver you for example might want to look at

the output of ``lspci -k``, as it lists devices on the PCI/PCIe bus and the

kernel module driving it::



       [user@something ~]$ lspci -k

       [...]

       3a:00.0 Network controller: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter (rev 32)

         Subsystem: Bigfoot Networks, Inc. Device 1535

         Kernel driver in use: ath10k_pci

         Kernel modules: ath10k_pci

       [...]



But this approach won't work if your WiFi chip is connected over USB or some

other internal bus. In those cases you might want to check your WiFi manager or

the output of ``ip link``. Look for the name of the problematic network

interface, which might be something like 'wlp58s0'. This name can be used like

this to find the module driving it::



       [user@something ~]$ realpath --relative-to=/sys/module/ /sys/class/net/wlp58s0/device/driver/module

       ath10k_pci



In case tricks like these don't bring you any further, try to search the

internet on how to narrow down the driver or subsystem in question. And if you

are unsure which it is: just try your best guess, somebody will help you if you

guessed poorly.



Once you know the driver or subsystem, you want to search for it in the

MAINTAINERS file. In the case of 'ath10k_pci' you won't find anything, as the

name is too specific. Sometimes you will need to search on the net for help;

but before doing so, try a somewhat shorted or modified name when searching the

MAINTAINERS file, as then you might find something like this::



       QUALCOMM ATHEROS ATH10K WIRELESS DRIVER

       Mail:          A. Some Human <shuman@example.com>

       Mailing list:  ath10k@lists.infradead.org

       Status:        Supported

       Web-page:      https://wireless.wiki.kernel.org/en/users/Drivers/ath10k

       SCM:           git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git

       Files:         drivers/net/wireless/ath/ath10k/



Note: the line description will be abbreviations, if you read the plain

MAINTAINERS file found in the root of the Linux source tree. 'Mail:' for

example will be 'M:', 'Mailing list:' will be 'L', and 'Status:' will be 'S:'.

A section near the top of the file explains these and other abbreviations.



First look at the line 'Status'. Ideally it should be 'Supported' or

'Maintained'. If it states 'Obsolete' then you are using some outdated approach

that was replaced by a newer solution you need to switch to. Sometimes the code

only has someone who provides 'Odd Fixes' when feeling motivated. And with

'Orphan' you are totally out of luck, as nobody takes care of the code anymore.

That only leaves these options: arrange yourself to live with the issue, fix it

yourself, or find a programmer somewhere willing to fix it.



After checking the status, look for a line starting with 'bugs:': it will tell

you where to find a subsystem specific bug tracker to file your issue. The

example above does not have such a line. That is the case for most sections, as

Linux kernel development is completely driven by mail. Very few subsystems use

a bug tracker, and only some of those rely on bugzilla.kernel.org.



In this and many other cases you thus have to look for lines starting with

'Mail:' instead. Those mention the name and the email addresses for the

maintainers of the particular code. Also look for a line starting with 'Mailing

list:', which tells you the public mailing list where the code is developed.

Your report later needs to go by mail to those addresses. Additionally, for all

issue reports sent by email, make sure to add the Linux Kernel Mailing List

(LKML) <linux-kernel@vger.kernel.org> to CC. Don't omit either of the mailing

lists when sending your issue report by mail later! Maintainers are busy people

and might leave some work for other developers on the subsystem specific list;

and LKML is important to have one place where all issue reports can be found.





Finding the maintainers with the help of a script

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



For people that have the Linux sources at hand there is a second option to find

the proper place to report: the script 'scripts/get_maintainer.pl' which tries

to find all people to contact. It queries the MAINTAINERS file and needs to be

called with a path to the source code in question. For drivers compiled as

module if often can be found with a command like this::



       $ modinfo ath10k_pci | grep filename | sed 's!/lib/modules/.*/kernel/!!; s!filename:!!; s!\.ko\(\|\.xz\)!!'

       drivers/net/wireless/ath/ath10k/ath10k_pci.ko



Pass parts of this to the script::



       $ ./scripts/get_maintainer.pl -f drivers/net/wireless/ath/ath10k*

       Some Human <shuman@example.com> (supporter:QUALCOMM ATHEROS ATH10K WIRELESS DRIVER)

       Another S. Human <asomehuman@example.com> (maintainer:NETWORKING DRIVERS)

       ath10k@lists.infradead.org (open list:QUALCOMM ATHEROS ATH10K WIRELESS DRIVER)

       linux-wireless@vger.kernel.org (open list:NETWORKING DRIVERS (WIRELESS))

       netdev@vger.kernel.org (open list:NETWORKING DRIVERS)

       linux-kernel@vger.kernel.org (open list)



Don't sent your report to all of them. Send it to the maintainers, which the

script calls "supporter:"; additionally CC the most specific mailing list for

the code as well as the Linux Kernel Mailing List (LKML). In this case you thus

would need to send the report to 'Some Human <shuman@example.com>' with

'ath10k@lists.infradead.org' and 'linux-kernel@vger.kernel.org' in CC.



Note: in case you cloned the Linux sources with git you might want to call

``get_maintainer.pl`` a second time with ``--git``. The script then will look

at the commit history to find which people recently worked on the code in

question, as they might be able to help. But use these results with care, as it

can easily send you in a wrong direction. That for example happens quickly in

areas rarely changed (like old or unmaintained drivers): sometimes such code is

modified during tree-wide cleanups by developers that do not care about the

particular driver at all.





Search for existing reports, second run

---------------------------------------



    *Search the archives of the bug tracker or mailing list in question

    thoroughly for reports that might match your issue. If you find anything,

    join the discussion instead of sending a new report.*



As mentioned earlier already: reporting an issue that someone else already

brought forward is often a waste of time for everyone involved, especially you

as the reporter. That's why you should search for existing report again, now

that you know where they need to be reported to. If it's mailing list, you will

often find its archives on `lore.kernel.org <https://lore.kernel.org/>`_.



But some list are hosted in different places. That for example is the case for

the ath10k WiFi driver used as example in the previous step. But you'll often

find the archives for these lists easily on the net. Searching for 'archive

ath10k@lists.infradead.org' for example will lead you to the `Info page for the

ath10k mailing list <https://lists.infradead.org/mailman/listinfo/ath10k>`_,

which at the top links to its

`list archives <https://lists.infradead.org/pipermail/ath10k/>`_. Sadly this and

quite a few other lists miss a way to search the archives. In those cases use a

regular internet search engine and add something like

'site:lists.infradead.org/pipermail/ath10k/' to your search terms, which limits

the results to the archives at that URL.



It's also wise to check the internet, LKML and maybe bugzilla.kernel.org again

at this point.



For details how to search and what to do if you find matching reports see

"Search for existing reports, first run" above.



Do not hurry with this step of the reporting process: spending 30 to 60 minutes

or even more time can save you and others quite a lot of time and trouble.





Install a fresh kernel for testing

----------------------------------



    *Unless you are already running the latest 'mainline' Linux kernel, better

    go and install it for the reporting process. Testing and reporting with

    the latest 'stable' Linux can be an acceptable alternative in some

    situations; during the merge window that actually might be even the best

    approach, but in that development phase it can be an even better idea to

    suspend your efforts for a few days anyway. Whatever version you choose,

    ideally use a 'vanilla' built. Ignoring these advices will dramatically

    increase the risk your report will be rejected or ignored.*



As mentioned in the detailed explanation for the first step already: Like most

programmers, Linux kernel developers don't like to spend time dealing with

reports for issues that don't even happen with the current code. It's just a

waste everybody's time, especially yours. That's why it's in everybody's

interest that you confirm the issue still exists with the latest upstream code

before reporting it. You are free to ignore this advice, but as outlined

earlier: doing so dramatically increases the risk that your issue report might

get rejected or simply ignored.



In the scope of the kernel "latest upstream" normally means:



 * Install a mainline kernel; the latest stable kernel can be an option, but

   most of the time is better avoided. Longterm kernels (sometimes called 'LTS

   kernels') are unsuitable at this point of the process. The next subsection

   explains all of this in more detail.



 * The over next subsection describes way to obtain and install such a kernel.

   It also outlines that using a pre-compiled kernel are fine, but better are

   vanilla, which means: it was built using Linux sources taken straight `from

   kernel.org <https://kernel.org/>`_ and not modified or enhanced in any way.



Choosing the right version for testing

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



Head over to `kernel.org <https://kernel.org/>`_ to find out which version you

want to use for testing. Ignore the big yellow button that says 'Latest release'

and look a little lower at the table. At its top you'll see a line starting with

mainline, which most of the time will point to a pre-release with a version

number like '5.8-rc2'. If that's the case, you'll want to use this mainline

kernel for testing, as that where all fixes have to be applied first. Do not let

that 'rc' scare you, these 'development kernels' are pretty reliable — and you

made a backup, as you were instructed above, didn't you?



In about two out of every nine to ten weeks, 'mainline' might point you to a

proper release with a version number like '5.7'. If that happens, consider

suspending the reporting process until the first pre-release of the next

version (5.8-rc1) shows up on kernel.org. That's because the Linux development

cycle then is in its two-week long 'merge window'. The bulk of the changes and

all intrusive ones get merged for the next release during this time. It's a bit

more risky to use mainline during this period. Kernel developers are also often

quite busy then and might have no spare time to deal with issue reports. It's

also quite possible that one of the many changes applied during the merge

window fixes the issue you face; that's why you soon would have to retest with

a newer kernel version anyway, as outlined below in the section 'Duties after

the report went out'.



That's why it might make sense to wait till the merge window is over. But don't

to that if you're dealing with something that shouldn't wait. In that case

consider obtaining the latest mainline kernel via git (see below) or use the

latest stable version offered on kernel.org. Using that is also acceptable in

case mainline for some reason does currently not work for you. An in general:

using it for reproducing the issue is also better than not reporting it issue

at all.



Better avoid using the latest stable kernel outside merge windows, as all fixes

must be applied to mainline first. That's why checking the latest mainline

kernel is so important: any issue you want to see fixed in older version lines

needs to be fixed in mainline first before it can get backported, which can

take a few days or weeks. Another reason: the fix you hope for might be too

hard or risky for backporting; reporting the issue again hence is unlikely to

change anything.



These aspects are also why longterm kernels (sometimes called "LTS kernels")

are unsuitable for this part of the reporting process: they are to distant from

the current code. Hence go and test mainline first and follow the process

further: if the issue doesn't occur with mainline it will guide you how to get

it fixed in older version lines, if that's in the cards for the fix in question.W
_______________________________________________
Ksummit-discuss mailing list
Ksummit-discuss@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss