All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tsuchiya Yuto <kitakar@gmail.com>
To: unlisted-recipients:; (no To-header on input)
Cc: Hans de Goede <hdegoede@redhat.com>,
	Patrik Gfeller <patrik.gfeller@gmail.com>,
	Tsuchiya Yuto <kitakar@gmail.com>,
	Mauro Carvalho Chehab <mchehab@kernel.org>,
	Sakari Ailus <sakari.ailus@linux.intel.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Kaixu Xia <kaixuxia@tencent.com>, Arnd Bergmann <arnd@arndb.de>,
	Dan Carpenter <dan.carpenter@oracle.com>,
	linux-media@vger.kernel.org, linux-staging@lists.linux.dev,
	linux-kernel@vger.kernel.org
Subject: [BUG 4/5] [BUG] media: atomisp: `modprobe -r` not working well (dup video4linux, ATOMISP_SUBDEV_{0,1})
Date: Mon, 18 Oct 2021 01:23:35 +0900	[thread overview]
Message-ID: <20211017162337.44860-5-kitakar@gmail.com> (raw)
In-Reply-To: <20211017162337.44860-1-kitakar@gmail.com>

This is just a BUG report and I don't have any patch for this issue.
For now, I use rmmod/insmod to reload atomisp modules.

When I use `modprobe -r` to unload atomisp drivers like the following,

        $ sudo modprobe -r atomisp
        $ sudo modprobe -r atomisp-ar0330
        $ sudo modprobe -r atomisp-ov883x

it will fail to initialize on next load (with both modprobe and insmod).

Steps to reproduce:

  1. boot without denylisting any atomisp modules (or load modules using
     modprobe manually)
  2. unload drivers with `modprobe -r`
  3. try to load drivers (modprobe or insmod)
  4. probe fails saying "duplicate filename [...] video4linux"
     (log available below)

Note:

  1. `modprobe -r` works when booting with denylisting then insmod-ing
  2. rmmod always works without errors

Here is the dmesg log:

        kern  :warn  : [  150.326240] sysfs: cannot create duplicate filename '/devices/pci0000:00/0000:00:03.0/video4linux'
        kern  :warn  : [  150.326246] CPU: 1 PID: 2739 Comm: modprobe Tainted: G         C OE     5.15.0-rc3-1-surface-mainline #1 22942c127770b6216143942411a0f5916a743a60
        kern  :warn  : [  150.326257] Hardware name: OEMB OEMB/OEMB, BIOS 1.51116.238 03/09/2015
        kern  :warn  : [  150.326262] Call Trace:
        kern  :warn  : [  150.326276]  dump_stack_lvl+0x46/0x5a
        kern  :warn  : [  150.326292]  sysfs_warn_dup.cold+0x17/0x24
        kern  :warn  : [  150.326304]  sysfs_create_dir_ns+0xcc/0xe0
        kern  :warn  : [  150.326318]  kobject_add_internal+0xbd/0x2b0
        kern  :warn  : [  150.326330]  kobject_add+0x98/0xd0
        kern  :warn  : [  150.326338]  ? kmem_cache_alloc_trace+0x15c/0x2f0
        kern  :warn  : [  150.326352]  get_device_parent+0x15e/0x1c0
        kern  :warn  : [  150.326365]  device_add+0xd9/0x9a0
        kern  :warn  : [  150.326375]  __video_register_device+0x8ee/0x11f0 [videodev 2ea57a870000384ef8d27d4d70e6e51a68e3f718]
        kern  :warn  : [  150.326420]  atomisp_subdev_register_entities+0x83/0x220 [atomisp 752573cebc128314ca99d40a85c7801ade7015b0]
        kern  :warn  : [  150.326528]  atomisp_pci_probe.cold+0xec3/0x1492 [atomisp 752573cebc128314ca99d40a85c7801ade7015b0]
        kern  :warn  : [  150.326625]  local_pci_probe+0x45/0x80
        kern  :warn  : [  150.326634]  ? pci_match_device+0xd7/0x130
        kern  :warn  : [  150.326640]  pci_device_probe+0xfa/0x1b0
        kern  :warn  : [  150.326650]  really_probe+0x1f5/0x3f0
        kern  :warn  : [  150.326661]  __driver_probe_device+0xfe/0x180
        kern  :warn  : [  150.326667]  driver_probe_device+0x1e/0x90
        kern  :warn  : [  150.326674]  __driver_attach+0xc0/0x1c0
        kern  :warn  : [  150.326680]  ? __device_attach_driver+0xe0/0xe0
        kern  :warn  : [  150.326685]  ? __device_attach_driver+0xe0/0xe0
        kern  :warn  : [  150.326691]  bus_for_each_dev+0x89/0xd0
        kern  :warn  : [  150.326698]  bus_add_driver+0x12b/0x1e0
        kern  :warn  : [  150.326704]  driver_register+0x8f/0xe0
        kern  :warn  : [  150.326712]  ? 0xffffffffc09e6000
        kern  :warn  : [  150.326717]  do_one_initcall+0x57/0x220
        kern  :warn  : [  150.326733]  do_init_module+0x5c/0x260
        kern  :warn  : [  150.326745]  load_module+0x24bd/0x26a0
        kern  :warn  : [  150.326760]  ? __do_sys_finit_module+0xae/0x110
        kern  :warn  : [  150.326766]  __do_sys_finit_module+0xae/0x110
        kern  :warn  : [  150.326777]  do_syscall_64+0x5c/0x80
        kern  :warn  : [  150.326789]  ? __audit_syscall_exit+0x24d/0x2a0
        kern  :warn  : [  150.326796]  ? syscall_exit_to_user_mode+0x23/0x40
        kern  :warn  : [  150.326804]  ? do_syscall_64+0x69/0x80
        kern  :warn  : [  150.326810]  ? do_syscall_64+0x69/0x80
        kern  :warn  : [  150.326816]  entry_SYSCALL_64_after_hwframe+0x44/0xae
        kern  :warn  : [  150.326824] RIP: 0033:0x7f33b9de918d
        kern  :warn  : [  150.326832] Code: b4 0c 00 0f 05 eb a9 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d b3 6c 0c 00 f7 d8 64 89 01 48
        kern  :warn  : [  150.326836] RSP: 002b:00007fffce4b44a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
        kern  :warn  : [  150.326846] RAX: ffffffffffffffda RBX: 00005643ae549e10 RCX: 00007f33b9de918d
        kern  :warn  : [  150.326851] RDX: 0000000000000000 RSI: 00005643acc65270 RDI: 0000000000000005
        kern  :warn  : [  150.326854] RBP: 0000000000040000 R08: 0000000000000000 R09: 00005643ae549f20
        kern  :warn  : [  150.326858] R10: 0000000000000005 R11: 0000000000000246 R12: 00005643acc65270
        kern  :warn  : [  150.326861] R13: 0000000000000000 R14: 00005643ae549cc0 R15: 00005643ae549e10
        kern  :err   : [  150.326871] kobject_add_internal failed for video4linux with -EEXIST, don't try to register things with the same name in the same directory.
        kern  :err   : [  150.326878] videodev: __video_register_device: device_register failed
        kern  :err   : [  150.326903] atomisp-isp2 0000:00:03.0: atomisp_subdev_register_entities fail
        kern  :err   : [  150.326918] atomisp-isp2 0000:00:03.0: atomisp_register_entities failed (-17)
        kern  :warn  : [  150.328150] atomisp-isp2: probe of 0000:00:03.0 failed with error -17

Later, I found that after unloading atomisp module (by both
rmmod and `modprobe -r`), ATOMISP_SUBDEV_0 and ATOMISP_SUBDEV_1 were left
under /sys/class/video4linux directory:

        # list v4l device names
        $ grep . /sys/class/video4linux/*/name
        /sys/class/video4linux/v4l-subdev7/name:ATOMISP_SUBDEV_0
        /sys/class/video4linux/v4l-subdev8/name:ATOMISP_SUBDEV_1

These files will be added every time I reload atomisp module using
rmmod/insmod:

        /sys/class/video4linux/v4l-subdev7/name:ATOMISP_SUBDEV_0
        /sys/class/video4linux/v4l-subdev8/name:ATOMISP_SUBDEV_1
        /sys/class/video4linux/v4l-subdev9/name:ATOMISP_SUBDEV_0
        /sys/class/video4linux/v4l-subdev10/name:ATOMISP_SUBDEV_1

Note 1:
  Somehow I can't reproduce this issue on Android kernel (using
  Mi Pad 2) and my port of intel-aero atomisp [1].

[1] https://github.com/kitakar5525/linux-kernel/tree/mainline+aero_atomisp_wo_kapi_changes-2021-10-11

Note 2:
  Here is the complete reload script (using rmmod/insmod) I use on
  Surface 3:

	# reload atomisp camera modules
	# somehow `modprobe -r` does not work well
	sudo rmmod atomisp
	sudo rmmod atomisp-ar0330
	sudo rmmod atomisp-ov883x
	sudo rmmod atomisp_gmin_platform

	# load drivers needed for atomisp first for insmod
	# for sensor drivers
	sudo modprobe media # needed for older LTS
	sudo modprobe videodev
	sudo modprobe v4l2_common # needed for older LTS
	sudo modprobe v4l2_async # if using async_register
	# for atomisp pci driver
	sudo modprobe videobuf-core
	sudo modprobe videobuf-vmalloc

	# insmod upstreamed atomisp
	sudo insmod upst/atomisp_gmin_platform.ko
	sudo insmod upst/surface3/atomisp-ar0330.ko
	sudo insmod upst/surface3/atomisp-ov883x.ko
	sudo insmod upst/atomisp.ko dbg_level=1 #dyndbg
-- 
2.33.1


WARNING: multiple messages have this Message-ID (diff)
From: Tsuchiya Yuto <kitakar@gmail.com>
Cc: Hans de Goede <hdegoede@redhat.com>,
	Patrik Gfeller <patrik.gfeller@gmail.com>,
	Tsuchiya Yuto <kitakar@gmail.com>,
	Mauro Carvalho Chehab <mchehab@kernel.org>,
	Sakari Ailus <sakari.ailus@linux.intel.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Andy Shevchenko <andriy.shevchenko@linux.intel.com>,
	Kaixu Xia <kaixuxia@tencent.com>, Arnd Bergmann <arnd@arndb.de>,
	Dan Carpenter <dan.carpenter@oracle.com>,
	linux-media@vger.kernel.org, linux-staging@lists.linux.dev,
	linux-kernel@vger.kernel.org
Subject: [BUG 4/5] [BUG] media: atomisp: `modprobe -r` not working well (dup video4linux, ATOMISP_SUBDEV_{0,1})
Date: Mon, 18 Oct 2021 01:23:35 +0900	[thread overview]
Message-ID: <20211017162337.44860-5-kitakar@gmail.com> (raw)
In-Reply-To: <20211017162337.44860-1-kitakar@gmail.com>

This is just a BUG report and I don't have any patch for this issue.
For now, I use rmmod/insmod to reload atomisp modules.

When I use `modprobe -r` to unload atomisp drivers like the following,

        $ sudo modprobe -r atomisp
        $ sudo modprobe -r atomisp-ar0330
        $ sudo modprobe -r atomisp-ov883x

it will fail to initialize on next load (with both modprobe and insmod).

Steps to reproduce:

  1. boot without denylisting any atomisp modules (or load modules using
     modprobe manually)
  2. unload drivers with `modprobe -r`
  3. try to load drivers (modprobe or insmod)
  4. probe fails saying "duplicate filename [...] video4linux"
     (log available below)

Note:

  1. `modprobe -r` works when booting with denylisting then insmod-ing
  2. rmmod always works without errors

Here is the dmesg log:

        kern  :warn  : [  150.326240] sysfs: cannot create duplicate filename '/devices/pci0000:00/0000:00:03.0/video4linux'
        kern  :warn  : [  150.326246] CPU: 1 PID: 2739 Comm: modprobe Tainted: G         C OE     5.15.0-rc3-1-surface-mainline #1 22942c127770b6216143942411a0f5916a743a60
        kern  :warn  : [  150.326257] Hardware name: OEMB OEMB/OEMB, BIOS 1.51116.238 03/09/2015
        kern  :warn  : [  150.326262] Call Trace:
        kern  :warn  : [  150.326276]  dump_stack_lvl+0x46/0x5a
        kern  :warn  : [  150.326292]  sysfs_warn_dup.cold+0x17/0x24
        kern  :warn  : [  150.326304]  sysfs_create_dir_ns+0xcc/0xe0
        kern  :warn  : [  150.326318]  kobject_add_internal+0xbd/0x2b0
        kern  :warn  : [  150.326330]  kobject_add+0x98/0xd0
        kern  :warn  : [  150.326338]  ? kmem_cache_alloc_trace+0x15c/0x2f0
        kern  :warn  : [  150.326352]  get_device_parent+0x15e/0x1c0
        kern  :warn  : [  150.326365]  device_add+0xd9/0x9a0
        kern  :warn  : [  150.326375]  __video_register_device+0x8ee/0x11f0 [videodev 2ea57a870000384ef8d27d4d70e6e51a68e3f718]
        kern  :warn  : [  150.326420]  atomisp_subdev_register_entities+0x83/0x220 [atomisp 752573cebc128314ca99d40a85c7801ade7015b0]
        kern  :warn  : [  150.326528]  atomisp_pci_probe.cold+0xec3/0x1492 [atomisp 752573cebc128314ca99d40a85c7801ade7015b0]
        kern  :warn  : [  150.326625]  local_pci_probe+0x45/0x80
        kern  :warn  : [  150.326634]  ? pci_match_device+0xd7/0x130
        kern  :warn  : [  150.326640]  pci_device_probe+0xfa/0x1b0
        kern  :warn  : [  150.326650]  really_probe+0x1f5/0x3f0
        kern  :warn  : [  150.326661]  __driver_probe_device+0xfe/0x180
        kern  :warn  : [  150.326667]  driver_probe_device+0x1e/0x90
        kern  :warn  : [  150.326674]  __driver_attach+0xc0/0x1c0
        kern  :warn  : [  150.326680]  ? __device_attach_driver+0xe0/0xe0
        kern  :warn  : [  150.326685]  ? __device_attach_driver+0xe0/0xe0
        kern  :warn  : [  150.326691]  bus_for_each_dev+0x89/0xd0
        kern  :warn  : [  150.326698]  bus_add_driver+0x12b/0x1e0
        kern  :warn  : [  150.326704]  driver_register+0x8f/0xe0
        kern  :warn  : [  150.326712]  ? 0xffffffffc09e6000
        kern  :warn  : [  150.326717]  do_one_initcall+0x57/0x220
        kern  :warn  : [  150.326733]  do_init_module+0x5c/0x260
        kern  :warn  : [  150.326745]  load_module+0x24bd/0x26a0
        kern  :warn  : [  150.326760]  ? __do_sys_finit_module+0xae/0x110
        kern  :warn  : [  150.326766]  __do_sys_finit_module+0xae/0x110
        kern  :warn  : [  150.326777]  do_syscall_64+0x5c/0x80
        kern  :warn  : [  150.326789]  ? __audit_syscall_exit+0x24d/0x2a0
        kern  :warn  : [  150.326796]  ? syscall_exit_to_user_mode+0x23/0x40
        kern  :warn  : [  150.326804]  ? do_syscall_64+0x69/0x80
        kern  :warn  : [  150.326810]  ? do_syscall_64+0x69/0x80
        kern  :warn  : [  150.326816]  entry_SYSCALL_64_after_hwframe+0x44/0xae
        kern  :warn  : [  150.326824] RIP: 0033:0x7f33b9de918d
        kern  :warn  : [  150.326832] Code: b4 0c 00 0f 05 eb a9 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d b3 6c 0c 00 f7 d8 64 89 01 48
        kern  :warn  : [  150.326836] RSP: 002b:00007fffce4b44a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
        kern  :warn  : [  150.326846] RAX: ffffffffffffffda RBX: 00005643ae549e10 RCX: 00007f33b9de918d
        kern  :warn  : [  150.326851] RDX: 0000000000000000 RSI: 00005643acc65270 RDI: 0000000000000005
        kern  :warn  : [  150.326854] RBP: 0000000000040000 R08: 0000000000000000 R09: 00005643ae549f20
        kern  :warn  : [  150.326858] R10: 0000000000000005 R11: 0000000000000246 R12: 00005643acc65270
        kern  :warn  : [  150.326861] R13: 0000000000000000 R14: 00005643ae549cc0 R15: 00005643ae549e10
        kern  :err   : [  150.326871] kobject_add_internal failed for video4linux with -EEXIST, don't try to register things with the same name in the same directory.
        kern  :err   : [  150.326878] videodev: __video_register_device: device_register failed
        kern  :err   : [  150.326903] atomisp-isp2 0000:00:03.0: atomisp_subdev_register_entities fail
        kern  :err   : [  150.326918] atomisp-isp2 0000:00:03.0: atomisp_register_entities failed (-17)
        kern  :warn  : [  150.328150] atomisp-isp2: probe of 0000:00:03.0 failed with error -17

Later, I found that after unloading atomisp module (by both
rmmod and `modprobe -r`), ATOMISP_SUBDEV_0 and ATOMISP_SUBDEV_1 were left
under /sys/class/video4linux directory:

        # list v4l device names
        $ grep . /sys/class/video4linux/*/name
        /sys/class/video4linux/v4l-subdev7/name:ATOMISP_SUBDEV_0
        /sys/class/video4linux/v4l-subdev8/name:ATOMISP_SUBDEV_1

These files will be added every time I reload atomisp module using
rmmod/insmod:

        /sys/class/video4linux/v4l-subdev7/name:ATOMISP_SUBDEV_0
        /sys/class/video4linux/v4l-subdev8/name:ATOMISP_SUBDEV_1
        /sys/class/video4linux/v4l-subdev9/name:ATOMISP_SUBDEV_0
        /sys/class/video4linux/v4l-subdev10/name:ATOMISP_SUBDEV_1

Note 1:
  Somehow I can't reproduce this issue on Android kernel (using
  Mi Pad 2) and my port of intel-aero atomisp [1].

[1] https://github.com/kitakar5525/linux-kernel/tree/mainline+aero_atomisp_wo_kapi_changes-2021-10-11

Note 2:
  Here is the complete reload script (using rmmod/insmod) I use on
  Surface 3:

	# reload atomisp camera modules
	# somehow `modprobe -r` does not work well
	sudo rmmod atomisp
	sudo rmmod atomisp-ar0330
	sudo rmmod atomisp-ov883x
	sudo rmmod atomisp_gmin_platform

	# load drivers needed for atomisp first for insmod
	# for sensor drivers
	sudo modprobe media # needed for older LTS
	sudo modprobe videodev
	sudo modprobe v4l2_common # needed for older LTS
	sudo modprobe v4l2_async # if using async_register
	# for atomisp pci driver
	sudo modprobe videobuf-core
	sudo modprobe videobuf-vmalloc

	# insmod upstreamed atomisp
	sudo insmod upst/atomisp_gmin_platform.ko
	sudo insmod upst/surface3/atomisp-ar0330.ko
	sudo insmod upst/surface3/atomisp-ov883x.ko
	sudo insmod upst/atomisp.ko dbg_level=1 #dyndbg
-- 
2.33.1


  parent reply	other threads:[~2021-10-17 16:24 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-17 16:23 [BUG 0/5] bug reports for atomisp to make it work Tsuchiya Yuto
2021-10-17 16:23 ` Tsuchiya Yuto
2021-10-17 16:23 ` [BUG/RFC PATCH 1/5] [BUG][RFC] media: atomisp: pci: assume run_mode is PREVIEW Tsuchiya Yuto
2021-10-17 16:23   ` Tsuchiya Yuto
2021-10-29  9:02   ` Mauro Carvalho Chehab
2021-11-11  8:00     ` Tsuchiya Yuto
2021-10-17 16:23 ` [BUG/RFC PATCH 2/5] [BUG][RFC] media: atomisp: pci: remove dummy_ptr NULL check to avoid duplicate active_bo Tsuchiya Yuto
2021-10-17 16:23   ` Tsuchiya Yuto
2022-02-08  6:19   ` Mauro Carvalho Chehab
2021-10-17 16:23 ` [BUG/RFC PATCH 3/5] [BUG][RFC] media: atomisp: pci: add NULL check for asd obtained from atomisp_video_pipe Tsuchiya Yuto
2021-10-17 16:23   ` Tsuchiya Yuto
2021-11-02 13:02   ` Dan Carpenter
2021-11-02 14:44     ` Andy Shevchenko
2021-11-02 14:45       ` Andy Shevchenko
2021-11-02 15:05         ` Dan Carpenter
2021-11-02 15:49           ` Andy Shevchenko
2021-11-02 22:52             ` Mauro Carvalho Chehab
2021-11-08 15:48     ` Tsuchiya Yuto
2021-10-17 16:23 ` Tsuchiya Yuto [this message]
2021-10-17 16:23   ` [BUG 4/5] [BUG] media: atomisp: `modprobe -r` not working well (dup video4linux, ATOMISP_SUBDEV_{0,1}) Tsuchiya Yuto
2021-10-17 16:23 ` [BUG 5/5] [BUG] media: atomisp: atomisp causes touchscreen to stop working on Microsoft Surface 3 Tsuchiya Yuto
2021-10-17 16:23   ` Tsuchiya Yuto
2021-10-18  8:30   ` Hans de Goede
2021-10-21  9:52     ` Tsuchiya Yuto
2021-10-24  8:32       ` Hans de Goede
2021-10-26  9:35         ` Tsuchiya Yuto
2021-10-26 15:41           ` Hans de Goede
2021-10-27 16:07             ` Tsuchiya Yuto
2021-11-07 23:39       ` Hans de Goede
2021-11-08  7:41         ` Mauro Carvalho Chehab
2021-11-08  7:55           ` Hans de Goede
2021-11-08  8:01             ` Mauro Carvalho Chehab
2021-11-09  4:18             ` Tsuchiya Yuto
2021-11-09  9:08               ` Mauro Carvalho Chehab
2021-11-09  4:15         ` Tsuchiya Yuto
2021-11-09  7:35           ` Mauro Carvalho Chehab
2021-10-18  7:50 ` [BUG 0/5] bug reports for atomisp to make it work Hans de Goede
2021-10-18  8:10   ` Andy Shevchenko
2021-10-19 12:58     ` Tsuchiya Yuto
2021-10-19 14:06       ` Andy Shevchenko
2021-10-20  6:48 ` Mauro Carvalho Chehab
2021-10-20 12:35   ` Tsuchiya Yuto

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211017162337.44860-5-kitakar@gmail.com \
    --to=kitakar@gmail.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=arnd@arndb.de \
    --cc=dan.carpenter@oracle.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hdegoede@redhat.com \
    --cc=kaixuxia@tencent.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-staging@lists.linux.dev \
    --cc=mchehab@kernel.org \
    --cc=patrik.gfeller@gmail.com \
    --cc=peterz@infradead.org \
    --cc=sakari.ailus@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.