All of lore.kernel.org
 help / color / mirror / Atom feed
From: ebiederm@xmission.com (Eric W. Biederman)
To: Jordan Glover <Golden_Miller83@protonmail.ch>
Cc: LKML <linux-kernel@vger.kernel.org>,
	"linux-mm\@kvack.org" <linux-mm@kvack.org>,
	"legion\@kernel.org" <legion@kernel.org>,
	"containers\@lists.linux-foundation.org" 
	<containers@lists.linux-foundation.org>
Subject: Re: linux 5.14.3: free_user_ns causes NULL pointer dereference
Date: Wed, 15 Sep 2021 16:02:19 -0500	[thread overview]
Message-ID: <87ee9pa6xw.fsf@disp2133> (raw)
In-Reply-To: <1M9_d6wrcu6rdPe1ON0_k0lOxJMyyot3KAb1gdyuwzDPC777XVUWPHoTCEVmcK3fYfgu7sIo3PSaLe9KulUdm4TWVuqlbKyYGxRAjsf_Cpk=@protonmail.ch> (Jordan Glover's message of "Wed, 15 Sep 2021 19:49:31 +0000")

Jordan Glover <Golden_Miller83@protonmail.ch> writes:

> Hi, recently I hit system freeze after I was closing few containerized apps on my system. As for now it occurred only once on linux 5.14.3. I think it maybe be related to "Count rlimits in each user namespace" patchset merged during 5.14 window
>
> https://lore.kernel.org/all/257aa5fb1a7d81cf0f4c34f39ada2320c4284771.1619094428.git.legion@kernel.org/T/#u

So that warning comes from:

void dec_ucount(struct ucounts *ucounts, enum ucount_type type)
{
	struct ucounts *iter;
	for (iter = ucounts; iter; iter = iter->ns->ucounts) {
		long dec = atomic_long_dec_if_positive(&iter->ucount[type]);
		WARN_ON_ONCE(dec < 0);
	}
	put_ucounts(ucounts);
}

Which certainly looks like a reference count bug.  It could also be a
memory stomp somewhere close.

Do you have any idea what else was going on?  This location is the
symptom but not the actual cause.

Eric



>
> Logs below:
>
> ------------[ cut here ]------------
> WARNING: CPU: 1 PID: 26546 at kernel/ucount.c:253 dec_ucount+0x43/0x50
> Modules linked in: nft_ct nft_fib_ipv4 nft_fib wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic libchacha ccm algif_aead des_generic libdes ecb algif_skcipher cmac md4 algif_hash af_alg hid_sensor_custom_intel_hinge hid_sensor_als hid_sensor_magn_3d hid_sensor_rotation hid_sensor_accel_3d hid_sensor_gyro_3d hid_sensor_trigger industrialio_triggered_buffer hid_sensor_iio_common kfifo_buf industrialio hid_sensor_custom hid_sensor_hub cros_ec_ishtp cros_ec intel_ishtp_loader nft_counter intel_ishtp_hid snd_hda_codec_hdmi intel_rapl_msr xt_mark ipt_REJECT nf_reject_ipv4 snd_ctl_led xt_LOG snd_hda_codec_conexant nf_log_syslog snd_hda_codec_generic xt_addrtype xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv4 mei_hdcp snd_hda_intel nft_compat wmi_bmof nf_tables intel_rapl_common libcrc32c think_lmi intel_tcc_cooling snd_intel_dspcfg firmware_attributes_class nfnetlink iwlmvm
>  intel_wmi_thunderbolt mac80211 x86_pkg_temp_thermal snd_hda_codec intel_powerclamp coretemp libarc4 vfat fat kvm_intel rapl intel_cstate snd_hwdep intel_uncore iwlwifi snd_hda_core mousedev joydev snd_pcm psmouse cfg80211 snd_timer mei_me ucsi_acpi wacom intel_ish_ipc intel_xhci_usb_role_switch mei intel_pch_thermal typec_ucsi roles typec intel_ishtp wmi thinkpad_acpi ledtrig_audio platform_profile snd soundcore rfkill tpm_crb i2c_hid_acpi i2c_hid acpi_pad tpm_tis mac_hid tpm_tis_core pkcs8_key_parser fuse zram ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 usbhid dm_crypt cbc encrypted_keys trusted asn1_encoder tee tpm rng_core dm_mod rtsx_pci_sdmmc mmc_core serio_raw atkbd libps2 crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd xhci_pci rtsx_pci xhci_pci_renesas i8042 serio kvmgt mdev vfio_iommu_type1 vfio i915 i2c_algo_bit intel_gtt ttm agpgart video drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec
>  drm kvm irqbypass
> CPU: 1 PID: 26546 Comm: kworker/1:1 Not tainted 5.14.3 #1 c719caf0c6c208968387ed83e3061ac05d0faf2f
> Workqueue: events free_user_ns
> RIP: 0010:dec_ucount+0x43/0x50
> Code: 14 01 48 8b 02 48 89 c6 48 83 ee 01 78 1c f0 48 0f b1 32 75 f0 48 8b 41 10 48 8b 88 e8 01 00 00 48 85 c9 75 d9 e9 0d fd ff ff <0f> 0b eb e7 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 f8 48
> RSP: 0018:ffffa82cc2bd7e60 EFLAGS: 00010297
> RAX: 0000000000000000 RBX: ffffa2f53298ee50 RCX: ffffa2f3c0061000
> RDX: ffffa2f3c0061020 RSI: ffffffffffffffff RDI: ffffa2f3c0061000
> RBP: ffffa2f53298ebe0 R08: 0000000000000020 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000000 R12: ffffa2f3c0061000
> R13: 00000000ffffffff R14: 0000000000000000 R15: 0000000000000001
> FS:  0000000000000000(0000) GS:ffffa2f599680000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000628f892be9f8 CR3: 000000002880e004 CR4: 00000000003706e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  free_user_ns+0x73/0x110
>  process_one_work+0x1e1/0x380
>  worker_thread+0x50/0x3a0
>  ? rescuer_thread+0x360/0x360
>  kthread+0x127/0x150
>  ? set_kthread_struct+0x40/0x40
>  ret_from_fork+0x22/0x30
> ---[ end trace eb7a8d38b64b2d3a ]---
> BUG: kernel NULL pointer dereference, address: 00000000000001e8
> #PF: supervisor read access in kernel mode
> #PF: error_code(0x0000) - not-present page
> Oops: 0000 [#1] SMP PTI
> CPU: 1 PID: 26546 Comm: kworker/1:1 Tainted: G        W         5.14.3 #1 c719caf0c6c208968387ed83e3061ac05d0faf2f
> Workqueue: events free_user_ns
> RIP: 0010:dec_ucount+0x32/0x50
> Code: 74 34 89 f6 48 89 f9 4c 8d 04 f5 20 00 00 00 4a 8d 14 01 48 8b 02 48 89 c6 48 83 ee 01 78 1c f0 48 0f b1 32 75 f0 48 8b 41 10 <48> 8b 88 e8 01 00 00 48 85 c9 75 d9 e9 0d fd ff ff 0f 0b eb e7 66
> RSP: 0018:ffffa82cc2bd7e60 EFLAGS: 00010297
> RAX: 0000000000000000 RBX: ffffa2f53298ee50 RCX: ffffa2f3c0061000
> RDX: ffffa2f3c0061020 RSI: ffffffffffffffff RDI: ffffa2f3c0061000
> RBP: ffffa2f53298ebe0 R08: 0000000000000020 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000000 R12: ffffa2f3c0061000
> R13: 00000000ffffffff R14: 0000000000000000 R15: 0000000000000001
> FS:  0000000000000000(0000) GS:ffffa2f599680000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000000001e8 CR3: 000000002880e004 CR4: 00000000003706e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  free_user_ns+0x73/0x110
>  process_one_work+0x1e1/0x380
>  worker_thread+0x50/0x3a0
>  ? rescuer_thread+0x360/0x360
>  kthread+0x127/0x150
>  ? set_kthread_struct+0x40/0x40
>  ret_from_fork+0x22/0x30

WARNING: multiple messages have this Message-ID (diff)
From: ebiederm@xmission.com (Eric W. Biederman)
To: Jordan Glover <Golden_Miller83@protonmail.ch>
Cc: LKML <linux-kernel@vger.kernel.org>,
	 "linux-mm\@kvack.org" <linux-mm@kvack.org>,
	 "legion\@kernel.org" <legion@kernel.org>,
	 "containers\@lists.linux-foundation.org"
	<containers@lists.linux-foundation.org>
Subject: Re: linux 5.14.3: free_user_ns causes NULL pointer dereference
Date: Wed, 15 Sep 2021 16:02:19 -0500	[thread overview]
Message-ID: <87ee9pa6xw.fsf@disp2133> (raw)
In-Reply-To: <1M9_d6wrcu6rdPe1ON0_k0lOxJMyyot3KAb1gdyuwzDPC777XVUWPHoTCEVmcK3fYfgu7sIo3PSaLe9KulUdm4TWVuqlbKyYGxRAjsf_Cpk=@protonmail.ch> (Jordan Glover's message of "Wed, 15 Sep 2021 19:49:31 +0000")

Jordan Glover <Golden_Miller83@protonmail.ch> writes:

> Hi, recently I hit system freeze after I was closing few containerized apps on my system. As for now it occurred only once on linux 5.14.3. I think it maybe be related to "Count rlimits in each user namespace" patchset merged during 5.14 window
>
> https://lore.kernel.org/all/257aa5fb1a7d81cf0f4c34f39ada2320c4284771.1619094428.git.legion@kernel.org/T/#u

So that warning comes from:

void dec_ucount(struct ucounts *ucounts, enum ucount_type type)
{
	struct ucounts *iter;
	for (iter = ucounts; iter; iter = iter->ns->ucounts) {
		long dec = atomic_long_dec_if_positive(&iter->ucount[type]);
		WARN_ON_ONCE(dec < 0);
	}
	put_ucounts(ucounts);
}

Which certainly looks like a reference count bug.  It could also be a
memory stomp somewhere close.

Do you have any idea what else was going on?  This location is the
symptom but not the actual cause.

Eric



>
> Logs below:
>
> ------------[ cut here ]------------
> WARNING: CPU: 1 PID: 26546 at kernel/ucount.c:253 dec_ucount+0x43/0x50
> Modules linked in: nft_ct nft_fib_ipv4 nft_fib wireguard curve25519_x86_64 libcurve25519_generic libchacha20poly1305 chacha_x86_64 poly1305_x86_64 udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic libchacha ccm algif_aead des_generic libdes ecb algif_skcipher cmac md4 algif_hash af_alg hid_sensor_custom_intel_hinge hid_sensor_als hid_sensor_magn_3d hid_sensor_rotation hid_sensor_accel_3d hid_sensor_gyro_3d hid_sensor_trigger industrialio_triggered_buffer hid_sensor_iio_common kfifo_buf industrialio hid_sensor_custom hid_sensor_hub cros_ec_ishtp cros_ec intel_ishtp_loader nft_counter intel_ishtp_hid snd_hda_codec_hdmi intel_rapl_msr xt_mark ipt_REJECT nf_reject_ipv4 snd_ctl_led xt_LOG snd_hda_codec_conexant nf_log_syslog snd_hda_codec_generic xt_addrtype xt_tcpudp xt_conntrack nf_conntrack nf_defrag_ipv4 mei_hdcp snd_hda_intel nft_compat wmi_bmof nf_tables intel_rapl_common libcrc32c think_lmi intel_tcc_cooling snd_intel_dspcfg firmware_attributes_class nfnetlink iwlmv
 m
>  intel_wmi_thunderbolt mac80211 x86_pkg_temp_thermal snd_hda_codec intel_powerclamp coretemp libarc4 vfat fat kvm_intel rapl intel_cstate snd_hwdep intel_uncore iwlwifi snd_hda_core mousedev joydev snd_pcm psmouse cfg80211 snd_timer mei_me ucsi_acpi wacom intel_ish_ipc intel_xhci_usb_role_switch mei intel_pch_thermal typec_ucsi roles typec intel_ishtp wmi thinkpad_acpi ledtrig_audio platform_profile snd soundcore rfkill tpm_crb i2c_hid_acpi i2c_hid acpi_pad tpm_tis mac_hid tpm_tis_core pkcs8_key_parser fuse zram ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 usbhid dm_crypt cbc encrypted_keys trusted asn1_encoder tee tpm rng_core dm_mod rtsx_pci_sdmmc mmc_core serio_raw atkbd libps2 crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd xhci_pci rtsx_pci xhci_pci_renesas i8042 serio kvmgt mdev vfio_iommu_type1 vfio i915 i2c_algo_bit intel_gtt ttm agpgart video drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops cec
>  drm kvm irqbypass
> CPU: 1 PID: 26546 Comm: kworker/1:1 Not tainted 5.14.3 #1 c719caf0c6c208968387ed83e3061ac05d0faf2f
> Workqueue: events free_user_ns
> RIP: 0010:dec_ucount+0x43/0x50
> Code: 14 01 48 8b 02 48 89 c6 48 83 ee 01 78 1c f0 48 0f b1 32 75 f0 48 8b 41 10 48 8b 88 e8 01 00 00 48 85 c9 75 d9 e9 0d fd ff ff <0f> 0b eb e7 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 f8 48
> RSP: 0018:ffffa82cc2bd7e60 EFLAGS: 00010297
> RAX: 0000000000000000 RBX: ffffa2f53298ee50 RCX: ffffa2f3c0061000
> RDX: ffffa2f3c0061020 RSI: ffffffffffffffff RDI: ffffa2f3c0061000
> RBP: ffffa2f53298ebe0 R08: 0000000000000020 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000000 R12: ffffa2f3c0061000
> R13: 00000000ffffffff R14: 0000000000000000 R15: 0000000000000001
> FS:  0000000000000000(0000) GS:ffffa2f599680000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000628f892be9f8 CR3: 000000002880e004 CR4: 00000000003706e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  free_user_ns+0x73/0x110
>  process_one_work+0x1e1/0x380
>  worker_thread+0x50/0x3a0
>  ? rescuer_thread+0x360/0x360
>  kthread+0x127/0x150
>  ? set_kthread_struct+0x40/0x40
>  ret_from_fork+0x22/0x30
> ---[ end trace eb7a8d38b64b2d3a ]---
> BUG: kernel NULL pointer dereference, address: 00000000000001e8
> #PF: supervisor read access in kernel mode
> #PF: error_code(0x0000) - not-present page
> Oops: 0000 [#1] SMP PTI
> CPU: 1 PID: 26546 Comm: kworker/1:1 Tainted: G        W         5.14.3 #1 c719caf0c6c208968387ed83e3061ac05d0faf2f
> Workqueue: events free_user_ns
> RIP: 0010:dec_ucount+0x32/0x50
> Code: 74 34 89 f6 48 89 f9 4c 8d 04 f5 20 00 00 00 4a 8d 14 01 48 8b 02 48 89 c6 48 83 ee 01 78 1c f0 48 0f b1 32 75 f0 48 8b 41 10 <48> 8b 88 e8 01 00 00 48 85 c9 75 d9 e9 0d fd ff ff 0f 0b eb e7 66
> RSP: 0018:ffffa82cc2bd7e60 EFLAGS: 00010297
> RAX: 0000000000000000 RBX: ffffa2f53298ee50 RCX: ffffa2f3c0061000
> RDX: ffffa2f3c0061020 RSI: ffffffffffffffff RDI: ffffa2f3c0061000
> RBP: ffffa2f53298ebe0 R08: 0000000000000020 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000000 R12: ffffa2f3c0061000
> R13: 00000000ffffffff R14: 0000000000000000 R15: 0000000000000001
> FS:  0000000000000000(0000) GS:ffffa2f599680000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00000000000001e8 CR3: 000000002880e004 CR4: 00000000003706e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>  free_user_ns+0x73/0x110
>  process_one_work+0x1e1/0x380
>  worker_thread+0x50/0x3a0
>  ? rescuer_thread+0x360/0x360
>  kthread+0x127/0x150
>  ? set_kthread_struct+0x40/0x40
>  ret_from_fork+0x22/0x30


  reply	other threads:[~2021-09-15 21:02 UTC|newest]

Thread overview: 71+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-15 19:49 linux 5.14.3: free_user_ns causes NULL pointer dereference Jordan Glover
2021-09-15 19:49 ` Jordan Glover
2021-09-15 21:02 ` Eric W. Biederman [this message]
2021-09-15 21:02   ` Eric W. Biederman
2021-09-15 22:42   ` Jordan Glover
2021-09-15 22:42     ` Jordan Glover
2021-09-15 23:44     ` Yu Zhao
2021-09-15 23:44       ` Yu Zhao
2021-09-17 16:15       ` Eric W. Biederman
2021-09-17 16:15         ` Eric W. Biederman
2021-09-17 18:45         ` Yu Zhao
2021-09-17 18:45           ` Yu Zhao
2021-09-15 23:47     ` Jordan Glover
2021-09-15 23:47       ` Jordan Glover
2021-09-16 17:30       ` Eric W. Biederman
2021-09-16 17:30         ` Eric W. Biederman
2021-09-16 19:14         ` Alexey Gladkov
2021-09-16 19:14           ` Alexey Gladkov
2021-09-28 13:40         ` Jordan Glover
2021-09-28 13:40           ` Jordan Glover
2021-09-29 17:36           ` Alexey Gladkov
2021-09-29 17:36             ` Alexey Gladkov
2021-09-29 21:39             ` Jordan Glover
2021-09-29 21:39               ` Jordan Glover
2021-09-30 13:06               ` Alexey Gladkov
2021-09-30 22:27                 ` Yu Zhao
2021-09-30 22:27                   ` Yu Zhao
2021-10-04 17:10                   ` Eric W. Biederman
2021-10-04 17:19                     ` Eric W. Biederman
2021-10-04 21:34                       ` Yu Zhao
2021-10-06  7:57                       ` Rune Kleveland
2021-10-10  8:59                       ` Rune Kleveland
2021-10-11 13:09                         ` Hillf Danton
2021-10-12 17:31                         ` Eric W. Biederman
2021-10-15 22:10                         ` [CFT][PATCH] ucounts: Fix signal ucount refcounting Eric W. Biederman
2021-10-15 23:09                           ` Alexey Gladkov
2021-10-16 17:34                             ` Eric W. Biederman
2021-10-17 19:35                               ` Yu Zhao
2021-10-18 15:35                                 ` Eric W. Biederman
2021-10-16  2:08                           ` Hillf Danton
2021-10-16 18:00                             ` Eric W. Biederman
2021-10-17 16:47                           ` Rune Kleveland
2021-10-18  6:25                             ` Yu Zhao
2021-10-18 10:31                               ` Jordan Glover
2021-10-18 16:06                           ` [PATCH v2] " Eric W. Biederman
2021-10-18 17:21                             ` [PATCH 0/3] ucounts: misc fixes Eric W. Biederman
2021-10-18 17:23                               ` [PATCH 1/3] ucounts: Pair inc_rlimit_ucounts with dec_rlimit_ucoutns in commit_creds Eric W. Biederman
2021-10-18 17:23                               ` [PATCH 2/3] ucounts: Proper error handling in set_cred_ucounts Eric W. Biederman
2021-10-18 17:24                               ` [PATCH 3/3] ucounts: Move get_ucounts from cred_alloc_blank to key_change_session_keyring Eric W. Biederman
2021-10-24 17:36                                 ` kernel test robot
2021-10-25 14:13                                   ` Eric W. Biederman
2021-11-06  5:05                                 ` kernel test robot
2021-11-06  5:05                                   ` kernel test robot
2021-11-06 20:22                                 ` kernel test robot
2021-11-06 20:22                                   ` kernel test robot
2021-10-18 17:54                               ` [PATCH 0/4] ucounts: misc cleanups Eric W. Biederman
2021-10-18 17:55                                 ` [PATCH 1/4] ucounts: In set_cred_ucounts assume new->ucounts is non-NULL Eric W. Biederman
2021-10-18 17:56                                 ` [PATCH 2/4] ucounts: Remove unnecessary test for NULL ucount in get_ucounts Eric W. Biederman
2021-10-18 17:56                                 ` [PATCH 3/4] ucounts: Add get_ucounts_or_wrap for clarity Eric W. Biederman
2021-10-18 17:57                                 ` [PATCH 4/4] ucounts: Use atomic_long_sub_return " Eric W. Biederman
2021-10-18 22:29                                 ` [PATCH 0/4] ucounts: misc cleanups Yu Zhao
2021-10-18 22:28                               ` [PATCH 0/3] ucounts: misc fixes Yu Zhao
2021-10-18 22:26                             ` [PATCH v2] ucounts: Fix signal ucount refcounting Yu Zhao
2021-10-11 13:39                       ` linux 5.14.3: free_user_ns causes NULL pointer dereference Alexey Gladkov
2021-10-06  2:12                   ` Hillf Danton
2021-10-06  6:22                     ` Yu Zhao
2021-10-07 13:28                     ` Jordan Glover
2021-10-10 11:26                       ` Hillf Danton
2021-10-03 19:37             ` Jordan Glover
2021-10-03 19:37               ` Jordan Glover
2021-10-20  7:39 Antoine Martin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87ee9pa6xw.fsf@disp2133 \
    --to=ebiederm@xmission.com \
    --cc=Golden_Miller83@protonmail.ch \
    --cc=containers@lists.linux-foundation.org \
    --cc=legion@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.