All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Eric Dumazet <edumazet@google.com>,
	syzbot <syzkaller@googlegroups.com>,
	David Howells <dhowells@redhat.com>,
	"David S. Miller" <davem@davemloft.net>
Subject: [PATCH 5.0 63/89] rxrpc: fix race condition in rxrpc_input_packet()
Date: Tue, 30 Apr 2019 13:38:54 +0200	[thread overview]
Message-ID: <20190430113612.659509990@linuxfoundation.org> (raw)
In-Reply-To: <20190430113609.741196396@linuxfoundation.org>

From: Eric Dumazet <edumazet@google.com>

commit 032be5f19a94de51093851757089133dcc1e92aa upstream.

After commit 5271953cad31 ("rxrpc: Use the UDP encap_rcv hook"),
rxrpc_input_packet() is directly called from lockless UDP receive
path, under rcu_read_lock() protection.

It must therefore use RCU rules :

- udp_sk->sk_user_data can be cleared at any point in this function.
  rcu_dereference_sk_user_data() is what we need here.

- Also, since sk_user_data might have been set in rxrpc_open_socket()
  we must observe a proper RCU grace period before kfree(local) in
  rxrpc_lookup_local()

v4: @local can be NULL in xrpc_lookup_local() as reported by kbuild test robot <lkp@intel.com>
        and Julia Lawall <julia.lawall@lip6.fr>, thanks !

v3,v2 : addressed David Howells feedback, thanks !

syzbot reported :

kasan: CONFIG_KASAN_INLINE enabled
kasan: GPF could be caused by NULL-ptr deref or user memory access
general protection fault: 0000 [#1] PREEMPT SMP KASAN
CPU: 0 PID: 19236 Comm: syz-executor703 Not tainted 5.1.0-rc6 #79
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:__lock_acquire+0xbef/0x3fb0 kernel/locking/lockdep.c:3573
Code: 00 0f 85 a5 1f 00 00 48 81 c4 10 01 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 48 b8 00 00 00 00 00 fc ff df 4c 89 ea 48 c1 ea 03 <80> 3c 02 00 0f 85 4a 21 00 00 49 81 7d 00 20 54 9c 89 0f 84 cf f4
RSP: 0018:ffff88809d7aef58 EFLAGS: 00010002
RAX: dffffc0000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000026 RSI: 0000000000000000 RDI: 0000000000000001
RBP: ffff88809d7af090 R08: 0000000000000001 R09: 0000000000000001
R10: ffffed1015d05bc7 R11: ffff888089428600 R12: 0000000000000000
R13: 0000000000000130 R14: 0000000000000001 R15: 0000000000000001
FS:  00007f059044d700(0000) GS:ffff8880ae800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000004b6040 CR3: 00000000955ca000 CR4: 00000000001406f0
Call Trace:
 lock_acquire+0x16f/0x3f0 kernel/locking/lockdep.c:4211
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0x95/0xcd kernel/locking/spinlock.c:152
 skb_queue_tail+0x26/0x150 net/core/skbuff.c:2972
 rxrpc_reject_packet net/rxrpc/input.c:1126 [inline]
 rxrpc_input_packet+0x4a0/0x5536 net/rxrpc/input.c:1414
 udp_queue_rcv_one_skb+0xaf2/0x1780 net/ipv4/udp.c:2011
 udp_queue_rcv_skb+0x128/0x730 net/ipv4/udp.c:2085
 udp_unicast_rcv_skb.isra.0+0xb9/0x360 net/ipv4/udp.c:2245
 __udp4_lib_rcv+0x701/0x2ca0 net/ipv4/udp.c:2301
 udp_rcv+0x22/0x30 net/ipv4/udp.c:2482
 ip_protocol_deliver_rcu+0x60/0x8f0 net/ipv4/ip_input.c:208
 ip_local_deliver_finish+0x23b/0x390 net/ipv4/ip_input.c:234
 NF_HOOK include/linux/netfilter.h:289 [inline]
 NF_HOOK include/linux/netfilter.h:283 [inline]
 ip_local_deliver+0x1e9/0x520 net/ipv4/ip_input.c:255
 dst_input include/net/dst.h:450 [inline]
 ip_rcv_finish+0x1e1/0x300 net/ipv4/ip_input.c:413
 NF_HOOK include/linux/netfilter.h:289 [inline]
 NF_HOOK include/linux/netfilter.h:283 [inline]
 ip_rcv+0xe8/0x3f0 net/ipv4/ip_input.c:523
 __netif_receive_skb_one_core+0x115/0x1a0 net/core/dev.c:4987
 __netif_receive_skb+0x2c/0x1c0 net/core/dev.c:5099
 netif_receive_skb_internal+0x117/0x660 net/core/dev.c:5202
 napi_frags_finish net/core/dev.c:5769 [inline]
 napi_gro_frags+0xade/0xd10 net/core/dev.c:5843
 tun_get_user+0x2f24/0x3fb0 drivers/net/tun.c:1981
 tun_chr_write_iter+0xbd/0x156 drivers/net/tun.c:2027
 call_write_iter include/linux/fs.h:1866 [inline]
 do_iter_readv_writev+0x5e1/0x8e0 fs/read_write.c:681
 do_iter_write fs/read_write.c:957 [inline]
 do_iter_write+0x184/0x610 fs/read_write.c:938
 vfs_writev+0x1b3/0x2f0 fs/read_write.c:1002
 do_writev+0x15e/0x370 fs/read_write.c:1037
 __do_sys_writev fs/read_write.c:1110 [inline]
 __se_sys_writev fs/read_write.c:1107 [inline]
 __x64_sys_writev+0x75/0xb0 fs/read_write.c:1107
 do_syscall_64+0x103/0x610 arch/x86/entry/common.c:290
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

Fixes: 5271953cad31 ("rxrpc: Use the UDP encap_rcv hook")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Reported-by: syzbot <syzkaller@googlegroups.com>
Acked-by: David Howells <dhowells@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 net/rxrpc/input.c        |   12 ++++++++----
 net/rxrpc/local_object.c |    3 ++-
 2 files changed, 10 insertions(+), 5 deletions(-)

--- a/net/rxrpc/input.c
+++ b/net/rxrpc/input.c
@@ -1155,19 +1155,19 @@ int rxrpc_extract_header(struct rxrpc_sk
  * handle data received on the local endpoint
  * - may be called in interrupt context
  *
- * The socket is locked by the caller and this prevents the socket from being
- * shut down and the local endpoint from going away, thus sk_user_data will not
- * be cleared until this function returns.
+ * [!] Note that as this is called from the encap_rcv hook, the socket is not
+ * held locked by the caller and nothing prevents sk_user_data on the UDP from
+ * being cleared in the middle of processing this function.
  *
  * Called with the RCU read lock held from the IP layer via UDP.
  */
 int rxrpc_input_packet(struct sock *udp_sk, struct sk_buff *skb)
 {
+	struct rxrpc_local *local = rcu_dereference_sk_user_data(udp_sk);
 	struct rxrpc_connection *conn;
 	struct rxrpc_channel *chan;
 	struct rxrpc_call *call = NULL;
 	struct rxrpc_skb_priv *sp;
-	struct rxrpc_local *local = udp_sk->sk_user_data;
 	struct rxrpc_peer *peer = NULL;
 	struct rxrpc_sock *rx = NULL;
 	unsigned int channel;
@@ -1175,6 +1175,10 @@ int rxrpc_input_packet(struct sock *udp_
 
 	_enter("%p", udp_sk);
 
+	if (unlikely(!local)) {
+		kfree_skb(skb);
+		return 0;
+	}
 	if (skb->tstamp == 0)
 		skb->tstamp = ktime_get_real();
 
--- a/net/rxrpc/local_object.c
+++ b/net/rxrpc/local_object.c
@@ -304,7 +304,8 @@ nomem:
 	ret = -ENOMEM;
 sock_error:
 	mutex_unlock(&rxnet->local_mutex);
-	kfree(local);
+	if (local)
+		call_rcu(&local->rcu, rxrpc_local_rcu);
 	_leave(" = %d", ret);
 	return ERR_PTR(ret);
 



  parent reply	other threads:[~2019-04-30 11:50 UTC|newest]

Thread overview: 107+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-30 11:37 [PATCH 5.0 00/89] 5.0.11-stable review Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 01/89] netfilter: nf_tables: bogus EBUSY when deleting set after flush Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 02/89] netfilter: nf_tables: bogus EBUSY in helper removal from transaction Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 03/89] intel_th: gth: Fix an off-by-one in output unassigning Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 04/89] powerpc/vdso32: fix CLOCK_MONOTONIC on PPC64 Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 05/89] ALSA: hda/realtek - Move to ACT_INIT state Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 06/89] fs/proc/proc_sysctl.c: Fix a NULL pointer dereference Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 07/89] block, bfq: fix use after free in bfq_bfqq_expire Greg Kroah-Hartman
2019-04-30 11:37 ` [PATCH 5.0 08/89] cifs: fix memory leak in SMB2_read Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 09/89] cifs: fix page reference leak with readv/writev Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 10/89] cifs: do not attempt cifs operation on smb2+ rename error Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 11/89] tracing: Fix a memory leak by early error exit in trace_pid_write() Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 12/89] tracing: Fix buffer_ref pipe ops Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 13/89] crypto: xts - Fix atomic sleep when walking skcipher Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 14/89] crypto: lrw " Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 15/89] gpio: eic: sprd: Fix incorrect irq type setting for the sync EIC Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 16/89] zram: pass down the bvec we need to read into in the work struct Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 17/89] lib/Kconfig.debug: fix build error without CONFIG_BLOCK Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 18/89] MIPS: scall64-o32: Fix indirect syscall number load Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 19/89] trace: Fix preempt_enable_no_resched() abuse Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 20/89] mm: do not boost watermarks to avoid fragmentation for the DISCONTIG memory model Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 21/89] arm64: mm: Ensure tail of unaligned initrd is reserved Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 22/89] IB/rdmavt: Fix frwr memory registration Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 23/89] RDMA/mlx5: Do not allow the user to write to the clock page Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 24/89] RDMA/mlx5: Use rdma_user_map_io for mapping BAR pages Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 25/89] RDMA/ucontext: Fix regression with disassociate Greg Kroah-Hartman
2019-05-03 11:47   ` Michal Kubecek
2019-05-03 11:48     ` Michal Kubecek
2019-04-30 11:38 ` [PATCH 5.0 26/89] sched/numa: Fix a possible divide-by-zero Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 27/89] ceph: only use d_name directly when parent is locked Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 28/89] ceph: ensure d_name stability in ceph_dentry_hash() Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 29/89] ceph: fix ci->i_head_snapc leak Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 30/89] nfsd: Dont release the callback slot unless it was actually held Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 31/89] nfsd: wake waiters blocked on file_lock before deleting it Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 32/89] nfsd: wake blocked file lock waiters before sending callback Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 33/89] sunrpc: dont mark uninitialised items as VALID Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 34/89] perf/x86/intel: Update KBL Package C-state events to also include PC8/PC9/PC10 counters Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 35/89] Input: synaptics-rmi4 - write config register values to the right offset Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 36/89] vfio/type1: Limit DMA mappings per container Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 37/89] dmaengine: sh: rcar-dmac: With cyclic DMA residue 0 is valid Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 38/89] dmaengine: sh: rcar-dmac: Fix glitch in dmaengine_tx_status Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 39/89] dmaengine: mediatek-cqdma: fix wrong register usage in mtk_cqdma_start Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 40/89] ARM: 8857/1: efi: enable CP15 DMB instructions before cleaning the cache Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 41/89] powerpc/mm/radix: Make Radix require HUGETLB_PAGE Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 42/89] drm/vc4: Fix memory leak during gpu reset Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 43/89] drm/ttm: fix re-init of global structures Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 44/89] Revert "drm/i915/fbdev: Actually configure untiled displays" Greg Kroah-Hartman
2019-05-01 13:02   ` Sasha Levin
2019-05-01 13:06     ` Sasha Levin
2019-05-01 13:08     ` Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 45/89] drm/vc4: Fix compilation error reported by kbuild test bot Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 46/89] USB: Add new USB LPM helpers Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 47/89] USB: Consolidate LPM checks to avoid enabling LPM twice Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 48/89] ext4: fix some error pointer dereferences Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 49/89] loop: do not print warn message if partition scan is successful Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 50/89] tipc: handle the err returned from cmd header function Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 51/89] slip: make slhc_free() silently accept an error pointer Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 52/89] workqueue: Try to catch flush_work() without INIT_WORK() Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 53/89] binder: fix handling of misaligned binder object Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 54/89] sched/deadline: Correctly handle active 0-lag timers Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 55/89] mac80211_hwsim: calculate if_combination.max_interfaces Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 56/89] NFS: Forbid setting AF_INET6 to "struct sockaddr_in"->sin_family Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 57/89] netfilter: ebtables: CONFIG_COMPAT: drop a bogus WARN_ON Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 58/89] fm10k: Fix a potential NULL pointer dereference Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 59/89] tipc: check bearer name with right length in tipc_nl_compat_bearer_enable Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 60/89] tipc: check link name with right length in tipc_nl_compat_link_set Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 61/89] net: netrom: Fix error cleanup path of nr_proto_init Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 62/89] net/rds: Check address length before reading address family Greg Kroah-Hartman
2019-04-30 11:38 ` Greg Kroah-Hartman [this message]
2019-04-30 11:38 ` [PATCH 5.0 64/89] pin iocb through aio Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 65/89] aio: fold lookup_kiocb() into its sole caller Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 66/89] aio: keep io_event in aio_kiocb Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 67/89] aio: store event at final iocb_put() Greg Kroah-Hartman
2019-04-30 11:38 ` [PATCH 5.0 68/89] Fix aio_poll() races Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 69/89] x86, retpolines: Raise limit for generating indirect calls from switch-case Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 70/89] x86/retpolines: Disable switch jump tables when retpolines are enabled Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 71/89] rdma: fix build errors on s390 and MIPS due to bad ZERO_PAGE use Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 72/89] ipv4: add sanity checks in ipv4_link_failure() Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 73/89] ipv4: set the tcp_min_rtt_wlen range from 0 to one day Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 74/89] mlxsw: spectrum: Fix autoneg status in ethtool Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 75/89] net/mlx5e: ethtool, Remove unsupported SFP EEPROM high pages query Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 76/89] net: rds: exchange of 8K and 1M pool Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 77/89] net/rose: fix unbound loop in rose_loopback_timer() Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 78/89] net: stmmac: move stmmac_check_ether_addr() to driver probe Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 79/89] net/tls: fix refcount adjustment in fallback Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 80/89] stmmac: pci: Adjust IOT2000 matching Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 81/89] team: fix possible recursive locking when add slaves Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 82/89] net: socionext: replace napi_alloc_frag with the netdev variant on init Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 83/89] net/ncsi: handle overflow when incrementing mac address Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 84/89] mlxsw: pci: Reincrease PCI reset timeout Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 85/89] mlxsw: spectrum: Put MC TCs into DWRR mode Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 86/89] net/mlx5e: Fix the max MTU check in case of XDP Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 87/89] net/mlx5e: Fix use-after-free after xdp_return_frame Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 88/89] net/tls: avoid potential deadlock in tls_set_device_offload_rx() Greg Kroah-Hartman
2019-04-30 11:39 ` [PATCH 5.0 89/89] net/tls: dont leak IV and record seq when offload fails Greg Kroah-Hartman
2019-04-30 17:06 ` [PATCH 5.0 00/89] 5.0.11-stable review kernelci.org bot
2019-04-30 22:33 ` shuah
2019-05-01  7:55   ` Greg Kroah-Hartman
2019-05-01  6:21 ` Naresh Kamboju
2019-05-01  7:55   ` Greg Kroah-Hartman
2019-05-01  8:26 ` Jon Hunter
2019-05-01  8:26   ` Jon Hunter
2019-05-01  8:43   ` Greg Kroah-Hartman
2019-05-01 16:44 ` Guenter Roeck
2019-05-01 17:14   ` Greg Kroah-Hartman
2019-05-02  5:30 ` Bharath Vedartham
2019-05-02  6:44   ` Greg Kroah-Hartman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190430113612.659509990@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=davem@davemloft.net \
    --cc=dhowells@redhat.com \
    --cc=edumazet@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=syzkaller@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.