linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Su Yanjun <suyj.fnst@cn.fujitsu.com>,
	"David S. Miller" <davem@davemloft.net>
Subject: [PATCH 4.14 05/89] net: 8139cp: fix a BUG triggered by changing mtu with network traffic
Date: Fri, 14 Dec 2018 12:59:18 +0100	[thread overview]
Message-ID: <20181214115729.931823768@linuxfoundation.org> (raw)
In-Reply-To: <20181214115729.658859279@linuxfoundation.org>

4.14-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Su Yanjun <suyj.fnst@cn.fujitsu.com>

[ Upstream commit a5d4a89245ead1f37ed135213653c5beebea4237 ]

When changing mtu many times with traffic, a bug is triggered:

[ 1035.684037] kernel BUG at lib/dynamic_queue_limits.c:26!
[ 1035.684042] invalid opcode: 0000 [#1] SMP
[ 1035.684049] Modules linked in: loop binfmt_misc 8139cp(OE) macsec
tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag tcp_lp
fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE
nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4
nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun
bridge stp llc ebtable_filter ebtables ip6table_filter devlink
ip6_tables iptable_filter sunrpc snd_hda_codec_generic snd_hda_intel
snd_hda_codec snd_hda_core snd_hwdep ppdev snd_seq iosf_mbi crc32_pclmul
parport_pc snd_seq_device ghash_clmulni_intel parport snd_pcm
aesni_intel joydev lrw snd_timer virtio_balloon sg gf128mul glue_helper
ablk_helper cryptd snd soundcore i2c_piix4 pcspkr ip_tables xfs
libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic
[ 1035.684102]  pata_acpi virtio_console qxl drm_kms_helper syscopyarea
sysfillrect sysimgblt floppy fb_sys_fops crct10dif_pclmul
crct10dif_common ttm crc32c_intel serio_raw ata_piix drm libata 8139too
virtio_pci drm_panel_orientation_quirks virtio_ring virtio mii dm_mirror
dm_region_hash dm_log dm_mod [last unloaded: 8139cp]
[ 1035.684132] CPU: 9 PID: 25140 Comm: if-mtu-change Kdump: loaded
Tainted: G           OE  ------------ T 3.10.0-957.el7.x86_64 #1
[ 1035.684134] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 1035.684136] task: ffff8f59b1f5a080 ti: ffff8f5a2e32c000 task.ti:
ffff8f5a2e32c000
[ 1035.684149] RIP: 0010:[<ffffffffba3a40d0>]  [<ffffffffba3a40d0>]
dql_completed+0x180/0x190
[ 1035.684162] RSP: 0000:ffff8f5a75483e50  EFLAGS: 00010093
[ 1035.684162] RAX: 00000000000000c2 RBX: ffff8f5a6f91c000 RCX:
0000000000000000
[ 1035.684162] RDX: 0000000000000000 RSI: 0000000000000184 RDI:
ffff8f599fea3ec0
[ 1035.684162] RBP: ffff8f5a75483ea8 R08: 00000000000000c2 R09:
0000000000000000
[ 1035.684162] R10: 00000000000616ef R11: ffff8f5a75483b56 R12:
ffff8f599fea3e00
[ 1035.684162] R13: 0000000000000001 R14: 0000000000000000 R15:
0000000000000184
[ 1035.684162] FS:  00007fa8434de740(0000) GS:ffff8f5a75480000(0000)
knlGS:0000000000000000
[ 1035.684162] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1035.684162] CR2: 00000000004305d0 CR3: 000000024eb66000 CR4:
00000000001406e0
[ 1035.684162] Call Trace:
[ 1035.684162]  <IRQ>
[ 1035.684162]  [<ffffffffc08cbaf8>] ? cp_interrupt+0x478/0x580 [8139cp]
[ 1035.684162]  [<ffffffffba14a294>]
__handle_irq_event_percpu+0x44/0x1c0
[ 1035.684162]  [<ffffffffba14a442>] handle_irq_event_percpu+0x32/0x80
[ 1035.684162]  [<ffffffffba14a4cc>] handle_irq_event+0x3c/0x60
[ 1035.684162]  [<ffffffffba14db29>] handle_fasteoi_irq+0x59/0x110
[ 1035.684162]  [<ffffffffba02e554>] handle_irq+0xe4/0x1a0
[ 1035.684162]  [<ffffffffba7795dd>] do_IRQ+0x4d/0xf0
[ 1035.684162]  [<ffffffffba76b362>] common_interrupt+0x162/0x162
[ 1035.684162]  <EOI>
[ 1035.684162]  [<ffffffffba0c2ae4>] ? __wake_up_bit+0x24/0x70
[ 1035.684162]  [<ffffffffba1e46f5>] ? do_set_pte+0xd5/0x120
[ 1035.684162]  [<ffffffffba1b64fb>] unlock_page+0x2b/0x30
[ 1035.684162]  [<ffffffffba1e4879>] do_read_fault.isra.61+0x139/0x1b0
[ 1035.684162]  [<ffffffffba1e9134>] handle_pte_fault+0x2f4/0xd10
[ 1035.684162]  [<ffffffffba1ebc6d>] handle_mm_fault+0x39d/0x9b0
[ 1035.684162]  [<ffffffffba76f5e3>] __do_page_fault+0x203/0x500
[ 1035.684162]  [<ffffffffba76f9c6>] trace_do_page_fault+0x56/0x150
[ 1035.684162]  [<ffffffffba76ef42>] do_async_page_fault+0x22/0xf0
[ 1035.684162]  [<ffffffffba76b788>] async_page_fault+0x28/0x30
[ 1035.684162] Code: 54 c7 47 54 ff ff ff ff 44 0f 49 ce 48 8b 35 48 2f
9c 00 48 89 77 58 e9 fe fe ff ff 0f 1f 80 00 00 00 00 41 89 d1 e9 ef fe
ff ff <0f> 0b 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 55 8d 42 ff 48
[ 1035.684162] RIP  [<ffffffffba3a40d0>] dql_completed+0x180/0x190
[ 1035.684162]  RSP <ffff8f5a75483e50>

It's not the same as in 7fe0ee09 patch described.
As 8139cp uses shared irq mode, other device irq will trigger
cp_interrupt to execute.

cp_change_mtu
 -> cp_close
 -> cp_open

In cp_close routine  just before free_irq(), some interrupt may occur.
In my environment, cp_interrupt exectutes and IntrStatus is 0x4,
exactly TxOk. That will cause cp_tx to wake device queue.

As device queue is started, cp_start_xmit and cp_open will run at same
time which will cause kernel BUG.

For example:
[#] for tx descriptor

At start:

[#][#][#]
num_queued=3

After cp_init_hw->cp_start_hw->netdev_reset_queue:

[#][#][#]
num_queued=0

When 8139cp starts to work then cp_tx will check
num_queued mismatchs the complete_bytes.

The patch will check IntrMask before check IntrStatus in cp_interrupt.
When 8139cp interrupt is disabled, just return.

Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/net/ethernet/realtek/8139cp.c |    5 +++++
 1 file changed, 5 insertions(+)

--- a/drivers/net/ethernet/realtek/8139cp.c
+++ b/drivers/net/ethernet/realtek/8139cp.c
@@ -571,6 +571,7 @@ static irqreturn_t cp_interrupt (int irq
 	struct cp_private *cp;
 	int handled = 0;
 	u16 status;
+	u16 mask;
 
 	if (unlikely(dev == NULL))
 		return IRQ_NONE;
@@ -578,6 +579,10 @@ static irqreturn_t cp_interrupt (int irq
 
 	spin_lock(&cp->lock);
 
+	mask = cpr16(IntrMask);
+	if (!mask)
+		goto out_unlock;
+
 	status = cpr16(IntrStatus);
 	if (!status || (status == 0xFFFF))
 		goto out_unlock;



  parent reply	other threads:[~2018-12-14 12:07 UTC|newest]

Thread overview: 106+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-14 11:59 [PATCH 4.14 00/89] 4.14.89-stable review Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 01/89] ipv4: ipv6: netfilter: Adjust the frag mem limit when truesize changes Greg Kroah-Hartman
2018-12-15 19:50   ` jwiesner
2018-12-14 11:59 ` [PATCH 4.14 02/89] ipv6: Check available headroom in ip6_xmit() even without options Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 03/89] neighbour: Avoid writing before skb->head in neigh_hh_output() Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 04/89] ipv6: sr: properly initialize flowi6 prior passing to ip6_route_output Greg Kroah-Hartman
2018-12-14 11:59 ` Greg Kroah-Hartman [this message]
2018-12-14 11:59 ` [PATCH 4.14 06/89] net/mlx4_core: Correctly set PFC param if global pause is turned off Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 07/89] net/mlx4_en: Change min MTU size to ETH_MIN_MTU Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 08/89] net: phy: dont allow __set_phy_supported to add unsupported modes Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 09/89] net: Prevent invalid access to skb->prev in __qdisc_drop_all Greg Kroah-Hartman
2018-12-14 15:52   ` Christoph Paasch
2018-12-14 15:54     ` Christoph Paasch
2018-12-14 16:28       ` Eric Dumazet
2018-12-14 19:05     ` David Miller
2018-12-14 19:11       ` Christoph Paasch
2018-12-14 11:59 ` [PATCH 4.14 10/89] rtnetlink: ndo_dflt_fdb_dump() only work for ARPHRD_ETHER devices Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 11/89] sctp: kfree_rcu asoc Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 12/89] tcp: Do not underestimate rwnd_limited Greg Kroah-Hartman
2018-12-14 14:03   ` Sudip Mukherjee
2018-12-14 14:26     ` Greg Kroah-Hartman
     [not found]       ` <CANn89iKnuwQAybYEtXwvRFfQ8Rp8VihO16bVucAjDn1GRBtv9w@mail.gmail.com>
2018-12-14 14:36         ` Greg Kroah-Hartman
2018-12-14 19:03           ` David Miller
2018-12-14 19:07             ` Greg KH
2018-12-14 18:58       ` David Miller
2018-12-14 11:59 ` [PATCH 4.14 13/89] tcp: fix NULL ref in tail loss probe Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 14/89] tun: forbid iface creation with rtnl ops Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 15/89] virtio-net: keep vnet header zeroed after processing XDP Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 16/89] ARM: OMAP2+: prm44xx: Fix section annotation on omap44xx_prm_enable_io_wakeup Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 17/89] ASoC: rsnd: fixup clock start checker Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 18/89] staging: rtl8723bs: Fix the return value in case of error in rtw_wx_read32() Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 19/89] ARM: dts: logicpd-somlv: Fix interrupt on mmc3_dat1 Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 20/89] ARM: OMAP1: ams-delta: Fix possible use of uninitialized field Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 21/89] sysv: return err instead of 0 in __sysv_write_inode Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 22/89] selftests: add script to stress-test nft packet path vs. control plane Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 23/89] netfilter: nf_tables: fix use-after-free when deleting compat expressions Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 24/89] hwmon (ina2xx) Fix NULL id pointer in probe() Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 25/89] ASoC: wm_adsp: Fix dma-unsafe read of scratch registers Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 26/89] s390/cpum_cf: Reject request for sampling in event initialization Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 27/89] hwmon: (ina2xx) Fix current value calculation Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 28/89] ASoC: omap-abe-twl6040: Fix missing audio card caused by deferred probing Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 29/89] ASoC: dapm: Recalculate audio map forcely when card instantiated Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 30/89] iio/hid-sensors: Fix IIO_CHAN_INFO_RAW returning wrong values for signed numbers Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 31/89] netfilter: xt_hashlimit: fix a possible memory leak in htable_create() Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 32/89] hwmon: (w83795) temp4_type has writable permission Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 33/89] perf tools: Restore proper cwd on return from mnt namespace Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 34/89] PCI: imx6: Fix link training status detection in link up check Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 35/89] objtool: Fix double-free in .cold detection error path Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 36/89] objtool: Fix segfault in .cold detection with -ffunction-sections Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 37/89] ARM: dts: at91: sama5d2: use the divided clock for SMC Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 38/89] Btrfs: send, fix infinite loop due to directory rename dependencies Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 39/89] RDMA/mlx5: Fix fence type for IB_WR_LOCAL_INV WR Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 40/89] RDMA/rdmavt: Fix rvt_create_ah function signature Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 41/89] ASoC: omap-mcbsp: Fix latency value calculation for pm_qos Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 42/89] ASoC: omap-mcpdm: Add pm_qos handling to avoid under/overruns with CPU_IDLE Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 43/89] ASoC: omap-dmic: Add pm_qos handling to avoid overruns " Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 44/89] exportfs: do not read dentry after free Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 45/89] bpf: fix check of allowed specifiers in bpf_trace_printk Greg Kroah-Hartman
2018-12-14 11:59 ` [PATCH 4.14 46/89] ipvs: call ip_vs_dst_notifier earlier than ipv6_dev_notf Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 47/89] USB: omap_udc: use devm_request_irq() Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 48/89] USB: omap_udc: fix crashes on probe error and module removal Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 49/89] USB: omap_udc: fix omap_udc_start() on 15xx machines Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 50/89] USB: omap_udc: fix USB gadget functionality on Palm Tungsten E Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 51/89] USB: omap_udc: fix rejection of out transfers when DMA is used Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 52/89] drm/meson: add support for 1080p25 mode Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 53/89] netfilter: ipv6: Preserve link scope traffic original oif Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 54/89] IB/mlx5: Fix page fault handling for MW Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 55/89] KVM: x86: fix empty-body warnings Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 56/89] x86/kvm/vmx: fix old-style function declaration Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 57/89] net: thunderx: fix NULL pointer dereference in nic_remove Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 58/89] usb: gadget: u_ether: fix unsafe list iteration Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 59/89] netfilter: nf_tables: deactivate expressions in rule replecement routine Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 60/89] cachefiles: Fix page leak in cachefiles_read_backing_file while vmscan is active Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 61/89] igb: fix uninitialized variables Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 62/89] ixgbe: recognize 1000BaseLX SFP modules as 1Gbps Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 63/89] net: hisilicon: remove unexpected free_netdev Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 64/89] drm/amdgpu: Add delay after enable RLC ucode Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 65/89] drm/ast: fixed reading monitor EDID not stable issue Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 66/89] xen: xlate_mmu: add missing header to fix W=1 warning Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 67/89] Revert "xen/balloon: Mark unallocated host memory as UNUSABLE" Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 68/89] pstore/ram: Correctly calculate usable PRZ bytes Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 69/89] fscache: fix race between enablement and dropping of object Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 70/89] fscache, cachefiles: remove redundant variable cache Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 71/89] nvme: flush namespace scanning work just before removing namespaces Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 72/89] ACPI/IORT: Fix iort_get_platform_device_domain() uninitialized pointer value Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 73/89] ocfs2: fix deadlock caused by ocfs2_defrag_extent() Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 74/89] mm/page_alloc.c: fix calculation of pgdat->nr_zones Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 75/89] hfs: do not free node before using Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 76/89] hfsplus: " Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 77/89] debugobjects: avoid recursive calls with kmemleak Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 78/89] ocfs2: fix potential use after free Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 79/89] Revert "printk: Never set console_may_schedule in console_trylock()" Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 80/89] printk: Add console owner and waiter logic to load balance console writes Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 81/89] printk: Hide console waiter logic into helpers Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 82/89] printk: Never set console_may_schedule in console_trylock() Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 83/89] printk: Wake klogd when passing console_lock owner Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 84/89] lib/rbtree-test: lower default params Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 85/89] flexfiles: enforce per-mirror stateid only for v4 DSes Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 86/89] staging: speakup: Replace strncpy with memcpy Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 87/89] ALSA: fireface: fix reference to wrong register for clock configuration Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 88/89] ALSA: hda/realtek - Fixed headphone issue for ALC700 Greg Kroah-Hartman
2018-12-14 12:00 ` [PATCH 4.14 89/89] IB/hfi1: Fix an out-of-bounds access in get_hw_stats Greg Kroah-Hartman
2018-12-14 17:33 ` [PATCH 4.14 00/89] 4.14.89-stable review kernelci.org bot
2018-12-14 20:12 ` shuah
2018-12-15  2:07 ` Guenter Roeck
2018-12-15 16:52 ` Dan Rue

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181214115729.931823768@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=suyj.fnst@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).