iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
From: Jerry Snitselaar <jsnitsel@redhat.com>
To: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org
Subject: Re: dmar pte read access not set error messages on hp dl388 gen8 systems
Date: Fri, 6 Dec 2019 00:24:53 -0700	[thread overview]
Message-ID: <20191206072453.5jjwrjedqbjimx45@cantor> (raw)
In-Reply-To: <d5fbedce-7a06-79b4-75fa-0aa3cc3ef73b@linux.intel.com>

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

On Fri Dec 06 19, Lu Baolu wrote:
[snip]
>
>Can you please try below change? Let's check whether the afending
>address has been mapped for device 01.00.2.
>
>$ git diff
>diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>index db7bfd4f2d20..d9daf66be849 100644
>--- a/drivers/iommu/iommu.c
>+++ b/drivers/iommu/iommu.c
>@@ -663,6 +663,8 @@ static int 
>iommu_group_create_direct_mappings(struct iommu_group *group,
>                        ret = iommu_map(domain, addr, addr, pg_size, 
>entry->prot);
>                        if (ret)
>                                goto out;
>+
>+                       dev_info(dev, "Setting identity map [0x%Lx - 
>0x%Lx] for group %d\n", addr, addr + pg_size, group->id);
>                }
>
>        }
>
>I am doubting that device 01.00.2 is not in the device scope of
>
>[    4.485108] DMAR: RMRR base: 0x000000bdf6f000 end: 0x000000bdf7efff
>
>By the way, does device 01.00.2 works well after binding the driver?
>

When I boot it with passthrough it doesn't get to a point where I can
login. I think the serial console on these systems is tied to the ilo,
so the conserver connection could be making things
worse. Unfortunately the system is remote. I should have more time now
to focus on debugging this.

Attaching console output for the above patch.

Regards,
Jerry

>Best regards,
>baolu
>

[-- Attachment #2: direct-mappings --]
[-- Type: text/plain, Size: 15657 bytes --]

[   21.833145] DMAR: dmar0: Using Queued invalidation
[   21.834960] DMAR: dmar1: Using Queued invalidation
[   21.837860] pci 0000:00:00.0: Adding to iommu group 0
[   21.839733] pci 0000:00:01.0: Adding to iommu group 1
[   21.841601] pci 0000:00:01.1: Adding to iommu group 2
[   21.843410] pci 0000:00:02.0: Adding to iommu group 3
[   21.845652] pci 0000:00:02.1: Adding to iommu group 4
[   21.847473] pci 0000:00:02.2: Adding to iommu group 5
[   21.849299] pci 0000:00:02.3: Adding to iommu group 6
[   21.851163] pci 0000:00:03.0: Adding to iommu group 7
[   21.852918] pci 0000:00:03.1: Adding to iommu group 8
[   21.854720] pci 0000:00:03.2: Adding to iommu group 9
[   21.856761] pci 0000:00:03.3: Adding to iommu group 10
[   21.858998] pci 0000:00:04.0: Adding to iommu group 11
[   21.860937] pci 0000:00:04.1: Adding to iommu group 11
[   21.863149] pci 0000:00:04.2: Adding to iommu group 11
[   21.864941] pci 0000:00:04.3: Adding to iommu group 11
[   21.866744] pci 0000:00:04.4: Adding to iommu group 11
[   21.868567] pci 0000:00:04.5: Adding to iommu group 11
[   21.870458] pci 0000:00:04.6: Adding to iommu group 11
[   21.872254] pci 0000:00:04.7: Adding to iommu group 11
[   21.874231] pci 0000:00:05.0: Adding to iommu group 12
[   21.876047] pci 0000:00:05.2: Adding to iommu group 12
[   21.877908] pci 0000:00:05.4: Adding to iommu group 12
[   21.879814] pci 0000:00:11.0: Adding to iommu group 13
[   21.881684] pci 0000:00:1a.0: Adding to iommu group 14
[   21.883516] pci 0000:00:1c.0: Adding to iommu group 15
[   21.885324] pci 0000:00:1c.7: Adding to iommu group 16
[   21.887116] pci 0000:00:1d.0: Adding to iommu group 17
[   21.888907] pci 0000:00:1e.0: Adding to iommu group 18
[   21.891913] pci 0000:00:1e.0: Using iommu dma mapping
[   21.894001] pci 0000:00:1f.0: Adding to iommu group 19
[   21.895979] pci 0000:00:1f.2: Adding to iommu group 19
[   21.898870] pci 0000:00:1f.2: DMAR: Setting identity map [0xe8000 - 0xe8fff]
[   21.901428] pci 0000:00:1f.2: DMAR: Setting identity map [0xf4000 - 0xf4fff]
[   21.903790] pci 0000:00:1f.2: DMAR: Setting identity map [0xbdf6e000 - 0xbdf6efff]
[   21.906347] pci 0000:00:1f.2: DMAR: Setting identity map [0xbdf6f000 - 0xbdf7efff]
[   21.908885] pci 0000:00:1f.2: DMAR: Setting identity map [0xbdf7f000 - 0xbdf82fff]
[   21.911620] pci 0000:00:1f.2: DMAR: Setting identity map [0xbdf83000 - 0xbdf84fff]
[   21.914141] pci 0000:00:1f.2: DMAR: Device uses a private dma domain.
[   21.916690] pci 0000:03:00.0: Adding to iommu group 20
[   21.919589] pci 0000:03:00.0: Using iommu dma mapping
[   21.921708] pci 0000:03:00.1: Setting identity map [0xe8000 - 0xe9000] for group 20
[   21.924309] pci 0000:03:00.1: Setting identity map [0xf4000 - 0xf5000] for group 20
[   21.926854] pci 0000:03:00.1: Setting identity map [0xbdf6e000 - 0xbdf6f000] for group 20
[   21.929560] pci 0000:03:00.1: Setting identity map [0xbdf6f000 - 0xbdf70000] for group 20
[   21.932366] pci 0000:03:00.1: Setting identity map [0xbdf00 - 0xbdf71000] for group 20
[   22.035075] pci 0000:03:00.1: Setting identity00 - 0xbdf72000] for group 20
[   22.437744] pci 0000:03:00.1: Setting identityp [0xbdf72000 - 0xbdf73000] for group 20
[   22.540574] pci 0000:03:00.1: Setting identity map [0xbdf73000 - 0xbdf74000] for group 20
[   22.543522] pci 0000:03:00.1: Setting identity map [0xbdf74000 - 0xbdf75000] for group 20
[   22.546270] pci 0000:03:00.1: Setting identity map [0xbdf75000 - 0xbdf76000] for group 20
[   22.548969] pci 0000:03:00.1: Setting identity map [0xbdf76000 - 0xbdf77000] for group 20
[   22.551801] pci 0000:03:00.1: Setting identity map [0xbdf77000 - 0xbdf78000] for group 20
[   22.554471] pci 0000:03:00.1: Setting identity map [0xbdf78000 - 0xbdf79000] for group 20
[   22.557096] pci 0000:03:00.1: Setting identity map [0xbdf79000 - 0xbdf7a000] for group 20
[   22.559795] pci 0000:03:00.1: Setting identity map [0xbdf7a000 - 0xbdf7b000] for group 20
[   22.562529] pci 0000:03:00.1: Setting identity map [0xbdf7b000 - 0xbdf7c000] for group 20
[   22.565240] pci 0000:03:00.1: Setting identity map [0xbdf7c000 - 0xbdf7d000] for group 20
[   22.567941] pci 0000:03:00.1: Setting identity map [0xbdf7d000 - 0xbdf7e000] for group 20
[   22.570725] pci 0000:03:00.1: Setting identity map [0xbdf7e000 - 0xbdf7f000] for group 20
[   22.573472] pci 0000:03:00.1: Setting identity map [0xbdf7f000 - 0xbdf80000] for group 20
[   22.576160] pci 0000:03:00.1: Setting identity map [0xbdf80000 - 0xbdf81000] for group 20
[   22.578883] pci Setting identity map [0xbdf81000 - 0xbdf82000] for group 20
[   22.981597] pci 0000:03:00.1: Setting identity map [0xbdf82000 - 0xbdf83000] for group 20
[   22.984383] pci 0000:03:00.1: Setting identity map [0xbdf83000 - 0xbdf84000] for group 20
[   22.987477] pci 0000:03:00.1: Setting identity map [0xbdf84000 - 0xbdf85000] for group 20
[   22.990217] pci 0000:03:00.1: Adding to iommu group 20
[   22.992042] pci 0000:03:00.2: Adding to iommu group 20
[   22.993837] pci 0000:03:00.3: Adding to iommu group 20
[   22.995635] pci 0000:02:00.0: Adding to iommu group 21
[   22.998067] pci 0000:02:00.0: Using iommu dma mapping
[   22.999885] pci 0000:04:00.0: Adding to iommu group 22
[   23.001683] pci 0000:05:02.0: Adding to iommu group 23
[   23.003458] pci 0000:05:04.0: Adding to iommu group 24
[   23.005189] pci 0000:06:00.0: Adding to iommu group 23
[   23.007671] pci 0000:06:00.0: DMAR: Setting identity map [0xe8000 - 0xe8fff]
[   23.010100] pci 0000:06:00.0: DMAR: Setting identity map [0xf4000 - 0xf4fff]
[   23.012512] pci 0000:06:00.0: DMAR: Setting identity map [0xbdf6e000 - 0xbdf6efff]
[   23.015026] pci 0000:06:00.0: DMAR: Setting identity map [0xbdf6f000 - 0xbdf7efff]
[   23.017526] pci 0000:06:00.0: DMAR:map [0xbdf7f000 - 0xbdf82fff]
[   23.519906] pci 0000:06:00.0: DMAR: Setting identity map [0xbdf83000 - 0xbdf84fff]
[   23.522565] pci 0000:06:00.0: DMAR: Device uses a private dma domain.
[   23.524746] pci 0000:06:00.1: Adding to iommu group 23
[   23.527624] pci 0000:06:00.1: DMAR: Setting identity map [0xe8000 - 0xe8fff]
[   23.530068] pci 0000:06:00.1: DMAR: Setting identity map [0xf4000 - 0xf4fff]
[   23.532469] pci 0000:06:00.1: DMAR: Setting identity map [0xbdf6e000 - 0xbdf6efff]
[   23.534990] pci 0000:06:00.1: DMAR: Setting identity map [0xbdf6f000 - 0xbdf7efff]
[   23.537505] pci 0000:06:00.1: DMAR: Setting identity map [0xbdf7f000 - 0xbdf82fff]
[   23.540005] pci 0000:06:00.1: DMAR: Setting identity map [0xbdf83000 - 0xbdf84fff]
[   23.542588] pci 0000:06:00.1: DMAR: Device uses a private dma domain.
[   23.544745] pci 0000:07:00.0: Adding to iommu group 24
[   23.547493] pci 0000:07:00.0: DMAR: Setting identity map [0xe8000 - 0xe8fff]
[   23.549882] pci 0000:07:00.0: DMAR: Setting identity map [0xf4000 - 0xf4fff]
[   23.552241] pci 0000:07:00.0: DMAR: Setting identity map [0xbdf6e000 - 0xbdf6efff]
[   23.554781] pci 0000:07:00.0: DMAR: Setting identity  0xbdf7efff]
[   24.057156] pci 0000:07:00.0: DMAR: Setting identity map [0xbdf7f000 - 0xbdf82fff]
[   24.059713] pci 0000:07:00.0: DMAR: Setting identity map [0xbdf83000 - 0xbdf84fff]
[   24.062355] pci 0000:07:00.0: DMAR: Device uses a private dma domain.
[   24.064527] pci 0000:07:00.1: Adding to iommu group 24
[   24.067080] pci 0000:07:00.1: DMAR: Setting identity map [0xe8000 - 0xe8fff]
[   24.069454] pci 0000:07:00.1: DMAR: Setting identity map [0xf4000 - 0xf4fff]
[   24.071886] pci 0000:07:00.1: DMAR: Setting identity map [0xbdf6e000 - 0xbdf6efff]
[   24.074449] pci 0000:07:00.1: DMAR: Setting identity map [0xbdf6f000 - 0xbdf7efff]
[   24.076935] pci 0000:07:00.1: DMAR: Setting identity map [0xbdf7f000 - 0xbdf82fff]
[   24.079460] pci 0000:07:00.1: DMAR: Setting identity map [0xbdf83000 - 0xbdf84fff]
[   24.082101] pci 0000:07:00.1: DMAR: Device uses a private dma domain.
[   24.084437] pci 0000:01:00.0: Adding to iommu group 25
[   24.087304] pci 0000:01:00.0: Using iommu dma mapping
[   24.089151] pci 0000:01:00.1: Adding to iommu group 25
[   24.090974] pci 0000:01:00.1: DMAR: Device uses a private identity domain.
[   24.093322] pci 0000:01:00.2: Adding to iommu group 25
[   24.095073] pci 0000:01:00iommu group 25
[   24.496813] pci 0000:01:00.4: DMAR: Device uses a private identity domain.
[   24.499147] pci 0000:20:00.0: Adding to iommu group 26
[   24.501024] pci 0000:20:01.0: Adding to iommu group 27
[   24.502809] pci 0000:20:01.1: Adding to iommu group 28
[   24.504565] pci 0000:20:02.0: Adding to iommu group 29
[   24.506303] pci 0000:20:02.1: Adding to iommu group 30
[   24.508073] pci 0000:20:02.2: Adding to iommu group 31
[   24.509843] pci 0000:20:02.3: Adding to iommu group 32
[   24.511726] pci 0000:20:03.0: Adding to iommu group 33
[   24.513520] pci 0000:20:03.1: Adding to iommu group 34
[   24.515275] pci 0000:20:03.2: Adding to iommu group 35
[   24.517049] pci 0000:20:03.3: Adding to iommu group 36
[   24.519033] pci 0000:20:04.0: Adding to iommu group 37
[   24.520935] pci 0000:20:04.1: Adding to iommu group 37
[   24.522714] pci 0000:20:04.2: Adding to iommu group 37
[   24.524498] pci 0000:20:04.3: Adding to iommu group 37
[   24.526242] pci 0000:20:04.4: Adding to iommu group 37
[   24.528055] pci 0000:20:04.5: Adding to iommu group 37
[   24.529829] pci 0000:20:04.6: Adding to iommu group 37
[   24.531694] pci 0000:20:04.7: Adding to iommu group 37
[   24.533545] pci 0000:20:05.0: Adding to iommu group 38
[   24.535316] pci 0000:20:05.2: Adding to iommu group 38
[   24.537106] pci 0000:20:05.4: Adding to iommu group 38
[   24.538893] pci 0000:24:00.0: Adding to iommu group 39
[   24.541532] pci 0000:24:00.0: Using iommu dma mapping
[   24.543317] pci 0000:24:00.1: Adding to iommu group 40
[   24.545811] pci 0000:24:00.1: Using iommu dma mapping
[   24.547663] pci 0000:24:00.2: Adding to iommu group 41
[   24.550232] pci 0000:24:00.2: Using iomm  24.962894] pci 0000:24:00.3: Adding to iommu group 42
[   25.054536] pci 0000:24:00.3: Using iommu dma mapping
[   25.056415] pci 0000:1f:08.0: Adding to iommu group 43
[   25.058192] pci 0000:1f:08.3: Adding to iommu group 43
[   25.060061] pci 0000:1f:08.4: Adding to iommu group 43
[   25.061907] pci 0000:1f:09.0: Adding to iommu group 44
[   25.063712] pci 0000:1f:09.3: Adding to iommu group 44
[   25.065540] pci 0000:1f:09.4: Adding to iommu group 44
[   25.067393] pci 0000:1f:0a.0: Adding to iommu group 45
[   25.069174] pci 0000:1f:0a.1: Adding to iommu group 45
[   25.071034] pci 0000:1f:0a.2: Adding to iommu group 45
[   25.072825] pci 0000:1f:0a.3: Adding to iommu group 45
[   25.074653] pci 0000:1f:0b.0: Adding to iommu group 46
[   25.076441] pci 0000:1f:0b.3: Adding to iommu group 46
[   25.078287] pci 0000:1f:0c.0: Adding to iommu group 47
[   25.080119] pci 0000:1f:0c.1: Adding to iommu group 47
[   25.081922] pci 0000:1f:0c.2: Adding to iommu group 47
[   25.083703] pci 0000:1f:0c.6: Adding to iommu group 47
[   25.085509] pci 0000:1f:0c.7: Adding to iommu group 47
[   25.087669] pci 0000:1f:0d.0: Adding to iommu group 48
[   25.089506] pci 0000:1f:0d.1: Adding to iommu group 48
[   25.091349] pci 0000:1f:0d.2: Adding to[   25.558963] pci 0000:1f:0d.6: Adding to iommu group 48
[   25.595177] pci 0000:1f:0e.0: Adding to iommu group 49
[   25.597051] pci 0000:1f:0e.1: Adding to iommu group 49
[   25.599218] pci 0000:1f:0f.0: Adding to iommu group 50
[   25.601170] pci 0000:1f:0f.1: Adding to iommu group 50
[   25.603056] pci 0000:1f:0f.2: Adding to iommu group 50
[   25.604894] pci 0000:1f:0f.3: Adding to iommu group 50
[   25.606751] pci 0000:1f:0f.4: Adding to iommu group 50
[   25.608594] pci 0000:1f:0f.5: Adding to iommu group 50
[   25.610436] pci 0000:1f:0f.6: Adding to iommu group 50
[   25.613043] pci 0000:1f:10.0: Adding to iommu group 51
[   25.614906] pci 0000:1f:10.1: Adding to iommu group 51
[   25.616759] pci 0000:1f:10.2: Adding to iommu group 51
[   25.618603] pci 0000:1f:10.3: Adding to iommu group 51
[   25.620458] pci 0000:1f:10.4: Adding to iommu group 51
[   25.622266] pci 0000:1f:10.5: Adding to iommu group 51
[   25.624154] pci 0000:1f:10.6: Adding to iommu group 51
[   25.626088] pci 0000:1f:10.7: Adding to iommu group 51
[   25.627980] pci 0000:1f:11.0: Adding to iommu group 52
[   25.630043] pci 0000:1f:13.0: Adding to iommu  25.931279] pci 0000:1f:13.1: Adding to iommu group 53
[   26.033506] pci 0000:1f:13.4: Adding to iommu group 53
[   26.035287] pci 0000:1f:13.5: Adding to iommu group 53
[   26.037070] pci 0000:1f:13.6: Adding to iommu group 53
[   26.038925] pci 0000:3f:08.0: Adding to iommu group 54
[   26.040799] pci 0000:3f:08.3: Adding to iommu group 54
[   26.042586] pci 0000:3f:08.4: Adding to iommu group 54
[   26.044397] pci 0000:3f:09.0: Adding to iommu group 55
[   26.046198] pci 0000:3f:09.3: Adding to iommu group 55
[   26.047991] pci 0000:3f:09.4: Adding to iommu group 55
[   26.049854] pci 0000:3f:0a.0: Adding to iommu group 56
[   26.051740] pci 0000:3f:0a.1: Adding to iommu group 56
[   26.053526] pci 0000:3f:0a.2: Adding to iommu group 56
[   26.055285] pci 0000:3f:0a.3: Adding to iommu group 56
[   26.057294] pci 0000:3f:0b.0: Adding to iommu group 57
[   26.059107] pci 0000:3f:0b.3: Adding to iommu group 57
[   26.061092] pci 0000:3f:0c.0: Adding to iommu group 58
[   26.062881] pci 0000:3f:0c.1: Adding to iommu group 58
[   26.064666] pci 0000:3f:0c.2: Adding to iommu group 58
[   26.066430] pci 0000:3f:0c.6: Adding to iommu group 58
[   26.068214] pci 0g to iommu group 58
[   26.570012] pci 0000:3f:0d.0: Adding to iommu group 59
[   26.571914] pci 0000:3f:0d.1: Adding to iommu group 59
[   26.573717] pci 0000:3f:0d.2: Adding to iommu group 59
[   26.575480] pci 0000:3f:0d.6: Adding to iommu group 59
[   26.577292] pci 0000:3f:0e.0: Adding to iommu group 60
[   26.579095] pci 0000:3f:0e.1: Adding to iommu group 60
[   26.581097] pci 0000:3f:0f.0: Adding to iommu group 61
[   26.582907] pci 0000:3f:0f.1: Adding to iommu group 61
[   26.584692] pci 0000:3f:0f.2: Adding to iommu group 61
[   26.586468] pci 0000:3f:0f.3: Adding to iommu group 61
[   26.588240] pci 0000:3f:0f.4: Adding to iommu group 61
[   26.590054] pci 0000:3f:0f.5: Adding to iommu group 61
[   26.591912] pci 0000:3f:0f.6: Adding to iommu group 61
[   26.593870] pci 0000:3f:10.0: Adding to iommu group 62
[   26.595927] pci 0000:3f:10.1: Adding to iommu group 62
[   26.597721] pci 0000:3f:10.2: Adding to iommu group 62
[   26.599485] pci 0000:3f:10.3: Adding to iommu group 62
[   26.601364] pci 0000:3f:10.4: Adding to iommu group 62
[   26.603161] pci 0000:3f:10.5: Adding to iommu group 62
[   26.604967] pci 0000:3f:10.6: Adding to iommu group 62
[   26.606804] pci 0000:3f:10.7: Adding to iommu group 62
[   26.608566] pci 0000:3f:11.0: Adding to iommu group 63
[   26.610464] pci 0000:3f:13.0: Adding to iommu group 64
[   26.612655] pci 0000:3f:13.1: Adding to iommu group 64
[   26.614433] pci 0000:3f:13.4: Adding to iommu group 64
[   26.616233] pci 0000:3f:13.5: Adding to iommu group 64
[   26.618007] Adding to iommu group 64
[   27.119637] DMAR: Intel(R) Virtualization Technology for Directed I/O
[   27.899649] DMAR: DRHD: handling fault status reg 2
[   27.899649] DMAR: [DMA Read] Request device [02:00.0] PASID ffffffff fault addr bdf83000 [fault reason 06] PTE Read access is not set
[   27.899649] DMAR: [DMA Read] Request device [01:00.2] PASID ffffffff fault addr f4000 [fault reason 06] PTE Read access is not set

[-- Attachment #3: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  reply	other threads:[~2019-12-06  7:25 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-02  6:34 dmar pte read access not set error messages on hp dl388 gen8 systems Jerry Snitselaar
2019-12-02  6:41 ` Lu Baolu
2019-12-02  7:14   ` Jerry Snitselaar
2019-12-02 16:13     ` Jerry Snitselaar
2019-12-03  1:59       ` Lu Baolu
2019-12-03  9:56         ` Jerry Snitselaar
2019-12-04  0:17           ` Lu Baolu
2019-12-04 20:53             ` Jerry Snitselaar
2019-12-05  1:39               ` Lu Baolu
2019-12-05  2:25                 ` Jerry Snitselaar
2019-12-05  2:44                   ` Lu Baolu
2019-12-05  2:53                     ` Jerry Snitselaar
2019-12-06  1:52                       ` Lu Baolu
2019-12-06  7:24                         ` Jerry Snitselaar [this message]
2019-12-07  1:53                           ` Lu Baolu
2019-12-07  2:29                             ` Jerry Snitselaar
2019-12-07  2:41                               ` Jerry Snitselaar
2019-12-08  6:26                                 ` Lu Baolu
2019-12-10  0:52                                   ` Jerry Snitselaar
2019-12-10  1:29                                     ` Lu Baolu
2019-12-10  3:47                                       ` Jerry Snitselaar
2019-12-10  5:03                                         ` Lu Baolu
2019-12-10  5:18                                         ` Jerry Snitselaar
2019-12-10  5:43                                           ` Lu Baolu
2019-12-10 22:12                                             ` Jerry Snitselaar
2019-12-10  5:43                                           ` Jerry Snitselaar
2019-12-10  6:16                                             ` Jerry Snitselaar
2019-12-10  6:26                                               ` Lu Baolu
2019-12-10  7:44                                                 ` Jerry Snitselaar
2019-12-08  6:04                           ` Lu Baolu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191206072453.5jjwrjedqbjimx45@cantor \
    --to=jsnitsel@redhat.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=iommu@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).