All of lore.kernel.org
 help / color / mirror / Atom feed

diff for duplicates of <3E85B4D4-9EBC-4299-8209-2D8740947764@raithlin.com>

diff --git a/a/1.txt b/N1/1.txt
index e657e2c..08d61fb 100644
--- a/a/1.txt
+++ b/N1/1.txt
@@ -22,7 +22,4 @@ As long as legA, legB and the RC are all connected to the same switch then order
 
 I hope this helps!
 
-Stephen
-
-
-N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dʇڙë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš
\ No newline at end of file
+Stephen
\ No newline at end of file
diff --git a/a/content_digest b/N1/content_digest
index 3444a70..7c663f0 100644
--- a/a/content_digest
+++ b/N1/content_digest
@@ -14,10 +14,7 @@
   "ref\0004df229d8-8124-664a-9bc4-6401bc034be1\@grimberg.me\0"
 ]
 [
-  "ref\0004df229d8-8124-664a-9bc4-6401bc034be1-NQWnxTmZq1alnMjI0IkVqw\@public.gmane.org\0"
-]
-[
-  "From\0Stephen Bates <sbates-pv7U853sEMVWk0Htik3J/w\@public.gmane.org>\0"
+  "From\0Stephen Bates <sbates\@raithlin.com>\0"
 ]
 [
   "Subject\0Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem\0"
@@ -26,25 +23,25 @@
   "Date\0Fri, 7 Apr 2017 11:19:30 +0000\0"
 ]
 [
-  "To\0Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw\@public.gmane.org>",
-  " Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/\@public.gmane.org>\0"
+  "To\0Sagi Grimberg <sagi\@grimberg.me>",
+  " Jason Gunthorpe <jgunthorpe\@obsidianresearch.com>\0"
 ]
 [
-  "Cc\0Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w\@public.gmane.org>",
-  " Christoph Hellwig <hch-jcswGhMUV9g\@public.gmane.org>",
-  " James E.J. Bottomley <jejb-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8\@public.gmane.org>",
-  " Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA\@public.gmane.org>",
-  " Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw\@public.gmane.org>",
-  " Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW\@public.gmane.org>",
-  " Max Gurtovoy <maxg-VPRAkNaXOzVWk0Htik3J/w\@public.gmane.org>",
-  " Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w\@public.gmane.org>",
-  " Keith Busch <keith.busch-ral2JQCrhuEAvxtiuMwx3w\@public.gmane.org>",
-  " linux-pci-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-pci-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-scsi-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-scsi-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r\@public.gmane.org <linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r\@public.gmane.org>",
-  " linux-rdma-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-rdma-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-nvdimm-y27Ovi1pjclAfugRpC6u6w\@public.gmane.org <linux-nvdimm-y27Ovi1pjclAfugRpC6u6w\@public.gmane.org>",
-  " linux-kernel-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-kernel-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>\0"
+  "Cc\0Logan Gunthorpe <logang\@deltatee.com>",
+  " Christoph Hellwig <hch\@lst.de>",
+  " James E.J. Bottomley <jejb\@linux.vnet.ibm.com>",
+  " Martin K. Petersen <martin.petersen\@oracle.com>",
+  " Jens Axboe <axboe\@kernel.dk>",
+  " Steve Wise <swise\@opengridcomputing.com>",
+  " Max Gurtovoy <maxg\@mellanox.com>",
+  " Dan Williams <dan.j.williams\@intel.com>",
+  " Keith Busch <keith.busch\@intel.com>",
+  " linux-pci\@vger.kernel.org <linux-pci\@vger.kernel.org>",
+  " linux-scsi\@vger.kernel.org <linux-scsi\@vger.kernel.org>",
+  " linux-nvme\@lists.infradead.org <linux-nvme\@lists.infradead.org>",
+  " linux-rdma\@vger.kernel.org <linux-rdma\@vger.kernel.org>",
+  " linux-nvdimm\@ml01.01.org <linux-nvdimm\@ml01.01.org>",
+  " linux-kernel\@vger.kernel.org <linux-kernel\@vger.kernel.org>\0"
 ]
 [
   "\0000:1\0"
@@ -77,10 +74,7 @@
   "\n",
   "I hope this helps!\n",
   "\n",
-  "Stephen\n",
-  "\n",
-  "\n",
-  "N\302\213\302\247\302\262\303\246\303\254r\302\270\302\233y\303\272\303\250\302\232\303\230b\302\262X\302\254\302\266\303\207\302\247v\303\230^\302\226)\303\236\302\272{.n\303\207+\302\211\302\267\302\245\302\212{\302\261\302\255\303\231\302\232\302\212{ay\302\272\35\303\212\302\207\303\232\302\231\303\253,j\a\302\255\302\242f\302\243\302\242\302\267h\302\232\302\213\302\273\303\266\303\254\27/oSc\302\276\302\231\303\232\302\2639\302\230u\303\200\302\246\303\246\303\245\302\211\303\210&jw\302\250\302\256\3(\302\255\303\251\302\232\302\216\302\212\303\235\302\242j\"\302\235\303\272\32\302\266\em\302\247\303\277\303\257\302\201\303\252\303\244z\302\271\303\236\302\226\302\212\303\240\303\276f\302\243\302\242\302\267h\302\232\302\210\302\247~\302\210m\302\232"
+  "Stephen"
 ]
 
-b078d2a1d9ef181930008e34eaa6b104a8f6df924271bdf0297effbcd3f4a52f
+7175325e9a662d60c52b49e73e128decd7a8f021cb99fc20d9cab370890b467c

diff --git a/a/1.txt b/N2/1.txt
index e657e2c..233d833 100644
--- a/a/1.txt
+++ b/N2/1.txt
@@ -1,28 +1,32 @@
-On 2017-04-06, 6:33 AM, "Sagi Grimberg" <sagi@grimberg.me> wrote:
-
-> Say it's connected via 2 legs, the bar is accessed from leg A and the
-> data from the disk comes via leg B. In this case, the data is heading
-> towards the p2p device via leg B (might be congested), the completion
-> goes directly to the RC, and then the host issues a read from the
-> bar via leg A. I don't understand what can guarantee ordering here.
-
-> Stephen told me that this still guarantees ordering, but I honestly
-> can't understand how, perhaps someone can explain to me in a simple
-> way that I can understand.
-
-Sagi
-
-As long as legA, legB and the RC are all connected to the same switch then ordering will be preserved (I think many other topologies also work). Here is how it would work for the problem case you are concerned about (which is a read from the NVMe drive).
-
-1. Disk device DMAs out the data to the p2pmem device via a string of PCIe MemWr TLPs.
-2. Disk device writes to the completion queue (in system memory) via a MemWr TLP.
-3. The last of the MemWrs from step 1 might have got stalled in the PCIe switch due to congestion but if so they are stalled in the egress path of the switch for the p2pmem port.
-4. The RC determines the IO is complete when the TLP associated with step 2 updates the memory associated with the CQ. It issues some operation to read the p2pmem.
-5. Regardless of whether the MemRd TLP comes from the RC or another device connected to the switch it is queued in the egress queue for the p2pmem FIO behind the last DMA TLP (from step 1). PCIe ordering ensures that this MemRd cannot overtake the MemWr (Reads can never pass writes). Therefore the MemRd can never get to the p2pmem device until after the last DMA MemWr has.
-
-I hope this helps!
-
-Stephen
-
-
-N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dʇڙë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš
\ No newline at end of file
+T24gMjAxNy0wNC0wNiwgNjozMyBBTSwgIlNhZ2kgR3JpbWJlcmciIDxzYWdpQGdyaW1iZXJnLm1l
+PiB3cm90ZToNCg0KPiBTYXkgaXQncyBjb25uZWN0ZWQgdmlhIDIgbGVncywgdGhlIGJhciBpcyBh
+Y2Nlc3NlZCBmcm9tIGxlZyBBIGFuZCB0aGUNCj4gZGF0YSBmcm9tIHRoZSBkaXNrIGNvbWVzIHZp
+YSBsZWcgQi4gSW4gdGhpcyBjYXNlLCB0aGUgZGF0YSBpcyBoZWFkaW5nDQo+IHRvd2FyZHMgdGhl
+IHAycCBkZXZpY2UgdmlhIGxlZyBCIChtaWdodCBiZSBjb25nZXN0ZWQpLCB0aGUgY29tcGxldGlv
+bg0KPiBnb2VzIGRpcmVjdGx5IHRvIHRoZSBSQywgYW5kIHRoZW4gdGhlIGhvc3QgaXNzdWVzIGEg
+cmVhZCBmcm9tIHRoZQ0KPiBiYXIgdmlhIGxlZyBBLiBJIGRvbid0IHVuZGVyc3RhbmQgd2hhdCBj
+YW4gZ3VhcmFudGVlIG9yZGVyaW5nIGhlcmUuDQoNCj4gU3RlcGhlbiB0b2xkIG1lIHRoYXQgdGhp
+cyBzdGlsbCBndWFyYW50ZWVzIG9yZGVyaW5nLCBidXQgSSBob25lc3RseQ0KPiBjYW4ndCB1bmRl
+cnN0YW5kIGhvdywgcGVyaGFwcyBzb21lb25lIGNhbiBleHBsYWluIHRvIG1lIGluIGEgc2ltcGxl
+DQo+IHdheSB0aGF0IEkgY2FuIHVuZGVyc3RhbmQuDQoNClNhZ2kNCg0KQXMgbG9uZyBhcyBsZWdB
+LCBsZWdCIGFuZCB0aGUgUkMgYXJlIGFsbCBjb25uZWN0ZWQgdG8gdGhlIHNhbWUgc3dpdGNoIHRo
+ZW4gb3JkZXJpbmcgd2lsbCBiZSBwcmVzZXJ2ZWQgKEkgdGhpbmsgbWFueSBvdGhlciB0b3BvbG9n
+aWVzIGFsc28gd29yaykuIEhlcmUgaXMgaG93IGl0IHdvdWxkIHdvcmsgZm9yIHRoZSBwcm9ibGVt
+IGNhc2UgeW91IGFyZSBjb25jZXJuZWQgYWJvdXQgKHdoaWNoIGlzIGEgcmVhZCBmcm9tIHRoZSBO
+Vk1lIGRyaXZlKS4NCg0KMS4gRGlzayBkZXZpY2UgRE1BcyBvdXQgdGhlIGRhdGEgdG8gdGhlIHAy
+cG1lbSBkZXZpY2UgdmlhIGEgc3RyaW5nIG9mIFBDSWUgTWVtV3IgVExQcy4NCjIuIERpc2sgZGV2
+aWNlIHdyaXRlcyB0byB0aGUgY29tcGxldGlvbiBxdWV1ZSAoaW4gc3lzdGVtIG1lbW9yeSkgdmlh
+IGEgTWVtV3IgVExQLg0KMy4gVGhlIGxhc3Qgb2YgdGhlIE1lbVdycyBmcm9tIHN0ZXAgMSBtaWdo
+dCBoYXZlIGdvdCBzdGFsbGVkIGluIHRoZSBQQ0llIHN3aXRjaCBkdWUgdG8gY29uZ2VzdGlvbiBi
+dXQgaWYgc28gdGhleSBhcmUgc3RhbGxlZCBpbiB0aGUgZWdyZXNzIHBhdGggb2YgdGhlIHN3aXRj
+aCBmb3IgdGhlIHAycG1lbSBwb3J0Lg0KNC4gVGhlIFJDIGRldGVybWluZXMgdGhlIElPIGlzIGNv
+bXBsZXRlIHdoZW4gdGhlIFRMUCBhc3NvY2lhdGVkIHdpdGggc3RlcCAyIHVwZGF0ZXMgdGhlIG1l
+bW9yeSBhc3NvY2lhdGVkIHdpdGggdGhlIENRLiBJdCBpc3N1ZXMgc29tZSBvcGVyYXRpb24gdG8g
+cmVhZCB0aGUgcDJwbWVtLg0KNS4gUmVnYXJkbGVzcyBvZiB3aGV0aGVyIHRoZSBNZW1SZCBUTFAg
+Y29tZXMgZnJvbSB0aGUgUkMgb3IgYW5vdGhlciBkZXZpY2UgY29ubmVjdGVkIHRvIHRoZSBzd2l0
+Y2ggaXQgaXMgcXVldWVkIGluIHRoZSBlZ3Jlc3MgcXVldWUgZm9yIHRoZSBwMnBtZW0gRklPIGJl
+aGluZCB0aGUgbGFzdCBETUEgVExQIChmcm9tIHN0ZXAgMSkuIFBDSWUgb3JkZXJpbmcgZW5zdXJl
+cyB0aGF0IHRoaXMgTWVtUmQgY2Fubm90IG92ZXJ0YWtlIHRoZSBNZW1XciAoUmVhZHMgY2FuIG5l
+dmVyIHBhc3Mgd3JpdGVzKS4gVGhlcmVmb3JlIHRoZSBNZW1SZCBjYW4gbmV2ZXIgZ2V0IHRvIHRo
+ZSBwMnBtZW0gZGV2aWNlIHVudGlsIGFmdGVyIHRoZSBsYXN0IERNQSBNZW1XciBoYXMuDQoNCkkg
+aG9wZSB0aGlzIGhlbHBzIQ0KDQpTdGVwaGVuDQoNCg0K
\ No newline at end of file
diff --git a/a/content_digest b/N2/content_digest
index 3444a70..5ed459c 100644
--- a/a/content_digest
+++ b/N2/content_digest
@@ -14,10 +14,7 @@
   "ref\0004df229d8-8124-664a-9bc4-6401bc034be1\@grimberg.me\0"
 ]
 [
-  "ref\0004df229d8-8124-664a-9bc4-6401bc034be1-NQWnxTmZq1alnMjI0IkVqw\@public.gmane.org\0"
-]
-[
-  "From\0Stephen Bates <sbates-pv7U853sEMVWk0Htik3J/w\@public.gmane.org>\0"
+  "From\0Stephen Bates <sbates\@raithlin.com>\0"
 ]
 [
   "Subject\0Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem\0"
@@ -26,25 +23,25 @@
   "Date\0Fri, 7 Apr 2017 11:19:30 +0000\0"
 ]
 [
-  "To\0Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw\@public.gmane.org>",
-  " Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/\@public.gmane.org>\0"
+  "To\0Sagi Grimberg <sagi\@grimberg.me>",
+  " Jason Gunthorpe <jgunthorpe\@obsidianresearch.com>\0"
 ]
 [
-  "Cc\0Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w\@public.gmane.org>",
-  " Christoph Hellwig <hch-jcswGhMUV9g\@public.gmane.org>",
-  " James E.J. Bottomley <jejb-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8\@public.gmane.org>",
-  " Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA\@public.gmane.org>",
-  " Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw\@public.gmane.org>",
-  " Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW\@public.gmane.org>",
-  " Max Gurtovoy <maxg-VPRAkNaXOzVWk0Htik3J/w\@public.gmane.org>",
-  " Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w\@public.gmane.org>",
-  " Keith Busch <keith.busch-ral2JQCrhuEAvxtiuMwx3w\@public.gmane.org>",
-  " linux-pci-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-pci-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-scsi-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-scsi-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r\@public.gmane.org <linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r\@public.gmane.org>",
-  " linux-rdma-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-rdma-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-nvdimm-y27Ovi1pjclAfugRpC6u6w\@public.gmane.org <linux-nvdimm-y27Ovi1pjclAfugRpC6u6w\@public.gmane.org>",
-  " linux-kernel-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-kernel-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>\0"
+  "Cc\0Logan Gunthorpe <logang\@deltatee.com>",
+  " Christoph Hellwig <hch\@lst.de>",
+  " James E.J. Bottomley <jejb\@linux.vnet.ibm.com>",
+  " Martin K. Petersen <martin.petersen\@oracle.com>",
+  " Jens Axboe <axboe\@kernel.dk>",
+  " Steve Wise <swise\@opengridcomputing.com>",
+  " Max Gurtovoy <maxg\@mellanox.com>",
+  " Dan Williams <dan.j.williams\@intel.com>",
+  " Keith Busch <keith.busch\@intel.com>",
+  " linux-pci\@vger.kernel.org <linux-pci\@vger.kernel.org>",
+  " linux-scsi\@vger.kernel.org <linux-scsi\@vger.kernel.org>",
+  " linux-nvme\@lists.infradead.org <linux-nvme\@lists.infradead.org>",
+  " linux-rdma\@vger.kernel.org <linux-rdma\@vger.kernel.org>",
+  " linux-nvdimm\@ml01.01.org <linux-nvdimm\@ml01.01.org>",
+  " linux-kernel\@vger.kernel.org <linux-kernel\@vger.kernel.org>\0"
 ]
 [
   "\0000:1\0"
@@ -53,34 +50,38 @@
   "b\0"
 ]
 [
-  "On 2017-04-06, 6:33 AM, \"Sagi Grimberg\" <sagi\@grimberg.me> wrote:\n",
-  "\n",
-  "> Say it's connected via 2 legs, the bar is accessed from leg A and the\n",
-  "> data from the disk comes via leg B. In this case, the data is heading\n",
-  "> towards the p2p device via leg B (might be congested), the completion\n",
-  "> goes directly to the RC, and then the host issues a read from the\n",
-  "> bar via leg A. I don't understand what can guarantee ordering here.\n",
-  "\n",
-  "> Stephen told me that this still guarantees ordering, but I honestly\n",
-  "> can't understand how, perhaps someone can explain to me in a simple\n",
-  "> way that I can understand.\n",
-  "\n",
-  "Sagi\n",
-  "\n",
-  "As long as legA, legB and the RC are all connected to the same switch then ordering will be preserved (I think many other topologies also work). Here is how it would work for the problem case you are concerned about (which is a read from the NVMe drive).\n",
-  "\n",
-  "1. Disk device DMAs out the data to the p2pmem device via a string of PCIe MemWr TLPs.\n",
-  "2. Disk device writes to the completion queue (in system memory) via a MemWr TLP.\n",
-  "3. The last of the MemWrs from step 1 might have got stalled in the PCIe switch due to congestion but if so they are stalled in the egress path of the switch for the p2pmem port.\n",
-  "4. The RC determines the IO is complete when the TLP associated with step 2 updates the memory associated with the CQ. It issues some operation to read the p2pmem.\n",
-  "5. Regardless of whether the MemRd TLP comes from the RC or another device connected to the switch it is queued in the egress queue for the p2pmem FIO behind the last DMA TLP (from step 1). PCIe ordering ensures that this MemRd cannot overtake the MemWr (Reads can never pass writes). Therefore the MemRd can never get to the p2pmem device until after the last DMA MemWr has.\n",
-  "\n",
-  "I hope this helps!\n",
-  "\n",
-  "Stephen\n",
-  "\n",
-  "\n",
-  "N\302\213\302\247\302\262\303\246\303\254r\302\270\302\233y\303\272\303\250\302\232\303\230b\302\262X\302\254\302\266\303\207\302\247v\303\230^\302\226)\303\236\302\272{.n\303\207+\302\211\302\267\302\245\302\212{\302\261\302\255\303\231\302\232\302\212{ay\302\272\35\303\212\302\207\303\232\302\231\303\253,j\a\302\255\302\242f\302\243\302\242\302\267h\302\232\302\213\302\273\303\266\303\254\27/oSc\302\276\302\231\303\232\302\2639\302\230u\303\200\302\246\303\246\303\245\302\211\303\210&jw\302\250\302\256\3(\302\255\303\251\302\232\302\216\302\212\303\235\302\242j\"\302\235\303\272\32\302\266\em\302\247\303\277\303\257\302\201\303\252\303\244z\302\271\303\236\302\226\302\212\303\240\303\276f\302\243\302\242\302\267h\302\232\302\210\302\247~\302\210m\302\232"
+  "T24gMjAxNy0wNC0wNiwgNjozMyBBTSwgIlNhZ2kgR3JpbWJlcmciIDxzYWdpQGdyaW1iZXJnLm1l\n",
+  "PiB3cm90ZToNCg0KPiBTYXkgaXQncyBjb25uZWN0ZWQgdmlhIDIgbGVncywgdGhlIGJhciBpcyBh\n",
+  "Y2Nlc3NlZCBmcm9tIGxlZyBBIGFuZCB0aGUNCj4gZGF0YSBmcm9tIHRoZSBkaXNrIGNvbWVzIHZp\n",
+  "YSBsZWcgQi4gSW4gdGhpcyBjYXNlLCB0aGUgZGF0YSBpcyBoZWFkaW5nDQo+IHRvd2FyZHMgdGhl\n",
+  "IHAycCBkZXZpY2UgdmlhIGxlZyBCIChtaWdodCBiZSBjb25nZXN0ZWQpLCB0aGUgY29tcGxldGlv\n",
+  "bg0KPiBnb2VzIGRpcmVjdGx5IHRvIHRoZSBSQywgYW5kIHRoZW4gdGhlIGhvc3QgaXNzdWVzIGEg\n",
+  "cmVhZCBmcm9tIHRoZQ0KPiBiYXIgdmlhIGxlZyBBLiBJIGRvbid0IHVuZGVyc3RhbmQgd2hhdCBj\n",
+  "YW4gZ3VhcmFudGVlIG9yZGVyaW5nIGhlcmUuDQoNCj4gU3RlcGhlbiB0b2xkIG1lIHRoYXQgdGhp\n",
+  "cyBzdGlsbCBndWFyYW50ZWVzIG9yZGVyaW5nLCBidXQgSSBob25lc3RseQ0KPiBjYW4ndCB1bmRl\n",
+  "cnN0YW5kIGhvdywgcGVyaGFwcyBzb21lb25lIGNhbiBleHBsYWluIHRvIG1lIGluIGEgc2ltcGxl\n",
+  "DQo+IHdheSB0aGF0IEkgY2FuIHVuZGVyc3RhbmQuDQoNClNhZ2kNCg0KQXMgbG9uZyBhcyBsZWdB\n",
+  "LCBsZWdCIGFuZCB0aGUgUkMgYXJlIGFsbCBjb25uZWN0ZWQgdG8gdGhlIHNhbWUgc3dpdGNoIHRo\n",
+  "ZW4gb3JkZXJpbmcgd2lsbCBiZSBwcmVzZXJ2ZWQgKEkgdGhpbmsgbWFueSBvdGhlciB0b3BvbG9n\n",
+  "aWVzIGFsc28gd29yaykuIEhlcmUgaXMgaG93IGl0IHdvdWxkIHdvcmsgZm9yIHRoZSBwcm9ibGVt\n",
+  "IGNhc2UgeW91IGFyZSBjb25jZXJuZWQgYWJvdXQgKHdoaWNoIGlzIGEgcmVhZCBmcm9tIHRoZSBO\n",
+  "Vk1lIGRyaXZlKS4NCg0KMS4gRGlzayBkZXZpY2UgRE1BcyBvdXQgdGhlIGRhdGEgdG8gdGhlIHAy\n",
+  "cG1lbSBkZXZpY2UgdmlhIGEgc3RyaW5nIG9mIFBDSWUgTWVtV3IgVExQcy4NCjIuIERpc2sgZGV2\n",
+  "aWNlIHdyaXRlcyB0byB0aGUgY29tcGxldGlvbiBxdWV1ZSAoaW4gc3lzdGVtIG1lbW9yeSkgdmlh\n",
+  "IGEgTWVtV3IgVExQLg0KMy4gVGhlIGxhc3Qgb2YgdGhlIE1lbVdycyBmcm9tIHN0ZXAgMSBtaWdo\n",
+  "dCBoYXZlIGdvdCBzdGFsbGVkIGluIHRoZSBQQ0llIHN3aXRjaCBkdWUgdG8gY29uZ2VzdGlvbiBi\n",
+  "dXQgaWYgc28gdGhleSBhcmUgc3RhbGxlZCBpbiB0aGUgZWdyZXNzIHBhdGggb2YgdGhlIHN3aXRj\n",
+  "aCBmb3IgdGhlIHAycG1lbSBwb3J0Lg0KNC4gVGhlIFJDIGRldGVybWluZXMgdGhlIElPIGlzIGNv\n",
+  "bXBsZXRlIHdoZW4gdGhlIFRMUCBhc3NvY2lhdGVkIHdpdGggc3RlcCAyIHVwZGF0ZXMgdGhlIG1l\n",
+  "bW9yeSBhc3NvY2lhdGVkIHdpdGggdGhlIENRLiBJdCBpc3N1ZXMgc29tZSBvcGVyYXRpb24gdG8g\n",
+  "cmVhZCB0aGUgcDJwbWVtLg0KNS4gUmVnYXJkbGVzcyBvZiB3aGV0aGVyIHRoZSBNZW1SZCBUTFAg\n",
+  "Y29tZXMgZnJvbSB0aGUgUkMgb3IgYW5vdGhlciBkZXZpY2UgY29ubmVjdGVkIHRvIHRoZSBzd2l0\n",
+  "Y2ggaXQgaXMgcXVldWVkIGluIHRoZSBlZ3Jlc3MgcXVldWUgZm9yIHRoZSBwMnBtZW0gRklPIGJl\n",
+  "aGluZCB0aGUgbGFzdCBETUEgVExQIChmcm9tIHN0ZXAgMSkuIFBDSWUgb3JkZXJpbmcgZW5zdXJl\n",
+  "cyB0aGF0IHRoaXMgTWVtUmQgY2Fubm90IG92ZXJ0YWtlIHRoZSBNZW1XciAoUmVhZHMgY2FuIG5l\n",
+  "dmVyIHBhc3Mgd3JpdGVzKS4gVGhlcmVmb3JlIHRoZSBNZW1SZCBjYW4gbmV2ZXIgZ2V0IHRvIHRo\n",
+  "ZSBwMnBtZW0gZGV2aWNlIHVudGlsIGFmdGVyIHRoZSBsYXN0IERNQSBNZW1XciBoYXMuDQoNCkkg\n",
+  "aG9wZSB0aGlzIGhlbHBzIQ0KDQpTdGVwaGVuDQoNCg0K"
 ]
 
-b078d2a1d9ef181930008e34eaa6b104a8f6df924271bdf0297effbcd3f4a52f
+846f1bc8ebd4f3f77fa30e0dd38f00c43dd730b9aacc7c8427c29a7a3f3d1174

diff --git a/a/1.txt b/N3/1.txt
index e657e2c..08d61fb 100644
--- a/a/1.txt
+++ b/N3/1.txt
@@ -22,7 +22,4 @@ As long as legA, legB and the RC are all connected to the same switch then order
 
 I hope this helps!
 
-Stephen
-
-
-N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dʇڙë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš
\ No newline at end of file
+Stephen
\ No newline at end of file
diff --git a/a/content_digest b/N3/content_digest
index 3444a70..df5ad08 100644
--- a/a/content_digest
+++ b/N3/content_digest
@@ -14,38 +14,14 @@
   "ref\0004df229d8-8124-664a-9bc4-6401bc034be1\@grimberg.me\0"
 ]
 [
-  "ref\0004df229d8-8124-664a-9bc4-6401bc034be1-NQWnxTmZq1alnMjI0IkVqw\@public.gmane.org\0"
+  "From\0sbates\@raithlin.com (Stephen Bates)\0"
 ]
 [
-  "From\0Stephen Bates <sbates-pv7U853sEMVWk0Htik3J/w\@public.gmane.org>\0"
-]
-[
-  "Subject\0Re: [RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem\0"
+  "Subject\0[RFC 6/8] nvmet: Be careful about using iomem accesses when dealing with p2pmem\0"
 ]
 [
   "Date\0Fri, 7 Apr 2017 11:19:30 +0000\0"
 ]
-[
-  "To\0Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw\@public.gmane.org>",
-  " Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/\@public.gmane.org>\0"
-]
-[
-  "Cc\0Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w\@public.gmane.org>",
-  " Christoph Hellwig <hch-jcswGhMUV9g\@public.gmane.org>",
-  " James E.J. Bottomley <jejb-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8\@public.gmane.org>",
-  " Martin K. Petersen <martin.petersen-QHcLZuEGTsvQT0dZR+AlfA\@public.gmane.org>",
-  " Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw\@public.gmane.org>",
-  " Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW\@public.gmane.org>",
-  " Max Gurtovoy <maxg-VPRAkNaXOzVWk0Htik3J/w\@public.gmane.org>",
-  " Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w\@public.gmane.org>",
-  " Keith Busch <keith.busch-ral2JQCrhuEAvxtiuMwx3w\@public.gmane.org>",
-  " linux-pci-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-pci-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-scsi-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-scsi-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r\@public.gmane.org <linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r\@public.gmane.org>",
-  " linux-rdma-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-rdma-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>",
-  " linux-nvdimm-y27Ovi1pjclAfugRpC6u6w\@public.gmane.org <linux-nvdimm-y27Ovi1pjclAfugRpC6u6w\@public.gmane.org>",
-  " linux-kernel-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org <linux-kernel-u79uwXL29TY76Z2rM5mHXA\@public.gmane.org>\0"
-]
 [
   "\0000:1\0"
 ]
@@ -77,10 +53,7 @@
   "\n",
   "I hope this helps!\n",
   "\n",
-  "Stephen\n",
-  "\n",
-  "\n",
-  "N\302\213\302\247\302\262\303\246\303\254r\302\270\302\233y\303\272\303\250\302\232\303\230b\302\262X\302\254\302\266\303\207\302\247v\303\230^\302\226)\303\236\302\272{.n\303\207+\302\211\302\267\302\245\302\212{\302\261\302\255\303\231\302\232\302\212{ay\302\272\35\303\212\302\207\303\232\302\231\303\253,j\a\302\255\302\242f\302\243\302\242\302\267h\302\232\302\213\302\273\303\266\303\254\27/oSc\302\276\302\231\303\232\302\2639\302\230u\303\200\302\246\303\246\303\245\302\211\303\210&jw\302\250\302\256\3(\302\255\303\251\302\232\302\216\302\212\303\235\302\242j\"\302\235\303\272\32\302\266\em\302\247\303\277\303\257\302\201\303\252\303\244z\302\271\303\236\302\226\302\212\303\240\303\276f\302\243\302\242\302\267h\302\232\302\210\302\247~\302\210m\302\232"
+  "Stephen"
 ]
 
-b078d2a1d9ef181930008e34eaa6b104a8f6df924271bdf0297effbcd3f4a52f
+3cfaa26e058c2d824b0922f13e7932e5480f893d95f0f912ed96c4faff4fd923
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.