Cabling for NTB between Intel Xeon E5 v4 servers

* Cabling for NTB between Intel Xeon E5 v4 servers
@ 2017-05-27  2:00 Ed Swierk
  2017-05-30 16:34 ` Dave Jiang
  0 siblings, 1 reply; 3+ messages in thread
From: Ed Swierk @ 2017-05-27  2:00 UTC (permalink / raw)
  To: linux-ntb; +Cc: dave.jiang

[-- Attachment #1: Type: text/plain, Size: 1217 bytes --]

I'm trying to set up a PCIe link between Intel Xeon E5 v4 (S2600WT) servers
running Linux 4.10.17, as a testbed for performance measurements. My goal
is to evaluate feasibility NTB on a custom Xeon D-based plug-in card, but
as a first step I'm sticking with systems on which NTB is already known to
(or supposed to) work.

I read https://github.com/davejiang/linux/wiki/Intel-NTB-Startup-Guide and
every other document I could find on Intel NTB, but I'm stuck on a basic
question: what kind of cable should I use to connect the PCIe slots?

I found some card-and-cable kits at
http://www.onestopsystems.com/expansion-kits, but these are quite expensive
($thousands).

Another idea is to use so-called mini-SAS HD aka SFF-8643, which I
understand can carry PCIe x4, along with a pair of PCIe-to-SF-8643 adapters
(e.g.
https://www.startech.com/HDD/Adapters/pcie-nvme-u2-ssd-adapter~PEX4SFF8643).
The TX-to-RX wiring seems like it should work, but I have no idea about
signal integrity through such a cable.

There's also the SuperMicro AOC-SLG3-2E4R, which is similar to the StarTech
card but claims to have redriver circuitry.

What cabling have folks successfully used for NTB between Intel Xeon-based
servers?

--Ed

[-- Attachment #2: Type: text/html, Size: 1653 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread