* [PATCH 0/3] ntb: Asynchronous NTB devices support
@ 2016-07-26 19:50 Serge Semin
2016-07-26 19:50 ` [PATCH 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Serge Semin @ 2016-07-26 19:50 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
Currently developed NTB-bus supports AMD and Intel Non-transparent PCIe-bridges
only. These bridges implement "synchronous" interfaces, which means that memory
window translated addresses can be direcly specified from one side to another
by writing to a corresponding value to a register. Unlike Intel and AMD there
is another vendor - IDT, which produces synchronous and asynchrnous
NTB PCIe-switches. If synchronous NTB IDT PCIe-switches works almost the same
way as AMD and Intel devices do, the asynchronous ones don't. Due to the
registers field specification the translated base addresses of memory windows
can not be directly specified to a peer. To solve the problem IDT asynchronous
PCIe-switches implement messaging interface to send any kind of information
from one RC to another including translated base addresses. In the framework of
this patches we developed a set of callback functions for the NTB-bus devices
programming interface and drivers for the NT-functions of the following IDT
PCIe-switches:
- 89HPES24NT6AG2,
- 89HPES32NT8AG2,
- 89HPES32NT8BG2,
- 89HPES12NT12G2,
- 89HPES16NT16G2,
- 89HPES24NT24G2,
- 89HPES32NT24AG2,
- 89HPES32NT24BG2,
and a set of NTB client drivers to test each subsystem of the IDT NTB drivers
separately.
The patches source code was tested on the hardware with IDT 89HPES32NT8AG2
PCIe-switch connected to root complexes running x64 big endian and x32 little
endian CPUs. The code was warningless built by x86_64, MIPS64 BE and
MIPS32 LE GNU GCC compilers.
To make it easely applicable to the Linux kernel NTB subsystem, the patches were
taken from the top of the "ntb-next" branch of the forked NTB git repository:
https://github.com/fancer/ntb/tree/ntb-next
(it is the fork of https://github.com/jonmason/ntb/tree/ntb-next)
so the hash of the last preceding branch commit was:
04a819bbee3fa4dfe22034e2724dab4efb63f0bd
There are a some types of checkpatch warnings I left unfixed in the framework of
the source code. Here is the reason why:
1) WARNING: Comparisons should place the constant on the right side of the test
Reason: The whole code is written for typos safety. Writing the constants on
the left side of comparisons prevents the code from compiling if "=" and "=="
are mistakenly confused.
2) WARNING: braces {} are not necessary for any arm of this statement
Reason: Even though the braces aren't really necessary, I left them in there
to make the code better distinguishable from the rest of the code.
3) WARNING: Avoid crashing the kernel - try using WARN_ON & recovery code
rather than BUG() or BUG_ON()
Reason: If the BUG()/BUG_ON() macro were invoked, something was really wrong,
so the developer should make sure all the configurations are correct. Otherwise
one should not use the driver.
4) WARNING: line over 80 characters
Reason: Most of these warnings appear due to just 3-5 excess characters, which
can not be excluded without reducing the code readability. The exception is the
registers tables, which are described in the item 8).
5) WARNING: Block comments use a trailing */ on a separate line
Reason: All of those warnings mean that the comment closure pattern"*/" is
specified in the end of the last comment line. If one puts the pattern to a
next line, comments visually get separated from the code they are made for. It
reduces the readability.
6) WARNING: quoted string split across lines
Reason: There was no way to reduce string lengths, so the only way to split
it across lines.
7) WARNING: do not add new typedefs
Reason: They are really needed there and added with full respect to the coding
style document.
8) ERROR: Macros with multiple statements should be enclosed in a do - while
loop
ERROR: Macros with complex values should be enclosed in parentheses
Reason: Those macros are used to create a very suitable registers table using
"enum-switch-case-macro" design pattern with o(1) access to the register
address, size and description. So adding the parentheses and do - while
statements will break the pattern. Additionally they aren't used anywhere else,
than in the framework of the pattern functions.
Of course all of these warnings are discussable, except the last one. It would
be really painful to remove it from the code =)
As one can see I thoroughly annotated and commented the drivers source code, so
it would be easier to study it by kernel hackers.
Here is what the current design of NTB-bus and IDT NTB function drivers
lacks of:
1) No support of Punch-Through functionlity of IDT PCIe-switches
2) DMA is not supported by the current revision of IDT NTB driver
3) Theoretically local and peer root complexes can have different CPU
architecture like x86 and x64. In this way, the dma_addr_t type will
have different size, which may lead to errors when x64 RC allocates
a shared buffer above 4GB memory.
One more suggestion. Since after adding this set of patches there will be two
types of devices the NTB-bus supports, we may want to implement the traditional
linux kernel bus device-drivers matching algorithm using some new
struct ntb_device_id and id_tables.
Thanks,
=============================
Serge V. Semin
Leading Programmer
Embedded SW development group
T-platforms
=============================
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
fancer (3):
ntb: Add asynchronous devices support to NTB-bus interface
ntb: IDT 89HPES*NT* PCIe-switches NTB device driver
ntb: Test client drivers for asynchronous NTB devices
drivers/ntb/Kconfig | 4 +-
drivers/ntb/hw/Kconfig | 1 +
drivers/ntb/hw/Makefile | 6 +-
drivers/ntb/hw/amd/ntb_hw_amd.c | 49 +-
drivers/ntb/hw/idt/Kconfig | 21 +
drivers/ntb/hw/idt/Makefile | 5 +
drivers/ntb/hw/idt/ntb_hw_idt.c | 4050 ++++++++++++++++++++++++++++++++
drivers/ntb/hw/idt/ntb_hw_idt.h | 390 +++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.c | 163 ++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.h | 114 +
drivers/ntb/hw/idt/ntb_hw_idt_regmap.h | 877 +++++++
drivers/ntb/hw/intel/ntb_hw_intel.c | 59 +-
drivers/ntb/ntb.c | 86 +-
drivers/ntb/ntb_transport.c | 19 +-
drivers/ntb/test/Kconfig | 32 +
drivers/ntb/test/Makefile | 9 +-
drivers/ntb/test/ntb_db_test.c | 677 ++++++
drivers/ntb/test/ntb_msg_test.c | 736 ++++++
drivers/ntb/test/ntb_mw_test.c | 1531 ++++++++++++
drivers/ntb/test/ntb_perf.c | 16 +-
drivers/ntb/test/ntb_pingpong.c | 5 +
drivers/ntb/test/ntb_tool.c | 25 +-
include/linux/ntb.h | 600 ++++-
23 files changed, 9309 insertions(+), 166 deletions(-)
create mode 100644 drivers/ntb/hw/idt/Kconfig
create mode 100644 drivers/ntb/hw/idt/Makefile
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
create mode 100644 drivers/ntb/test/ntb_db_test.c
create mode 100644 drivers/ntb/test/ntb_msg_test.c
create mode 100644 drivers/ntb/test/ntb_mw_test.c
--
2.6.6
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/3] ntb: Add asynchronous devices support to NTB-bus interface
2016-07-26 19:50 [PATCH 0/3] ntb: Asynchronous NTB devices support Serge Semin
@ 2016-07-26 19:50 ` Serge Semin
2016-07-26 19:50 ` [PATCH 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
` (2 subsequent siblings)
3 siblings, 0 replies; 11+ messages in thread
From: Serge Semin @ 2016-07-26 19:50 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
Currently supported AMD and Intel Non-transparent PCIe-bridges are synchronous
devices, so translated base address of memory windows can be direcly written
to peer registers. But there are some IDT PCIe-switches which implement
complex interfaces using Lookup Tables of translation addresses. Due to
the way the table is accessed, it can not be done synchronously from different
RCs, that's why the asynchronous interface should be developed.
For these purpose the Memory Window related interface is correspondingly split
as it is for Doorbell and Scratchpad registers. The definition of Memory Window
is following: "It is a virtual memory region, which locally reflects a physical
memory of peer device." So to speak the "ntb_peer_mw_"-prefixed methods control
the peers memory windows, "ntb_mw_"-prefixed functions work with the local
memory windows.
Here is the description of the Memory Window related NTB-bus callback
functions:
- ntb_mw_count() - number of local memory windows.
- ntb_mw_get_maprsc() - get the physical address and size of the local memory
window to map.
- ntb_mw_set_trans() - set translation address of local memory window (this
address should be somehow retrieved from a peer).
- ntb_mw_get_trans() - get translation address of local memory window.
- ntb_mw_get_align() - get alignment of translated base address and size of
local memory window. Additionally one can get the
upper size limit of the memory window.
- ntb_peer_mw_count() - number of peer memory windows (it can differ from the
local number).
- ntb_peer_mw_set_trans() - set translation address of peer memory window
- ntb_peer_mw_get_trans() - get translation address of peer memory window
- ntb_peer_mw_get_align() - get alignment of translated base address and size
of peer memory window.Additionally one can get the
upper size limit of the memory window.
As one can see current AMD and Intel NTB drivers mostly implement the
"ntb_peer_mw_"-prefixed methods. So this patch correspondingly renames the
driver functions. IDT NTB driver mostly expose "ntb_nw_"-prefixed methods,
since it doesn't have convenient access to the peer Lookup Table.
In order to pass information from one RC to another NTB functions of IDT
PCIe-switch implement Messaging subsystem. They currently support four message
registers to transfer DWORD sized data to a specified peer. So there are two
new callback methods are introduced:
- ntb_msg_size() - get the number of DWORDs supported by NTB function to send
and receive messages
- ntb_msg_post() - send message of size retrieved from ntb_msg_size()
to a peer
Additionally there is a new event function:
- ntb_msg_event() - it is invoked when either a new message was retrieved
(NTB_MSG_NEW), or last message was successfully sent
(NTB_MSG_SENT), or the last message failed to be sent
(NTB_MSG_FAIL).
The last change concerns the IDs (practically names) of NTB-devices on the
NTB-bus. It is not good to have the devices with same names in the system
and it brakes my IDT NTB driver from being loaded =) So I developed a simple
algorithm of NTB devices naming. Particulary it generates names "ntbS{N}" for
synchronous devices, "ntbA{N}" for asynchronous devices, and "ntbAS{N}" for
devices supporting both interfaces.
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
drivers/ntb/Kconfig | 4 +-
drivers/ntb/hw/amd/ntb_hw_amd.c | 49 ++-
drivers/ntb/hw/intel/ntb_hw_intel.c | 59 +++-
drivers/ntb/ntb.c | 86 +++++-
drivers/ntb/ntb_transport.c | 19 +-
drivers/ntb/test/ntb_perf.c | 16 +-
drivers/ntb/test/ntb_pingpong.c | 5 +
drivers/ntb/test/ntb_tool.c | 25 +-
include/linux/ntb.h | 600 +++++++++++++++++++++++++++++-------
9 files changed, 701 insertions(+), 162 deletions(-)
diff --git a/drivers/ntb/Kconfig b/drivers/ntb/Kconfig
index 95944e5..67d80c4 100644
--- a/drivers/ntb/Kconfig
+++ b/drivers/ntb/Kconfig
@@ -14,8 +14,6 @@ if NTB
source "drivers/ntb/hw/Kconfig"
-source "drivers/ntb/test/Kconfig"
-
config NTB_TRANSPORT
tristate "NTB Transport Client"
help
@@ -25,4 +23,6 @@ config NTB_TRANSPORT
If unsure, say N.
+source "drivers/ntb/test/Kconfig"
+
endif # NTB
diff --git a/drivers/ntb/hw/amd/ntb_hw_amd.c b/drivers/ntb/hw/amd/ntb_hw_amd.c
index 6ccba0d..ab6f353 100644
--- a/drivers/ntb/hw/amd/ntb_hw_amd.c
+++ b/drivers/ntb/hw/amd/ntb_hw_amd.c
@@ -55,6 +55,7 @@
#include <linux/pci.h>
#include <linux/random.h>
#include <linux/slab.h>
+#include <linux/sizes.h>
#include <linux/ntb.h>
#include "ntb_hw_amd.h"
@@ -84,11 +85,8 @@ static int amd_ntb_mw_count(struct ntb_dev *ntb)
return ntb_ndev(ntb)->mw_count;
}
-static int amd_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base,
- resource_size_t *size,
- resource_size_t *align,
- resource_size_t *align_size)
+static int amd_ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size)
{
struct amd_ntb_dev *ndev = ntb_ndev(ntb);
int bar;
@@ -103,17 +101,40 @@ static int amd_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
if (size)
*size = pci_resource_len(ndev->ntb.pdev, bar);
- if (align)
- *align = SZ_4K;
+ return 0;
+}
+
+static int amd_ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ return ntb_ndev(ntb)->mw_count;
+}
+
+static int amd_ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct amd_ntb_dev *ndev = ntb_ndev(ntb);
+ int bar;
+
+ bar = ndev_mw_to_bar(ndev, idx);
+ if (bar < 0)
+ return bar;
+
+ if (addr_align)
+ *addr_align = SZ_4K;
+
+ if (size_align)
+ *size_align = 1;
- if (align_size)
- *align_size = 1;
+ if (size_max)
+ *size_max = pci_resource_len(ndev->ntb.pdev, bar);
return 0;
}
-static int amd_ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
- dma_addr_t addr, resource_size_t size)
+static int amd_ntb_peer_mw_set_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size)
{
struct amd_ntb_dev *ndev = ntb_ndev(ntb);
unsigned long xlat_reg, limit_reg = 0;
@@ -432,8 +453,10 @@ static int amd_ntb_peer_spad_write(struct ntb_dev *ntb,
static const struct ntb_dev_ops amd_ntb_ops = {
.mw_count = amd_ntb_mw_count,
- .mw_get_range = amd_ntb_mw_get_range,
- .mw_set_trans = amd_ntb_mw_set_trans,
+ .mw_get_maprsc = amd_ntb_mw_get_maprsc,
+ .peer_mw_count = amd_ntb_peer_mw_count,
+ .peer_mw_get_align = amd_ntb_peer_mw_get_align,
+ .peer_mw_set_trans = amd_ntb_peer_mw_set_trans,
.link_is_up = amd_ntb_link_is_up,
.link_enable = amd_ntb_link_enable,
.link_disable = amd_ntb_link_disable,
diff --git a/drivers/ntb/hw/intel/ntb_hw_intel.c b/drivers/ntb/hw/intel/ntb_hw_intel.c
index 40d04ef..fdb2838 100644
--- a/drivers/ntb/hw/intel/ntb_hw_intel.c
+++ b/drivers/ntb/hw/intel/ntb_hw_intel.c
@@ -804,11 +804,8 @@ static int intel_ntb_mw_count(struct ntb_dev *ntb)
return ntb_ndev(ntb)->mw_count;
}
-static int intel_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base,
- resource_size_t *size,
- resource_size_t *align,
- resource_size_t *align_size)
+static int intel_ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size)
{
struct intel_ntb_dev *ndev = ntb_ndev(ntb);
int bar;
@@ -828,17 +825,51 @@ static int intel_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
*size = pci_resource_len(ndev->ntb.pdev, bar) -
(idx == ndev->b2b_idx ? ndev->b2b_off : 0);
- if (align)
- *align = pci_resource_len(ndev->ntb.pdev, bar);
+ return 0;
+}
+
+static int intel_ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ return ntb_ndev(ntb)->mw_count;
+}
+
+static int intel_ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct intel_ntb_dev *ndev = ntb_ndev(ntb);
+ resource_size_t bar_size, mw_size;
+ int bar;
+
+ if (idx >= ndev->b2b_idx && !ndev->b2b_off)
+ idx += 1;
+
+ bar = ndev_mw_to_bar(ndev, idx);
+ if (bar < 0)
+ return bar;
+
+ bar_size = pci_resource_len(ndev->ntb.pdev, bar);
+
+ if (idx == ndev->b2b_idx)
+ mw_size = bar_size - ndev->b2b_off;
+ else
+ mw_size = bar_size;
+
+ if (addr_align)
+ *addr_align = bar_size;
+
+ if (size_align)
+ *size_align = 1;
- if (align_size)
- *align_size = 1;
+ if (size_max)
+ *size_max = mw_size;
return 0;
}
-static int intel_ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
- dma_addr_t addr, resource_size_t size)
+static int intel_ntb_peer_mw_set_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size)
{
struct intel_ntb_dev *ndev = ntb_ndev(ntb);
unsigned long base_reg, xlat_reg, limit_reg;
@@ -2220,8 +2251,10 @@ static struct intel_b2b_addr xeon_b2b_dsd_addr = {
/* operations for primary side of local ntb */
static const struct ntb_dev_ops intel_ntb_ops = {
.mw_count = intel_ntb_mw_count,
- .mw_get_range = intel_ntb_mw_get_range,
- .mw_set_trans = intel_ntb_mw_set_trans,
+ .mw_get_maprsc = intel_ntb_mw_get_maprsc,
+ .peer_mw_count = intel_ntb_peer_mw_count,
+ .peer_mw_get_align = intel_ntb_peer_mw_get_align,
+ .peer_mw_set_trans = intel_ntb_peer_mw_set_trans,
.link_is_up = intel_ntb_link_is_up,
.link_enable = intel_ntb_link_enable,
.link_disable = intel_ntb_link_disable,
diff --git a/drivers/ntb/ntb.c b/drivers/ntb/ntb.c
index 2e25307..37c3b36 100644
--- a/drivers/ntb/ntb.c
+++ b/drivers/ntb/ntb.c
@@ -54,6 +54,7 @@
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/atomic.h>
#include <linux/ntb.h>
#include <linux/pci.h>
@@ -72,8 +73,62 @@ MODULE_AUTHOR(DRIVER_AUTHOR);
MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
static struct bus_type ntb_bus;
+static struct ntb_bus_data ntb_data;
static void ntb_dev_release(struct device *dev);
+static int ntb_gen_devid(struct ntb_dev *ntb)
+{
+ const char *name;
+ unsigned long *mask;
+ int id;
+
+ if (ntb_valid_sync_dev_ops(ntb) && ntb_valid_async_dev_ops(ntb)) {
+ name = "ntbAS%d";
+ mask = ntb_data.both_msk;
+ } else if (ntb_valid_sync_dev_ops(ntb)) {
+ name = "ntbS%d";
+ mask = ntb_data.sync_msk;
+ } else if (ntb_valid_async_dev_ops(ntb)) {
+ name = "ntbA%d";
+ mask = ntb_data.async_msk;
+ } else {
+ return -EINVAL;
+ }
+
+ for (id = 0; NTB_MAX_DEVID > id; id++) {
+ if (0 == test_and_set_bit(id, mask)) {
+ ntb->id = id;
+ break;
+ }
+ }
+
+ if (NTB_MAX_DEVID > id) {
+ dev_set_name(&ntb->dev, name, ntb->id);
+ } else {
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void ntb_free_devid(struct ntb_dev *ntb)
+{
+ unsigned long *mask;
+
+ if (ntb_valid_sync_dev_ops(ntb) && ntb_valid_async_dev_ops(ntb)) {
+ mask = ntb_data.both_msk;
+ } else if (ntb_valid_sync_dev_ops(ntb)) {
+ mask = ntb_data.sync_msk;
+ } else if (ntb_valid_async_dev_ops(ntb)) {
+ mask = ntb_data.async_msk;
+ } else {
+ /* It's impossible */
+ BUG();
+ }
+
+ clear_bit(ntb->id, mask);
+}
+
int __ntb_register_client(struct ntb_client *client, struct module *mod,
const char *mod_name)
{
@@ -99,13 +154,15 @@ EXPORT_SYMBOL(ntb_unregister_client);
int ntb_register_device(struct ntb_dev *ntb)
{
+ int ret;
+
if (!ntb)
return -EINVAL;
if (!ntb->pdev)
return -EINVAL;
if (!ntb->ops)
return -EINVAL;
- if (!ntb_dev_ops_is_valid(ntb->ops))
+ if (!ntb_valid_sync_dev_ops(ntb) && !ntb_valid_async_dev_ops(ntb))
return -EINVAL;
init_completion(&ntb->released);
@@ -114,13 +171,21 @@ int ntb_register_device(struct ntb_dev *ntb)
ntb->dev.bus = &ntb_bus;
ntb->dev.parent = &ntb->pdev->dev;
ntb->dev.release = ntb_dev_release;
- dev_set_name(&ntb->dev, "%s", pci_name(ntb->pdev));
ntb->ctx = NULL;
ntb->ctx_ops = NULL;
spin_lock_init(&ntb->ctx_lock);
- return device_register(&ntb->dev);
+ /* No need to wait for completion if failed */
+ ret = ntb_gen_devid(ntb);
+ if (ret)
+ return ret;
+
+ ret = device_register(&ntb->dev);
+ if (ret)
+ ntb_free_devid(ntb);
+
+ return ret;
}
EXPORT_SYMBOL(ntb_register_device);
@@ -128,6 +193,7 @@ void ntb_unregister_device(struct ntb_dev *ntb)
{
device_unregister(&ntb->dev);
wait_for_completion(&ntb->released);
+ ntb_free_devid(ntb);
}
EXPORT_SYMBOL(ntb_unregister_device);
@@ -191,6 +257,20 @@ void ntb_db_event(struct ntb_dev *ntb, int vector)
}
EXPORT_SYMBOL(ntb_db_event);
+void ntb_msg_event(struct ntb_dev *ntb, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg)
+{
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&ntb->ctx_lock, irqflags);
+ {
+ if (ntb->ctx_ops && ntb->ctx_ops->msg_event)
+ ntb->ctx_ops->msg_event(ntb->ctx, ev, msg);
+ }
+ spin_unlock_irqrestore(&ntb->ctx_lock, irqflags);
+}
+EXPORT_SYMBOL(ntb_msg_event);
+
static int ntb_probe(struct device *dev)
{
struct ntb_dev *ntb;
diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
index d5c5894..2626ba0 100644
--- a/drivers/ntb/ntb_transport.c
+++ b/drivers/ntb/ntb_transport.c
@@ -673,7 +673,7 @@ static void ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw)
if (!mw->virt_addr)
return;
- ntb_mw_clear_trans(nt->ndev, num_mw);
+ ntb_peer_mw_set_trans(nt->ndev, num_mw, 0, 0);
dma_free_coherent(&pdev->dev, mw->buff_size,
mw->virt_addr, mw->dma_addr);
mw->xlat_size = 0;
@@ -730,7 +730,8 @@ static int ntb_set_mw(struct ntb_transport_ctx *nt, int num_mw,
}
/* Notify HW the memory location of the receive buffer */
- rc = ntb_mw_set_trans(nt->ndev, num_mw, mw->dma_addr, mw->xlat_size);
+ rc = ntb_peer_mw_set_trans(nt->ndev, num_mw, mw->dma_addr,
+ mw->xlat_size);
if (rc) {
dev_err(&pdev->dev, "Unable to set mw%d translation", num_mw);
ntb_free_mw(nt, num_mw);
@@ -1060,7 +1061,11 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
int node;
int rc, i;
- mw_count = ntb_mw_count(ndev);
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ndev))
+ return -EINVAL;
+
+ mw_count = ntb_peer_mw_count(ndev);
if (ntb_spad_count(ndev) < (NUM_MWS + 1 + mw_count * 2)) {
dev_err(&ndev->dev, "Not enough scratch pad registers for %s",
NTB_TRANSPORT_NAME);
@@ -1094,8 +1099,12 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
for (i = 0; i < mw_count; i++) {
mw = &nt->mw_vec[i];
- rc = ntb_mw_get_range(ndev, i, &mw->phys_addr, &mw->phys_size,
- &mw->xlat_align, &mw->xlat_align_size);
+ rc = ntb_mw_get_maprsc(ndev, i, &mw->phys_addr, &mw->phys_size);
+ if (rc)
+ goto err1;
+
+ rc = ntb_peer_mw_get_align(ndev, i, &mw->xlat_align,
+ &mw->xlat_align_size, NULL);
if (rc)
goto err1;
diff --git a/drivers/ntb/test/ntb_perf.c b/drivers/ntb/test/ntb_perf.c
index 6a50f20..f2952f7 100644
--- a/drivers/ntb/test/ntb_perf.c
+++ b/drivers/ntb/test/ntb_perf.c
@@ -452,7 +452,7 @@ static void perf_free_mw(struct perf_ctx *perf)
if (!mw->virt_addr)
return;
- ntb_mw_clear_trans(perf->ntb, 0);
+ ntb_peer_mw_set_trans(perf->ntb, 0, 0, 0);
dma_free_coherent(&pdev->dev, mw->buf_size,
mw->virt_addr, mw->dma_addr);
mw->xlat_size = 0;
@@ -488,7 +488,7 @@ static int perf_set_mw(struct perf_ctx *perf, resource_size_t size)
mw->buf_size = 0;
}
- rc = ntb_mw_set_trans(perf->ntb, 0, mw->dma_addr, mw->xlat_size);
+ rc = ntb_peer_mw_set_trans(perf->ntb, 0, mw->dma_addr, mw->xlat_size);
if (rc) {
dev_err(&perf->ntb->dev, "Unable to set mw0 translation\n");
perf_free_mw(perf);
@@ -559,8 +559,12 @@ static int perf_setup_mw(struct ntb_dev *ntb, struct perf_ctx *perf)
mw = &perf->mw;
- rc = ntb_mw_get_range(ntb, 0, &mw->phys_addr, &mw->phys_size,
- &mw->xlat_align, &mw->xlat_align_size);
+ rc = ntb_mw_get_maprsc(ntb, 0, &mw->phys_addr, &mw->phys_size);
+ if (rc)
+ return rc;
+
+ rc = ntb_peer_mw_get_align(ntb, 0, &mw->xlat_align,
+ &mw->xlat_align_size, NULL);
if (rc)
return rc;
@@ -758,6 +762,10 @@ static int perf_probe(struct ntb_client *client, struct ntb_dev *ntb)
int node;
int rc = 0;
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ntb))
+ return -EINVAL;
+
if (ntb_spad_count(ntb) < MAX_SPAD) {
dev_err(&ntb->dev, "Not enough scratch pad registers for %s",
DRIVER_NAME);
diff --git a/drivers/ntb/test/ntb_pingpong.c b/drivers/ntb/test/ntb_pingpong.c
index 7d31179..e833649 100644
--- a/drivers/ntb/test/ntb_pingpong.c
+++ b/drivers/ntb/test/ntb_pingpong.c
@@ -214,6 +214,11 @@ static int pp_probe(struct ntb_client *client,
struct pp_ctx *pp;
int rc;
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
if (ntb_db_is_unsafe(ntb)) {
dev_dbg(&ntb->dev, "doorbell is unsafe\n");
if (!unsafe) {
diff --git a/drivers/ntb/test/ntb_tool.c b/drivers/ntb/test/ntb_tool.c
index 61bf2ef..5dfe12f 100644
--- a/drivers/ntb/test/ntb_tool.c
+++ b/drivers/ntb/test/ntb_tool.c
@@ -675,8 +675,11 @@ static int tool_setup_mw(struct tool_ctx *tc, int idx, size_t req_size)
if (mw->peer)
return 0;
- rc = ntb_mw_get_range(tc->ntb, idx, &base, &size, &align,
- &align_size);
+ rc = ntb_mw_get_maprsc(tc->ntb, idx, &base, &size);
+ if (rc)
+ return rc;
+
+ rc = ntb_peer_mw_get_align(tc->ntb, idx, &align, &align_size, NULL);
if (rc)
return rc;
@@ -689,7 +692,7 @@ static int tool_setup_mw(struct tool_ctx *tc, int idx, size_t req_size)
if (!mw->peer)
return -ENOMEM;
- rc = ntb_mw_set_trans(tc->ntb, idx, mw->peer_dma, mw->size);
+ rc = ntb_peer_mw_set_trans(tc->ntb, idx, mw->peer_dma, mw->size);
if (rc)
goto err_free_dma;
@@ -716,7 +719,7 @@ static void tool_free_mw(struct tool_ctx *tc, int idx)
struct tool_mw *mw = &tc->mws[idx];
if (mw->peer) {
- ntb_mw_clear_trans(tc->ntb, idx);
+ ntb_peer_mw_set_trans(tc->ntb, idx, 0, 0);
dma_free_coherent(&tc->ntb->pdev->dev, mw->size,
mw->peer,
mw->peer_dma);
@@ -751,8 +754,8 @@ static ssize_t tool_peer_mw_trans_read(struct file *filep,
if (!buf)
return -ENOMEM;
- ntb_mw_get_range(mw->tc->ntb, mw->idx,
- &base, &mw_size, &align, &align_size);
+ ntb_mw_get_maprsc(mw->tc->ntb, mw->idx, &base, &mw_size);
+ ntb_peer_mw_get_align(mw->tc->ntb, mw->idx, &align, &align_size, NULL);
off += scnprintf(buf + off, buf_size - off,
"Peer MW %d Information:\n", mw->idx);
@@ -827,8 +830,7 @@ static int tool_init_mw(struct tool_ctx *tc, int idx)
phys_addr_t base;
int rc;
- rc = ntb_mw_get_range(tc->ntb, idx, &base, &mw->win_size,
- NULL, NULL);
+ rc = ntb_mw_get_maprsc(tc->ntb, idx, &base, &mw->win_size);
if (rc)
return rc;
@@ -913,6 +915,11 @@ static int tool_probe(struct ntb_client *self, struct ntb_dev *ntb)
int rc;
int i;
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
if (ntb_db_is_unsafe(ntb))
dev_dbg(&ntb->dev, "doorbell is unsafe\n");
@@ -928,7 +935,7 @@ static int tool_probe(struct ntb_client *self, struct ntb_dev *ntb)
tc->ntb = ntb;
init_waitqueue_head(&tc->link_wq);
- tc->mw_count = min(ntb_mw_count(tc->ntb), MAX_MWS);
+ tc->mw_count = min(ntb_peer_mw_count(tc->ntb), MAX_MWS);
for (i = 0; i < tc->mw_count; i++) {
rc = tool_init_mw(tc, i);
if (rc)
diff --git a/include/linux/ntb.h b/include/linux/ntb.h
index 6f47562..d1937d3 100644
--- a/include/linux/ntb.h
+++ b/include/linux/ntb.h
@@ -159,13 +159,44 @@ static inline int ntb_client_ops_is_valid(const struct ntb_client_ops *ops)
}
/**
+ * struct ntb_msg - ntb driver message structure
+ * @type: Message type.
+ * @payload: Payload data to send to a peer
+ * @data: Array of u32 data to send (size might be hw dependent)
+ */
+#define NTB_MAX_MSGSIZE 4
+struct ntb_msg {
+ union {
+ struct {
+ u32 type;
+ u32 payload[NTB_MAX_MSGSIZE - 1];
+ };
+ u32 data[NTB_MAX_MSGSIZE];
+ };
+};
+
+/**
+ * enum NTB_MSG_EVENT - message event types
+ * @NTB_MSG_NEW: New message just arrived and passed to the handler
+ * @NTB_MSG_SENT: Posted message has just been successfully sent
+ * @NTB_MSG_FAIL: Posted message failed to be sent
+ */
+enum NTB_MSG_EVENT {
+ NTB_MSG_NEW,
+ NTB_MSG_SENT,
+ NTB_MSG_FAIL
+};
+
+/**
* struct ntb_ctx_ops - ntb driver context operations
* @link_event: See ntb_link_event().
* @db_event: See ntb_db_event().
+ * @msg_event: See ntb_msg_event().
*/
struct ntb_ctx_ops {
void (*link_event)(void *ctx);
void (*db_event)(void *ctx, int db_vector);
+ void (*msg_event)(void *ctx, enum NTB_MSG_EVENT ev, struct ntb_msg *msg);
};
static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
@@ -174,18 +205,24 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
return
/* ops->link_event && */
/* ops->db_event && */
+ /* ops->msg_event && */
1;
}
/**
* struct ntb_ctx_ops - ntb device operations
- * @mw_count: See ntb_mw_count().
- * @mw_get_range: See ntb_mw_get_range().
- * @mw_set_trans: See ntb_mw_set_trans().
- * @mw_clear_trans: See ntb_mw_clear_trans().
* @link_is_up: See ntb_link_is_up().
* @link_enable: See ntb_link_enable().
* @link_disable: See ntb_link_disable().
+ * @mw_count: See ntb_mw_count().
+ * @mw_get_maprsc: See ntb_mw_get_maprsc().
+ * @mw_set_trans: See ntb_mw_set_trans().
+ * @mw_get_trans: See ntb_mw_get_trans().
+ * @mw_get_align: See ntb_mw_get_align().
+ * @peer_mw_count: See ntb_peer_mw_count().
+ * @peer_mw_set_trans: See ntb_peer_mw_set_trans().
+ * @peer_mw_get_trans: See ntb_peer_mw_get_trans().
+ * @peer_mw_get_align: See ntb_peer_mw_get_align().
* @db_is_unsafe: See ntb_db_is_unsafe().
* @db_valid_mask: See ntb_db_valid_mask().
* @db_vector_count: See ntb_db_vector_count().
@@ -210,22 +247,38 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
* @peer_spad_addr: See ntb_peer_spad_addr().
* @peer_spad_read: See ntb_peer_spad_read().
* @peer_spad_write: See ntb_peer_spad_write().
+ * @msg_post: See ntb_msg_post().
+ * @msg_size: See ntb_msg_size().
*/
struct ntb_dev_ops {
- int (*mw_count)(struct ntb_dev *ntb);
- int (*mw_get_range)(struct ntb_dev *ntb, int idx,
- phys_addr_t *base, resource_size_t *size,
- resource_size_t *align, resource_size_t *align_size);
- int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
- dma_addr_t addr, resource_size_t size);
- int (*mw_clear_trans)(struct ntb_dev *ntb, int idx);
-
int (*link_is_up)(struct ntb_dev *ntb,
enum ntb_speed *speed, enum ntb_width *width);
int (*link_enable)(struct ntb_dev *ntb,
enum ntb_speed max_speed, enum ntb_width max_width);
int (*link_disable)(struct ntb_dev *ntb);
+ int (*mw_count)(struct ntb_dev *ntb);
+ int (*mw_get_maprsc)(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size);
+ int (*mw_get_align)(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max);
+ int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size);
+ int (*mw_get_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size);
+
+ int (*peer_mw_count)(struct ntb_dev *ntb);
+ int (*peer_mw_get_align)(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max);
+ int (*peer_mw_set_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size);
+ int (*peer_mw_get_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size);
+
int (*db_is_unsafe)(struct ntb_dev *ntb);
u64 (*db_valid_mask)(struct ntb_dev *ntb);
int (*db_vector_count)(struct ntb_dev *ntb);
@@ -259,47 +312,10 @@ struct ntb_dev_ops {
phys_addr_t *spad_addr);
u32 (*peer_spad_read)(struct ntb_dev *ntb, int idx);
int (*peer_spad_write)(struct ntb_dev *ntb, int idx, u32 val);
-};
-
-static inline int ntb_dev_ops_is_valid(const struct ntb_dev_ops *ops)
-{
- /* commented callbacks are not required: */
- return
- ops->mw_count &&
- ops->mw_get_range &&
- ops->mw_set_trans &&
- /* ops->mw_clear_trans && */
- ops->link_is_up &&
- ops->link_enable &&
- ops->link_disable &&
- /* ops->db_is_unsafe && */
- ops->db_valid_mask &&
- /* both set, or both unset */
- (!ops->db_vector_count == !ops->db_vector_mask) &&
-
- ops->db_read &&
- /* ops->db_set && */
- ops->db_clear &&
- /* ops->db_read_mask && */
- ops->db_set_mask &&
- ops->db_clear_mask &&
- /* ops->peer_db_addr && */
- /* ops->peer_db_read && */
- ops->peer_db_set &&
- /* ops->peer_db_clear && */
- /* ops->peer_db_read_mask && */
- /* ops->peer_db_set_mask && */
- /* ops->peer_db_clear_mask && */
- /* ops->spad_is_unsafe && */
- ops->spad_count &&
- ops->spad_read &&
- ops->spad_write &&
- /* ops->peer_spad_addr && */
- /* ops->peer_spad_read && */
- ops->peer_spad_write &&
- 1;
-}
+ int (*msg_post)(struct ntb_dev *ntb, struct ntb_msg *msg);
+ int (*msg_size)(struct ntb_dev *ntb);
+};
/**
* struct ntb_client - client interested in ntb devices
@@ -310,10 +326,22 @@ struct ntb_client {
struct device_driver drv;
const struct ntb_client_ops ops;
};
-
#define drv_ntb_client(__drv) container_of((__drv), struct ntb_client, drv)
/**
+ * struct ntb_bus_data - NTB bus data
+ * @sync_msk: Synchroous devices mask
+ * @async_msk: Asynchronous devices mask
+ * @both_msk: Both sync and async devices mask
+ */
+#define NTB_MAX_DEVID (8*BITS_PER_LONG)
+struct ntb_bus_data {
+ unsigned long sync_msk[8];
+ unsigned long async_msk[8];
+ unsigned long both_msk[8];
+};
+
+/**
* struct ntb_device - ntb device
* @dev: Linux device object.
* @pdev: Pci device entry of the ntb.
@@ -332,15 +360,151 @@ struct ntb_dev {
/* private: */
+ /* device id */
+ int id;
/* synchronize setting, clearing, and calling ctx_ops */
spinlock_t ctx_lock;
/* block unregister until device is fully released */
struct completion released;
};
-
#define dev_ntb(__dev) container_of((__dev), struct ntb_dev, dev)
/**
+ * ntb_valid_sync_dev_ops() - valid operations for synchronous hardware setup
+ * @ntb: NTB device
+ *
+ * There might be two types of NTB hardware differed by the way of the settings
+ * configuration. The synchronous chips allows to set the memory windows by
+ * directly writing to the peer registers. Additionally there can be shared
+ * Scratchpad registers for synchronous information exchange. Client drivers
+ * should call this function to make sure the hardware supports the proper
+ * functionality.
+ */
+static inline int ntb_valid_sync_dev_ops(const struct ntb_dev *ntb)
+{
+ const struct ntb_dev_ops *ops = ntb->ops;
+
+ /* Commented callbacks are not required, but might be developed */
+ return /* NTB link status ops */
+ ops->link_is_up &&
+ ops->link_enable &&
+ ops->link_disable &&
+
+ /* Synchronous memory windows ops */
+ ops->mw_count &&
+ ops->mw_get_maprsc &&
+ /* ops->mw_get_align && */
+ /* ops->mw_set_trans && */
+ /* ops->mw_get_trans && */
+ ops->peer_mw_count &&
+ ops->peer_mw_get_align &&
+ ops->peer_mw_set_trans &&
+ /* ops->peer_mw_get_trans && */
+
+ /* Doorbell ops */
+ /* ops->db_is_unsafe && */
+ ops->db_valid_mask &&
+ /* both set, or both unset */
+ (!ops->db_vector_count == !ops->db_vector_mask) &&
+ ops->db_read &&
+ /* ops->db_set && */
+ ops->db_clear &&
+ /* ops->db_read_mask && */
+ ops->db_set_mask &&
+ ops->db_clear_mask &&
+ /* ops->peer_db_addr && */
+ /* ops->peer_db_read && */
+ ops->peer_db_set &&
+ /* ops->peer_db_clear && */
+ /* ops->peer_db_read_mask && */
+ /* ops->peer_db_set_mask && */
+ /* ops->peer_db_clear_mask && */
+
+ /* Scratchpad ops */
+ /* ops->spad_is_unsafe && */
+ ops->spad_count &&
+ ops->spad_read &&
+ ops->spad_write &&
+ /* ops->peer_spad_addr && */
+ /* ops->peer_spad_read && */
+ ops->peer_spad_write &&
+
+ /* Messages IO ops */
+ /* ops->msg_post && */
+ /* ops->msg_size && */
+ 1;
+}
+
+/**
+ * ntb_valid_async_dev_ops() - valid operations for asynchronous hardware setup
+ * @ntb: NTB device
+ *
+ * There might be two types of NTB hardware differed by the way of the settings
+ * configuration. The asynchronous chips does not allow to set the memory
+ * windows by directly writing to the peer registers. Instead it implements
+ * the additional method to communinicate between NTB nodes like messages.
+ * Scratchpad registers aren't likely supported by such hardware. Client
+ * drivers should call this function to make sure the hardware supports
+ * the proper functionality.
+ */
+static inline int ntb_valid_async_dev_ops(const struct ntb_dev *ntb)
+{
+ const struct ntb_dev_ops *ops = ntb->ops;
+
+ /* Commented callbacks are not required, but might be developed */
+ return /* NTB link status ops */
+ ops->link_is_up &&
+ ops->link_enable &&
+ ops->link_disable &&
+
+ /* Asynchronous memory windows ops */
+ ops->mw_count &&
+ ops->mw_get_maprsc &&
+ ops->mw_get_align &&
+ ops->mw_set_trans &&
+ /* ops->mw_get_trans && */
+ ops->peer_mw_count &&
+ ops->peer_mw_get_align &&
+ /* ops->peer_mw_set_trans && */
+ /* ops->peer_mw_get_trans && */
+
+ /* Doorbell ops */
+ /* ops->db_is_unsafe && */
+ ops->db_valid_mask &&
+ /* both set, or both unset */
+ (!ops->db_vector_count == !ops->db_vector_mask) &&
+ ops->db_read &&
+ /* ops->db_set && */
+ ops->db_clear &&
+ /* ops->db_read_mask && */
+ ops->db_set_mask &&
+ ops->db_clear_mask &&
+ /* ops->peer_db_addr && */
+ /* ops->peer_db_read && */
+ ops->peer_db_set &&
+ /* ops->peer_db_clear && */
+ /* ops->peer_db_read_mask && */
+ /* ops->peer_db_set_mask && */
+ /* ops->peer_db_clear_mask && */
+
+ /* Scratchpad ops */
+ /* ops->spad_is_unsafe && */
+ /* ops->spad_count && */
+ /* ops->spad_read && */
+ /* ops->spad_write && */
+ /* ops->peer_spad_addr && */
+ /* ops->peer_spad_read && */
+ /* ops->peer_spad_write && */
+
+ /* Messages IO ops */
+ ops->msg_post &&
+ ops->msg_size &&
+ 1;
+}
+
+
+
+/**
* ntb_register_client() - register a client for interest in ntb devices
* @client: Client context.
*
@@ -441,10 +605,84 @@ void ntb_link_event(struct ntb_dev *ntb);
void ntb_db_event(struct ntb_dev *ntb, int vector);
/**
- * ntb_mw_count() - get the number of memory windows
+ * ntb_msg_event() - notify driver context of event in messaging subsystem
* @ntb: NTB device context.
+ * @ev: Event type caused the handler invocation
+ * @msg: Message related to the event
+ *
+ * Notify the driver context that there is some event happaned in the event
+ * subsystem. If NTB_MSG_NEW is emitted then the new message has just arrived.
+ * NTB_MSG_SENT is rised if some message has just been successfully sent to a
+ * peer. If a message failed to be sent then NTB_MSG_FAIL is emitted. The very
+ * last argument is used to pass the event related message. It discarded right
+ * after the handler returns.
+ */
+void ntb_msg_event(struct ntb_dev *ntb, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg);
+
+/**
+ * ntb_link_is_up() - get the current ntb link state
+ * @ntb: NTB device context.
+ * @speed: OUT - The link speed expressed as PCIe generation number.
+ * @width: OUT - The link width expressed as the number of PCIe lanes.
+ *
+ * Get the current state of the ntb link. It is recommended to query the link
+ * state once after every link event. It is safe to query the link state in
+ * the context of the link event callback.
+ *
+ * Return: One if the link is up, zero if the link is down, otherwise a
+ * negative value indicating the error number.
+ */
+static inline int ntb_link_is_up(struct ntb_dev *ntb,
+ enum ntb_speed *speed, enum ntb_width *width)
+{
+ return ntb->ops->link_is_up(ntb, speed, width);
+}
+
+/**
+ * ntb_link_enable() - enable the link on the secondary side of the ntb
+ * @ntb: NTB device context.
+ * @max_speed: The maximum link speed expressed as PCIe generation number.
+ * @max_width: The maximum link width expressed as the number of PCIe lanes.
*
- * Hardware and topology may support a different number of memory windows.
+ * Enable the link on the secondary side of the ntb. This can only be done
+ * from only one (primary or secondary) side of the ntb in primary or b2b
+ * topology. The ntb device should train the link to its maximum speed and
+ * width, or the requested speed and width, whichever is smaller, if supported.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_link_enable(struct ntb_dev *ntb,
+ enum ntb_speed max_speed,
+ enum ntb_width max_width)
+{
+ return ntb->ops->link_enable(ntb, max_speed, max_width);
+}
+
+/**
+ * ntb_link_disable() - disable the link on the secondary side of the ntb
+ * @ntb: NTB device context.
+ *
+ * Disable the link on the secondary side of the ntb. This can only be
+ * done from only one (primary or secondary) side of the ntb in primary or b2b
+ * topology. The ntb device should disable the link. Returning from this call
+ * must indicate that a barrier has passed, though with no more writes may pass
+ * in either direction across the link, except if this call returns an error
+ * number.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_link_disable(struct ntb_dev *ntb)
+{
+ return ntb->ops->link_disable(ntb);
+}
+
+/**
+ * ntb_mw_count() - get the number of local memory windows
+ * @ntb: NTB device context.
+ *
+ * Hardware and topology may support a different number of memory windows at
+ * local and remote devices
*
* Return: the number of memory windows.
*/
@@ -454,122 +692,186 @@ static inline int ntb_mw_count(struct ntb_dev *ntb)
}
/**
- * ntb_mw_get_range() - get the range of a memory window
+ * ntb_mw_get_maprsc() - get the range of a memory window to map
* @ntb: NTB device context.
* @idx: Memory window number.
* @base: OUT - the base address for mapping the memory window
* @size: OUT - the size for mapping the memory window
- * @align: OUT - the base alignment for translating the memory window
- * @align_size: OUT - the size alignment for translating the memory window
*
- * Get the range of a memory window. NULL may be given for any output
- * parameter if the value is not needed. The base and size may be used for
- * mapping the memory window, to access the peer memory. The alignment and
- * size may be used for translating the memory window, for the peer to access
- * memory on the local system.
+ * Get the map range of a memory window. The base and size may be used for
+ * mapping the memory window to access the peer memory.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base, resource_size_t *size,
- resource_size_t *align, resource_size_t *align_size)
+static inline int ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size)
{
- return ntb->ops->mw_get_range(ntb, idx, base, size,
- align, align_size);
+ return ntb->ops->mw_get_maprsc(ntb, idx, base, size);
+}
+
+/**
+ * ntb_mw_get_align() - get memory window alignment of the local node
+ * @ntb: NTB device context.
+ * @idx: Memory window number.
+ * @addr_align: OUT - the translated base address alignment of the memory window
+ * @size_align: OUT - the translated memory size alignment of the memory window
+ * @size_max: OUT - the translated memory maximum size
+ *
+ * Get the alignment parameters to allocate the proper memory window. NULL may
+ * be given for any output parameter if the value is not needed.
+ *
+ * Drivers of synchronous hardware don't have to support it.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ if (!ntb->ops->mw_get_align)
+ return -EINVAL;
+
+ return ntb->ops->mw_get_align(ntb, idx, addr_align, size_align, size_max);
}
/**
- * ntb_mw_set_trans() - set the translation of a memory window
+ * ntb_mw_set_trans() - set the translated base address of a peer memory window
* @ntb: NTB device context.
* @idx: Memory window number.
- * @addr: The dma address local memory to expose to the peer.
- * @size: The size of the local memory to expose to the peer.
+ * @addr: DMA memory address exposed by the peer.
+ * @size: Size of the memory exposed by the peer.
+ *
+ * Set the translated base address of a memory window. The peer preliminary
+ * allocates a memory, then someway passes the address to the remote node, that
+ * finally sets up the memory window at the address, up to the size. The address
+ * and size must be aligned to the parameters specified by ntb_mw_get_align() of
+ * the local node and ntb_peer_mw_get_align() of the peer, which must return the
+ * same values. Zero size effectively disables the memory window.
*
- * Set the translation of a memory window. The peer may access local memory
- * through the window starting at the address, up to the size. The address
- * must be aligned to the alignment specified by ntb_mw_get_range(). The size
- * must be aligned to the size alignment specified by ntb_mw_get_range().
+ * Drivers of synchronous hardware don't have to support it.
*
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
dma_addr_t addr, resource_size_t size)
{
+ if (!ntb->ops->mw_set_trans)
+ return -EINVAL;
+
return ntb->ops->mw_set_trans(ntb, idx, addr, size);
}
/**
- * ntb_mw_clear_trans() - clear the translation of a memory window
+ * ntb_mw_get_trans() - get the translated base address of a memory window
* @ntb: NTB device context.
* @idx: Memory window number.
+ * @addr: The dma memory address exposed by the peer.
+ * @size: The size of the memory exposed by the peer.
*
- * Clear the translation of a memory window. The peer may no longer access
- * local memory through the window.
+ * Get the translated base address of a memory window spicified for the local
+ * hardware and allocated by the peer. If the addr and size are zero, the
+ * memory window is effectively disabled.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int idx)
+static inline int ntb_mw_get_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size)
{
- if (!ntb->ops->mw_clear_trans)
- return ntb->ops->mw_set_trans(ntb, idx, 0, 0);
+ if (!ntb->ops->mw_get_trans)
+ return -EINVAL;
- return ntb->ops->mw_clear_trans(ntb, idx);
+ return ntb->ops->mw_get_trans(ntb, idx, addr, size);
}
/**
- * ntb_link_is_up() - get the current ntb link state
+ * ntb_peer_mw_count() - get the number of peer memory windows
* @ntb: NTB device context.
- * @speed: OUT - The link speed expressed as PCIe generation number.
- * @width: OUT - The link width expressed as the number of PCIe lanes.
*
- * Get the current state of the ntb link. It is recommended to query the link
- * state once after every link event. It is safe to query the link state in
- * the context of the link event callback.
+ * Hardware and topology may support a different number of memory windows at
+ * local and remote nodes.
*
- * Return: One if the link is up, zero if the link is down, otherwise a
- * negative value indicating the error number.
+ * Return: the number of memory windows.
*/
-static inline int ntb_link_is_up(struct ntb_dev *ntb,
- enum ntb_speed *speed, enum ntb_width *width)
+static inline int ntb_peer_mw_count(struct ntb_dev *ntb)
{
- return ntb->ops->link_is_up(ntb, speed, width);
+ return ntb->ops->peer_mw_count(ntb);
}
/**
- * ntb_link_enable() - enable the link on the secondary side of the ntb
+ * ntb_peer_mw_get_align() - get memory window alignment of the peer
* @ntb: NTB device context.
- * @max_speed: The maximum link speed expressed as PCIe generation number.
- * @max_width: The maximum link width expressed as the number of PCIe lanes.
+ * @idx: Memory window number.
+ * @addr_align: OUT - the translated base address alignment of the memory window
+ * @size_align: OUT - the translated memory size alignment of the memory window
+ * @size_max: OUT - the translated memory maximum size
*
- * Enable the link on the secondary side of the ntb. This can only be done
- * from the primary side of the ntb in primary or b2b topology. The ntb device
- * should train the link to its maximum speed and width, or the requested speed
- * and width, whichever is smaller, if supported.
+ * Get the alignment parameters to allocate the proper memory window for the
+ * peer. NULL may be given for any output parameter if the value is not needed.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_link_enable(struct ntb_dev *ntb,
- enum ntb_speed max_speed,
- enum ntb_width max_width)
+static inline int ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
{
- return ntb->ops->link_enable(ntb, max_speed, max_width);
+ if (!ntb->ops->peer_mw_get_align)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_get_align(ntb, idx, addr_align, size_align,
+ size_max);
}
/**
- * ntb_link_disable() - disable the link on the secondary side of the ntb
+ * ntb_peer_mw_set_trans() - set the translated base address of a peer
+ * memory window
* @ntb: NTB device context.
+ * @idx: Memory window number.
+ * @addr: Local DMA memory address exposed to the peer.
+ * @size: Size of the memory exposed to the peer.
*
- * Disable the link on the secondary side of the ntb. This can only be
- * done from the primary side of the ntb in primary or b2b topology. The ntb
- * device should disable the link. Returning from this call must indicate that
- * a barrier has passed, though with no more writes may pass in either
- * direction across the link, except if this call returns an error number.
+ * Set the translated base address of a memory window exposed to the peer.
+ * The local node preliminary allocates the window, then directly writes the
+ * address and size to the peer control registers. The address and size must
+ * be aligned to the parameters specified by ntb_peer_mw_get_align() of
+ * the local node and ntb_mw_get_align() of the peer, which must return the
+ * same values. Zero size effectively disables the memory window.
+ *
+ * Drivers of synchronous hardware must support it.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_link_disable(struct ntb_dev *ntb)
+static inline int ntb_peer_mw_set_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size)
{
- return ntb->ops->link_disable(ntb);
+ if (!ntb->ops->peer_mw_set_trans)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_set_trans(ntb, idx, addr, size);
+}
+
+/**
+ * ntb_peer_mw_get_trans() - get the translated base address of a peer
+ * memory window
+ * @ntb: NTB device context.
+ * @idx: Memory window number.
+ * @addr: Local dma memory address exposed to the peer.
+ * @size: Size of the memory exposed to the peer.
+ *
+ * Get the translated base address of a memory window spicified for the peer
+ * hardware. If the addr and size are zero then the memory window is effectively
+ * disabled.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_peer_mw_get_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size)
+{
+ if (!ntb->ops->peer_mw_get_trans)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_get_trans(ntb, idx, addr, size);
}
/**
@@ -751,6 +1053,8 @@ static inline int ntb_db_clear_mask(struct ntb_dev *ntb, u64 db_bits)
* append one additional dma memory copy with the doorbell register as the
* destination, after the memory copy operations.
*
+ * This is unusual, and hardware may not be suitable to implement it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_peer_db_addr(struct ntb_dev *ntb,
@@ -901,10 +1205,15 @@ static inline int ntb_spad_is_unsafe(struct ntb_dev *ntb)
*
* Hardware and topology may support a different number of scratchpads.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: the number of scratchpads.
*/
static inline int ntb_spad_count(struct ntb_dev *ntb)
{
+ if (!ntb->ops->spad_count)
+ return -EINVAL;
+
return ntb->ops->spad_count(ntb);
}
@@ -915,10 +1224,15 @@ static inline int ntb_spad_count(struct ntb_dev *ntb)
*
* Read the local scratchpad register, and return the value.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: The value of the local scratchpad register.
*/
static inline u32 ntb_spad_read(struct ntb_dev *ntb, int idx)
{
+ if (!ntb->ops->spad_read)
+ return 0;
+
return ntb->ops->spad_read(ntb, idx);
}
@@ -930,10 +1244,15 @@ static inline u32 ntb_spad_read(struct ntb_dev *ntb, int idx)
*
* Write the value to the local scratchpad register.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_spad_write(struct ntb_dev *ntb, int idx, u32 val)
{
+ if (!ntb->ops->spad_write)
+ return -EINVAL;
+
return ntb->ops->spad_write(ntb, idx, val);
}
@@ -946,6 +1265,8 @@ static inline int ntb_spad_write(struct ntb_dev *ntb, int idx, u32 val)
* Return the address of the peer doorbell register. This may be used, for
* example, by drivers that offload memory copy operations to a dma engine.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_peer_spad_addr(struct ntb_dev *ntb, int idx,
@@ -964,10 +1285,15 @@ static inline int ntb_peer_spad_addr(struct ntb_dev *ntb, int idx,
*
* Read the peer scratchpad register, and return the value.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: The value of the local scratchpad register.
*/
static inline u32 ntb_peer_spad_read(struct ntb_dev *ntb, int idx)
{
+ if (!ntb->ops->peer_spad_read)
+ return 0;
+
return ntb->ops->peer_spad_read(ntb, idx);
}
@@ -979,11 +1305,59 @@ static inline u32 ntb_peer_spad_read(struct ntb_dev *ntb, int idx)
*
* Write the value to the peer scratchpad register.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_peer_spad_write(struct ntb_dev *ntb, int idx, u32 val)
{
+ if (!ntb->ops->peer_spad_write)
+ return -EINVAL;
+
return ntb->ops->peer_spad_write(ntb, idx, val);
}
+/**
+ * ntb_msg_post() - post the message to the peer
+ * @ntb: NTB device context.
+ * @msg: Message
+ *
+ * Post the message to a peer. It shall be delivered to the peer by the
+ * corresponding hardware method. The peer should be notified about the new
+ * message by calling the ntb_msg_event() handler of NTB_MSG_NEW event type.
+ * If delivery is fails for some reasong the local node will get NTB_MSG_FAIL
+ * event. Otherwise the NTB_MSG_SENT is emitted.
+ *
+ * Synchronous hardware may not support it.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_msg_post(struct ntb_dev *ntb, struct ntb_msg *msg)
+{
+ if (!ntb->ops->msg_post)
+ return -EINVAL;
+
+ return ntb->ops->msg_post(ntb, msg);
+}
+
+/**
+ * ntb_msg_size() - size of the message data
+ * @ntb: NTB device context.
+ *
+ * Different hardware may support different number of message registers. This
+ * callback shall return the number of those used for data sending and
+ * receiving including the type field.
+ *
+ * Synchronous hardware may not support it.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_msg_size(struct ntb_dev *ntb)
+{
+ if (!ntb->ops->msg_size)
+ return 0;
+
+ return ntb->ops->msg_size(ntb);
+}
+
#endif
--
2.6.6
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver
2016-07-26 19:50 [PATCH 0/3] ntb: Asynchronous NTB devices support Serge Semin
2016-07-26 19:50 ` [PATCH 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
@ 2016-07-26 19:50 ` Serge Semin
2016-07-27 16:29 ` kbuild test robot
2016-07-26 19:50 ` [PATCH 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
2016-07-28 10:01 ` [PATCH v2 0/3] ntb: Asynchronous NTB devices support Serge Semin
3 siblings, 1 reply; 11+ messages in thread
From: Serge Semin @ 2016-07-26 19:50 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
IDT 89HPES*NT* PCIe-switches are the multi-port switches, which can be
configured to have more than two NT-functions, connected to each other.
Using these facility NT-function of local RC can access any NT-functions of
others. This driver implements Port-to-Ports architecture, so only Primary
port can have access to prededefined number of Secondary ports. Secondary
ports on the other hand can access the primary port memory only. The ports
hierarchy is configured using NTSDATA register of each NT-function, which
should be preinitialized with corresponding Primary port ID. Then driver will
evenly split the IDT PCIe-switch Doorbell and Memory Window resources between
all of the NT-function pairs, and add the corresponding devices to the NTB bus.
The NTSDATA register preinitialization can be done either by EEPROM or by linux
kernel PCI fixup interface. The default value is zero, so the port 0 is
considered being primary, the others are secondary.
The supported IDT PCIe-switches do not implement synchronous interface to
access Memory Windows configuration registers. There is no way to exclusively
access a peer translated base addresses Lookup Table. Instead the messaging
subsystem can be used to post the translated base address to a peer, so one
could properly initialize the table. In order to resolve the race conditions of
possible concurrent messages transfer by parallel tasks and different root
complexes, the ntb_msg_post() method places messages into the outbound messages
queue. An extra kernel work thread is used to perform an actual message
transfer with a constant frequency of attempts. If kernel thread fails to send
a message after a constant number of tries, then the corresponding client
driver is notified by rising the ntb_msg_event() method with NTB_MSG_FAIL
status. Additionaly there is a special tasklet, which reads messages from
incoming message registers and puts them into the inbound messages queue.
An another kernel thread is then started to deliver a message to a
corresponding client driver.
There are only 32 Global doorbell bits, which can be routed between any two
NT-functions. So the driver evenly distributes doorbell bits between all the
NT-function pairs. Of course it is not always possible being equally done, so
depending on the NT Primary/Secondary topology, one pair can have greater
number of Doorbell bits than another. The difference cannot exceed for more
than one bit.
IDT 89HPES*NT* PCIe-switches don't have Scrathpad registers, so the
corresponding callback methods are unsupported.
There are three DebugFS nodes, which can be used to debug the driver:
1. /sys/kernel/debug/ntb_hw_idt/ntbA{N}/info - contain a general information
about the local IDT NTB driver and NT-function, like port, partition, role
(primary or secondary) and number of connected peers, a link status, of all
the locally available pairs, global and local doorbell registers status,
doorbells mapping, NTB mapping table, chip teperature and many others.
2. /sys/kernel/debug/ntb_hw_idt/ntbA{N}/ntregs - debug file to print the state
of local NT-function registers. Thanks to the "enum-switch-case-macro" pattern
it can be easily done just by one loop.
3. /sys/kernel/debug/ntb_hw_idt/ntbA{N}/swregs - debug file to print the state
of PCIe-swtich global registers.
The detailed description of the driver design and source code navigation
can be found in the header of drivers/ntb/hw/idt/ntb_hw_idt.c file.
NOTE IDT 89HPES12NT3 PCIe-switch is not supported by this driver, since it has
synchronous interface.
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
drivers/ntb/hw/Kconfig | 1 +
drivers/ntb/hw/Makefile | 6 +-
drivers/ntb/hw/idt/Kconfig | 21 +
drivers/ntb/hw/idt/Makefile | 5 +
drivers/ntb/hw/idt/ntb_hw_idt.c | 4050 ++++++++++++++++++++++++++++++++
drivers/ntb/hw/idt/ntb_hw_idt.h | 390 +++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.c | 163 ++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.h | 114 +
drivers/ntb/hw/idt/ntb_hw_idt_regmap.h | 877 +++++++
9 files changed, 5625 insertions(+), 2 deletions(-)
create mode 100644 drivers/ntb/hw/idt/Kconfig
create mode 100644 drivers/ntb/hw/idt/Makefile
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
diff --git a/drivers/ntb/hw/Kconfig b/drivers/ntb/hw/Kconfig
index 7116472..a89243c 100644
--- a/drivers/ntb/hw/Kconfig
+++ b/drivers/ntb/hw/Kconfig
@@ -1,2 +1,3 @@
source "drivers/ntb/hw/amd/Kconfig"
+source "drivers/ntb/hw/idt/Kconfig"
source "drivers/ntb/hw/intel/Kconfig"
diff --git a/drivers/ntb/hw/Makefile b/drivers/ntb/hw/Makefile
index 532e085..5d438ea 100644
--- a/drivers/ntb/hw/Makefile
+++ b/drivers/ntb/hw/Makefile
@@ -1,2 +1,4 @@
-obj-$(CONFIG_NTB_AMD) += amd/
-obj-$(CONFIG_NTB_INTEL) += intel/
+obj-$(CONFIG_NTB_AMD) += amd/
+obj-$(CONFIG_NTB_IDT) += idt/
+obj-$(CONFIG_NTB_IDT_QUIRKS) += idt/
+obj-$(CONFIG_NTB_INTEL) += intel/
diff --git a/drivers/ntb/hw/idt/Kconfig b/drivers/ntb/hw/idt/Kconfig
new file mode 100644
index 0000000..45b2b01
--- /dev/null
+++ b/drivers/ntb/hw/idt/Kconfig
@@ -0,0 +1,21 @@
+config NTB_IDT
+ tristate "IDT PCIe-switch Non-Transparent Bridge support"
+ depends on PCI
+ help
+ This driver supports NTB of cappable IDT PCIe-switches.
+
+ If unsure, say N.
+
+# BAR's early enable quirks
+config NTB_IDT_QUIRKS
+ bool "Enable PCI fixups built-in kernel for IDT PCIe-switch"
+ depends on NTB_IDT
+ default y if NTB_IDT=y
+ select PCI_QUIRKS # NTB_IDT depends on PCI so it's ok to force quirks
+ help
+ There are some configurations of IDT PCIe-switch which must be made
+ before the kernel PCI subsystem starts bus-devices scan and
+ initialization. For instance, the corresponding NT-function BARs must
+ be enabled. This option builds the fixups into the kernel to bypass
+ the basic IDT quirks.
+
diff --git a/drivers/ntb/hw/idt/Makefile b/drivers/ntb/hw/idt/Makefile
new file mode 100644
index 0000000..3adcd9b
--- /dev/null
+++ b/drivers/ntb/hw/idt/Makefile
@@ -0,0 +1,5 @@
+# Driver core
+obj-$(CONFIG_NTB_IDT) += ntb_hw_idt.o
+
+# BAR's early activation quirk
+obj-$(CONFIG_NTB_IDT_QUIRKS) += ntb_hw_idt_quirks.o
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt.c b/drivers/ntb/hw/idt/ntb_hw_idt.c
new file mode 100644
index 0000000..0db1a4a
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt.c
@@ -0,0 +1,4050 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the IDT PCIe-switch NT-function driver design.
+ * Here is presented some lirics about the NT-functions of the IDT PCIe-switch
+ * and the driver concept.
+ *
+ * There are a lot of different architectures or configurations the IDT
+ * PCIe-switch can be placed into, like NT Bridge-to-Bridge, Port-to-Port,
+ * Ports-to-Ports, Port-to-Ports, etc. But there is always BUT! Here it is.
+ * But the problem is that the PCIe-switch resources are not balanced enough
+ * to create efficient, the most comprehensive driver for Ports-to-Ports
+ * configuration. Here is what each IDT PCIe-switch have (IDT family of
+ * PCIe-switch solutions):
+ * - up to 24 Memory Windows per each port (incredibly a lot comparing to the
+ * Intel and AMD controllers)
+ * - one 32 bits Doorbell register shared amongst all the ports (Why IDT, why
+ * would you do that? Why so few?!)
+ * - 4 Message registers per each port (IDT, thanks at least for that...)
+ * - No Scratchpad registers (Surprise, huh?!)
+ *
+ * Since there are no scratchpad registers integrated in the IDT family PCI
+ * ExpressR switches, the tradition synchronous Linux NTB device can't be
+ * implemented (unlike Intel and AMD controllers, that are strictly synchronous).
+ * Instead the Messaging mechanism should be used to exchange the necessary
+ * informatin among the NT-functions. It leads to the asynchronous
+ * interface since there is no easy way to pass the address of the locally
+ * allocated shared memory window to the opposite NT-function. It can only be
+ * done by sending a message, which must be correcly handled by a peer. If one
+ * is looking for strictly synchronous solutions, then it's better to use Intel
+ * and AMD controllers. Regarding the IDT PreciseTM family of PCI ExpressR
+ * switches, they actually support both synchronous (scratchpads) and
+ * asynchronous (message registers) interfaces, but there is no suitable driver
+ * to use them in Linux.
+ *
+ * Lets get back to the actual driver architecture. Since there are no enough
+ * doorbell registers and after a lot of thoughts of the possible sidewalks to
+ * bypass the PCIe-switch limitations we came to the conclusion, that the best
+ * architecture of the driver using as much resources as possible would be the
+ * Port-to-Port/Port-to-Ports one. Shortly speaking it implies the only one
+ * NT-function being able to communicate with all the other NT-functions
+ * simultaniously. Suppose there are eight ports working as NT-bridge, then the
+ * Primary port would have 7 devices on the NTB bus, but the Secondary ports
+ * will expose just one device. As one can see it also perfectly fits the
+ * Primary-Secondary topology of the Linux NTB bus. The NTSDATA registers must
+ * be preinitialized with the corresponding Primary side port numbers. It is the
+ * way how the NTB topology can be configurated. For instance, suppose there are
+ * only two NT-functions enabled on the IDT PCIe-switch ports 0 and 2, where
+ * port 2 is chosen to be the primary one. Then all NTSDATA of the both
+ * NT-functions must be preinitialized with value 2. Similarly the topology with
+ * several Primary ports can be created.
+ *
+ * Howto study the code below.
+ * Here is the content of the driver:
+ * 1. IDT PCIe-switch registers IO-functions
+ * 2. Synchronization methods: atomic queue ops
+ * 3. Link status operations
+ * 4. Memory Window subsystem
+ * 5. Doorbells subsystem
+ * 6. Messaging subsystem
+ * 7. IRQ-related functions
+ * 8. NTB bus initialization
+ * 9. IDT NT-functions topology
+ * 10. Basic initialization functions
+ * 11. DebugFS callback functions
+ * 12. PCI bus callback functions
+ *
+ * I would recommend to start from the chapter "1. IDT PCIe-switch registers
+ * IO-functions". Since there are a lot of registers must be initialized before
+ * the switch starts working, it's better to have the register addresses and
+ * the corresponding values being collected at some structured table.
+ * Particulary one can find these tables in ntb_hw_idt_regmap.h file as the set
+ * of preprocessor macro-functions. Regarding the chapter 1 in this file, it
+ * resides the basic functions used to create the NT-functions and Switch Global
+ * registers table and the registers fields table. There are also r/w functions
+ * determined in there.
+ *
+ * Since there are list structures used to handle in and out messages, then
+ * there has to be managed synchronous access to those lists. Therefore the
+ * operations with message queues are made atomic in chapter "2. Synchronization
+ * methods: atomic queue ops".
+ *
+ * Then I would get stright to the chapter "12. PCI bus callback functions",
+ * which perform the algorithm of the PCI-bus device basic initialzation.
+ * Particulary it checks whether the kernel supports IDT PCIe-switch NTB
+ * devices, allocates the necessary structures, initialize the PCI-related
+ * fields, scans the IDT PCIe-switch NT-functions topology, adds all the
+ * available peers, initalizes doorbells, memory windows and messaging
+ * subsystem, starts link polling work-thread, initialize the interrupt
+ * handlers and finally registers the NTB devices on the NTB linux kernel bus.
+ *
+ * The basic PCI-bus device initialization and data structures allocation are
+ * performed by means of methods defined in the chapter "10. Basic
+ * initialization functions". NTB topology scanning is made by function from
+ * the chapter "9. IDT NT-functions topology".
+ *
+ * The NTB basic interfaces like Link event handlers, memory windows, doorbells
+ * and messages subsystems are described in the chapters 3 - 6 with corresponding
+ * titles. They traditionally consist of helpers, initializing/deinitializing
+ * functions and particular NTB devices kernel driver methods. These kernel
+ * driver methods - are callback functions used to register the new devices on
+ * the linux kernel NTB bus defined in the chapter "8. NTB bus initialization".
+ */
+
+/*#define DEBUG*/
+
+#include <linux/stddef.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/bitops.h>
+#include <linux/sizes.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/init.h>
+#include <linux/ntb.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/pci.h>
+#include <linux/aer.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/debugfs.h>
+
+#include "ntb_hw_idt.h"
+#include "ntb_hw_idt_regmap.h"
+#include "ntb_hw_idt_quirks.h"
+
+#define NTB_NAME "ntb_hw_idt"
+#define NTB_DESC "IDT PCI-E Non-Transparent Bridge Driver"
+#define NTB_VER "1.0"
+#define NTB_IRQNAME "idt_ntb_irq"
+#define NTB_WQNAME "idt_ntb_wq"
+#define NTB_CACHENAME "idt_ntb_cache"
+
+MODULE_DESCRIPTION(NTB_DESC);
+MODULE_VERSION(NTB_VER);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_ndev(ndev, args...) \
+ dev_err(to_dev_ndev(ndev), ## args)
+#define dev_err_data(data, args...) \
+ dev_err(to_dev_data(data), ## args)
+#define dev_warn_ndev(ndev, args...) \
+ dev_warn(to_dev_ndev(ndev), ## args)
+#define dev_warn_data(data, args...) \
+ dev_warn(to_dev_data(data), ## args)
+#define dev_info_ndev(ndev, args...) \
+ dev_info(to_dev_ndev(ndev), ## args)
+#define dev_info_data(data, args...) \
+ dev_info(to_dev_data(data), ## args)
+#define dev_dbg_ndev(ndev, args...) \
+ dev_dbg(to_dev_ndev(ndev), ## args)
+#define dev_dbg_data(data, args...) \
+ dev_dbg(to_dev_data(data), ## args)
+
+/*
+ * NT Endpoint ports table with the corresponding pcie link status, signal data,
+ * control and status registers
+ */
+static struct idt_ntb_port portdata_tbl[IDT_NTB_MAXPORTS_CNT] = {
+/*0*/ {IDT_SW_PCI_NTP0_CMD, IDT_SW_PCI_NTP0_PCIELSTS,
+ IDT_SW_PCI_NTP0_NTSDATA, IDT_SW_PCI_NTP0_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT0CTL, IDT_SW_PCI_SWPORT0STS},
+/*1*/ {0},
+/*2*/ {IDT_SW_PCI_NTP2_CMD, IDT_SW_PCI_NTP2_PCIELSTS,
+ IDT_SW_PCI_NTP2_NTSDATA, IDT_SW_PCI_NTP2_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT2CTL, IDT_SW_PCI_SWPORT2STS},
+/*3*/ {0},
+/*4*/ {IDT_SW_PCI_NTP4_CMD, IDT_SW_PCI_NTP4_PCIELSTS,
+ IDT_SW_PCI_NTP4_NTSDATA, IDT_SW_PCI_NTP4_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT4CTL, IDT_SW_PCI_SWPORT4STS},
+/*5*/ {0},
+/*6*/ {IDT_SW_PCI_NTP6_CMD, IDT_SW_PCI_NTP6_PCIELSTS,
+ IDT_SW_PCI_NTP6_NTSDATA, IDT_SW_PCI_NTP6_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT6CTL, IDT_SW_PCI_SWPORT6STS},
+/*7*/ {0},
+/*8*/ {IDT_SW_PCI_NTP8_CMD, IDT_SW_PCI_NTP8_PCIELSTS,
+ IDT_SW_PCI_NTP8_NTSDATA, IDT_SW_PCI_NTP8_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT8CTL, IDT_SW_PCI_SWPORT8STS},
+/*9*/ {0},
+/*10*/ {0},
+/*11*/ {0},
+/*12*/ {IDT_SW_PCI_NTP12_CMD, IDT_SW_PCI_NTP12_PCIELSTS,
+ IDT_SW_PCI_NTP12_NTSDATA, IDT_SW_PCI_NTP12_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT12CTL, IDT_SW_PCI_SWPORT12STS},
+/*13*/ {0},
+/*14*/ {0},
+/*15*/ {0},
+/*16*/ {IDT_SW_PCI_NTP16_CMD, IDT_SW_PCI_NTP16_PCIELSTS,
+ IDT_SW_PCI_NTP16_NTSDATA, IDT_SW_PCI_NTP16_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT16CTL, IDT_SW_PCI_SWPORT16STS},
+/*17*/ {0},
+/*18*/ {0},
+/*19*/ {0},
+/*20*/ {IDT_SW_PCI_NTP20_CMD, IDT_SW_PCI_NTP20_PCIELSTS,
+ IDT_SW_PCI_NTP20_NTSDATA, IDT_SW_PCI_NTP20_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT20CTL, IDT_SW_PCI_SWPORT20STS},
+/*21*/ {0},
+/*22*/ {0},
+/*23*/ {0}
+};
+
+/*
+ * IDT PCIe-switch partitions table with the corresponding control, status
+ * and messages control registers
+ */
+static struct idt_ntb_part partdata_tbl[IDT_NTB_MAXPARTS_CNT] = {
+/*0*/ { IDT_SW_PCI_SWPART0CTL, IDT_SW_PCI_SWPART0STS,
+ {IDT_SW_PCI_SWP0MSGCTL0, IDT_SW_PCI_SWP0MSGCTL1,
+ IDT_SW_PCI_SWP0MSGCTL2, IDT_SW_PCI_SWP0MSGCTL3} },
+/*1*/ { IDT_SW_PCI_SWPART1CTL, IDT_SW_PCI_SWPART1STS,
+ {IDT_SW_PCI_SWP1MSGCTL0, IDT_SW_PCI_SWP1MSGCTL1,
+ IDT_SW_PCI_SWP1MSGCTL2, IDT_SW_PCI_SWP1MSGCTL3} },
+/*2*/ { IDT_SW_PCI_SWPART2CTL, IDT_SW_PCI_SWPART2STS,
+ {IDT_SW_PCI_SWP2MSGCTL0, IDT_SW_PCI_SWP2MSGCTL1,
+ IDT_SW_PCI_SWP2MSGCTL2, IDT_SW_PCI_SWP2MSGCTL3} },
+/*3*/ { IDT_SW_PCI_SWPART3CTL, IDT_SW_PCI_SWPART3STS,
+ {IDT_SW_PCI_SWP3MSGCTL0, IDT_SW_PCI_SWP3MSGCTL1,
+ IDT_SW_PCI_SWP3MSGCTL2, IDT_SW_PCI_SWP3MSGCTL3} },
+/*4*/ { IDT_SW_PCI_SWPART4CTL, IDT_SW_PCI_SWPART4STS,
+ {IDT_SW_PCI_SWP4MSGCTL0, IDT_SW_PCI_SWP4MSGCTL1,
+ IDT_SW_PCI_SWP4MSGCTL2, IDT_SW_PCI_SWP4MSGCTL3} },
+/*5*/ { IDT_SW_PCI_SWPART5CTL, IDT_SW_PCI_SWPART5STS,
+ {IDT_SW_PCI_SWP5MSGCTL0, IDT_SW_PCI_SWP5MSGCTL1,
+ IDT_SW_PCI_SWP5MSGCTL2, IDT_SW_PCI_SWP5MSGCTL3} },
+/*6*/ { IDT_SW_PCI_SWPART6CTL, IDT_SW_PCI_SWPART6STS,
+ {IDT_SW_PCI_SWP6MSGCTL0, IDT_SW_PCI_SWP6MSGCTL1,
+ IDT_SW_PCI_SWP6MSGCTL2, IDT_SW_PCI_SWP6MSGCTL3} },
+/*7*/ { IDT_SW_PCI_SWPART7CTL, IDT_SW_PCI_SWPART7STS,
+ {IDT_SW_PCI_SWP7MSGCTL0, IDT_SW_PCI_SWP7MSGCTL1,
+ IDT_SW_PCI_SWP7MSGCTL2, IDT_SW_PCI_SWP7MSGCTL3} }
+};
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_topdir;
+
+/*===========================================================================
+ * 1. IDT PCIe-switch registers IO-functions
+ *===========================================================================*/
+
+static void __idt_nt_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val);
+static u32 __idt_nt_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize);
+static void __idt_sw_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val);
+static u32 __idt_sw_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize);
+
+/*
+ * Registers IO contexts to perform the r/w operations either with NT-function
+ * registers or with the PCIe-switch Global registers. The context is chosen
+ * by the register type "enum idt_ntb_regtype"
+ */
+static struct idt_ntb_regctx regctx[2] = {
+ {.writereg = __idt_nt_writereg, .readreg = __idt_nt_readreg,
+ .iolock = __SPIN_LOCK_UNLOCKED(iolock)},
+ {.writereg = __idt_sw_writereg, .readreg = __idt_sw_readreg,
+ .iolock = __SPIN_LOCK_UNLOCKED(iolock)}
+};
+
+/*
+ * Internal function to set the value bits of a variable
+ */
+static inline u32 idt_ntb_setbits(u32 var, u32 mask, unsigned char offset, u32 val)
+{
+ return (var & ~(mask << offset)) | ((val & mask) << offset);
+}
+
+/*
+ * Internal function to retrieve the value bits of a variable
+ */
+static inline u32 idt_ntb_getbits(u32 var, u32 mask, unsigned char offset)
+{
+ return (var >> offset) & mask;
+}
+
+/*
+ * Retrieve the register type, address and size by the passed enumerated ID
+ * NOTE Compiler should produce the jump table for the subsequent switch-case
+ * statement which gives just simple o(1) complexity
+ */
+static int idt_ntb_regparams(const enum idt_ntb_cfgreg reg,
+ enum idt_ntb_regtype *type, ptrdiff_t *offset,
+ enum idt_ntb_regsize *size, const char **desc)
+{
+ const char *tmpdesc;
+
+ /* Determine the register type */
+ *type = (IDT_NTB_CFGREGS_SPLIT > reg) ? IDT_NT_REGTYPE : IDT_SW_REGTYPE;
+
+ /* Retrieve the register parameters by the enumerated ID */
+ switch (reg) {
+ IDT_NT_CFGREGS(PAIR_REGID_ACCESS, *offset, *size, tmpdesc)
+ IDT_SW_CFGREGS(PAIR_REGID_ACCESS, *offset, *size, tmpdesc)
+ default :
+ /* Got invalid register ID */
+ BUG();
+ return -EINVAL;
+ }
+
+ /* Return the pointer to the string with the register description
+ * only if the passed pointer isn't NULL*/
+ if (NULL != desc) {
+ *desc = tmpdesc;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Retrieve the registers fields parameters like the register id, mask
+ * and offset
+ * NOTE Compiler should produce the jump table for the subsequent switch-case
+ * statement which gives just simple o(1) complexity
+ */
+static int idt_ntb_fldparams(const enum idt_ntb_regfld fld,
+ enum idt_ntb_cfgreg *reg,
+ u32 *mask, unsigned char *offset)
+{
+ /* Retrieve the field parameters by the enumerated ID */
+ switch (fld) {
+ IDT_NT_REGFLDS(PAIR_FLDID_ACCESS, *reg, *mask, *offset)
+ IDT_SW_REGFLDS(PAIR_FLDID_ACCESS, *reg, *mask, *offset)
+ default :
+ /* Got invalid register field ID */
+ BUG();
+ return -EINVAL;
+ }
+ return SUCCESS;
+}
+
+/*
+ * Set the corresponding field of the passed variable
+ */
+static void idt_ntb_writefld_var(u32 *var, const enum idt_ntb_regfld fld,
+ const u32 val)
+{
+ enum idt_ntb_cfgreg reg;
+ unsigned char bitoffset;
+ u32 mask;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Init the corresponding bits of the passed variable */
+ *var = idt_ntb_setbits(*var, mask, bitoffset, val);
+}
+
+/*
+ * Get the corresponding field of the passed variable
+ */
+static u32 idt_ntb_readfld_var(u32 var, const enum idt_ntb_regfld fld)
+{
+ enum idt_ntb_cfgreg reg;
+ unsigned char bitoffset;
+ u32 mask;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Retrieve the corresponding field of the variable */
+ return idt_ntb_getbits(var, mask, bitoffset);
+}
+
+/*
+ * NT-function registers basic write function
+ *
+ * WARNING! Our target platform is Big Endian, but PCI registers are always
+ * Little endian. So corresponding write{w,l} operations must have
+ * embedded endiannes conversion. If your platform doesn't have it,
+ * the driver won't properly work.
+ */
+static void __idt_nt_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val)
+{
+ /* Perform fast IO operation */
+ switch (regsize) {
+ case REGBYTE:
+ writeb((u8)val, cfg_mmio + regoffset);
+ break;
+ case REGWORD:
+ writew((u16)val, cfg_mmio + regoffset);
+ break;
+ case REGDWORD:
+ writel((u32)val, cfg_mmio + regoffset);
+ break;
+ default:
+ /* Invalid register size was retrieved */
+ BUG();
+ break;
+ }
+}
+
+/*
+ * NT-function registers basic read function
+ *
+ * WARNING! Our target platform is Big Endian, but PCI registers are always
+ * Little endian. So corresponding read{w,l} operations must have
+ * embedded endiannes conversion. If your platform doesn't have it,
+ * the driver won't properly work.
+ */
+static u32 __idt_nt_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize)
+{
+ u32 retval;
+
+ /* Perform fast IO operation */
+ switch (regsize) {
+ case REGBYTE:
+ retval = readb(cfg_mmio + regoffset);
+ break;
+ case REGWORD:
+ retval = readw(cfg_mmio + regoffset);
+ break;
+ case REGDWORD:
+ retval = readl(cfg_mmio + regoffset);
+ break;
+ default:
+ /* Invalid register size was retrieved */
+ BUG();
+ break;
+ }
+
+ return retval;
+}
+
+/*
+ * IDT PCIe-switch Global registers basic write function
+ *
+ * WARNING! Our target platform is Big Endian, but PCI registers are always
+ * Little endian. So corresponding write{w,l} operations must have
+ * embedded endiannes conversion. If your platform doesn't have it,
+ * the driver won't properly work.
+ * In addition the GASA* registers support the 4 bytes R/W operations
+ * so the data must be correspondingly shifted
+ */
+static void __idt_sw_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val)
+{
+ u32 data, fldmask;
+ unsigned char fldoffset;
+
+ /* Post the IDT PCIe-switch register offset first */
+ writel((u32)regoffset, cfg_mmio + GASAADDR_OFFSET);
+
+ /* Read the data of the passed register, which offset is aligned with
+ * two last bits by the GASAADDR register */
+ data = readl(cfg_mmio + GASADATA_OFFSET);
+
+ /* Alter the corresponding field of the data with the passed value */
+ fldmask = GENMASK(BITS_PER_BYTE * regsize - 1, 0);
+ fldoffset = BITS_PER_BYTE * (regoffset & 0x3);
+ data = idt_ntb_setbits(data, fldmask, fldoffset, val);
+
+ /* Whatever the size of the register is, just write the value to the
+ * data register */
+ writel(data, cfg_mmio + GASADATA_OFFSET);
+}
+
+/*
+ * IDT PCIe-switch Global registers basic read function
+ */
+static u32 __idt_sw_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize)
+{
+ u32 data, fldmask;
+ unsigned char fldoffset;
+
+ /* Post the IDT PCIe-switch register offset first */
+ writel((u32)regoffset, cfg_mmio + GASAADDR_OFFSET);
+
+ /* Read the data of the passed register, which offset is aligned with
+ * two last bits by the GASAADDR register */
+ data = readl(cfg_mmio + GASADATA_OFFSET);
+
+ /* Alter the corresponding field of the data with the passed value */
+ fldmask = GENMASK(BITS_PER_BYTE * regsize - 1, 0);
+ fldoffset = BITS_PER_BYTE * (regoffset & 0x3);
+ data = idt_ntb_getbits(data, fldmask, fldoffset);
+
+ /* Return the corresponding field of the register */
+ return data;
+}
+
+/*
+ * General function to perform the write operation to the register
+ */
+static void idt_ntb_writereg(void __iomem *cfg_mmio,
+ const enum idt_ntb_cfgreg reg, const u32 val)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_regtype regtype;
+ ptrdiff_t regoffset;
+ enum idt_ntb_regsize regsize;
+ unsigned long irqflags;
+
+ /* Retrieve the register type, offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast write operation */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ curctx->writereg(cfg_mmio, regoffset, regsize, val);
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+}
+
+/*
+ * General function to perform the read operation from the register
+ */
+static u32 idt_ntb_readreg(void __iomem *cfg_mmio, const enum idt_ntb_cfgreg reg)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_regtype regtype;
+ ptrdiff_t regoffset;
+ enum idt_ntb_regsize regsize;
+ unsigned long irqflags;
+ u32 val;
+
+ /* Retrieve the register type, offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast read operation */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ val = curctx->readreg(cfg_mmio, regoffset, regsize);
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+
+ return val;
+}
+
+/*
+ * General function to perform the write operation to the field of the register
+ */
+static void idt_ntb_writefld_mem(void __iomem *cfg_mmio,
+ const enum idt_ntb_regfld fld, const u32 val)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_cfgreg reg;
+ enum idt_ntb_regsize regsize;
+ ptrdiff_t regoffset;
+ unsigned char bitoffset;
+ u32 mask, curval;
+ enum idt_ntb_regtype regtype;
+ unsigned long irqflags;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Retrieve the register offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register set context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast IO operations */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ /* Retrieve the current value of the register */
+ curval = curctx->readreg(cfg_mmio, regoffset, regsize);
+ /* Set the corresponding bits in there */
+ curval = idt_ntb_setbits(curval, mask, bitoffset, val);
+ /* Write the register value back */
+ curctx->writereg(cfg_mmio, regoffset, regsize, val);
+ /* The critical section is over */
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+}
+
+/*
+ * General function to perform the read operation from the field of the register
+ */
+static u32 idt_ntb_readfld_mem(void __iomem *cfg_mmio,
+ const enum idt_ntb_regfld fld)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_cfgreg reg;
+ enum idt_ntb_regsize regsize;
+ ptrdiff_t regoffset;
+ unsigned char bitoffset;
+ u32 mask, curval;
+ enum idt_ntb_regtype regtype;
+ unsigned long irqflags;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Retrieve the register offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register set context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast IO operations */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ /* Retrieve the current value of the register */
+ curval = curctx->readreg(cfg_mmio, regoffset, regsize);
+ /* The critical section is over */
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+
+ return idt_ntb_getbits(curval, mask, bitoffset);
+}
+
+/*===========================================================================
+ * 2. Synchronization methods: atomic queue ops
+ *===========================================================================*/
+
+/*
+ * Initialize the atomic queue structure
+ */
+static inline void atomic_queue_init(queue_atomic_t *queue)
+{
+ /* Init the queue head */
+ INIT_LIST_HEAD(&queue->head);
+
+ /* Initialize the spin lock protecting the queue head */
+ spin_lock_init(&queue->lock);
+}
+
+/*
+ * Add item to the atomic queue at the first position
+ */
+static inline void atomic_queue_add(queue_atomic_t *queue,
+ struct list_head *new)
+{
+ unsigned long irqflags;
+
+ /* Lock the list add operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ list_add(new, &queue->head);
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+}
+
+/*
+ * Add item to the atomic queue tail
+ */
+static inline void atomic_queue_add_tail(queue_atomic_t *queue,
+ struct list_head *new)
+{
+ unsigned long irqflags;
+
+ /* Lock the list add tail operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ list_add_tail(new, &queue->head);
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+}
+
+/*
+ * Get the very first entry from the atomic queue
+ */
+static inline struct list_head *atomic_queue_get(queue_atomic_t *queue)
+{
+ struct list_head *entry;
+ unsigned long irqflags;
+
+ /* Lock the list entry delete operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ if (!list_empty(&queue->head)) {
+ entry = queue->head.next;
+ list_del(entry);
+ } else /* if (entry != &queue->head) */ {
+ entry = NULL;
+ }
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+
+ return entry;
+}
+
+/*
+ * Check whether the atomic queue is empty
+ */
+static inline bool atomic_queue_empty(queue_atomic_t *queue)
+{
+ unsigned long irqflags;
+ bool ret;
+
+ /* Lock the list empty operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ ret = list_empty(&queue->head);
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+
+ return ret;
+}
+
+/*===========================================================================
+ * 3. Link status operations
+ *===========================================================================*/
+
+/*
+ * Effectively enable the NTB link.
+ *
+ * From the moment of return from this function the inter-partition
+ * communications are enabled as well as translating Request and Complition TLPs.
+ * This function is called by the Primary side on the initialization phase. The
+ * Secondary ports can invoke it by calling the ntb_link_enable() callback.
+ */
+static void idt_ntb_link_effective_enable(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntctl = 0, reqid, ntmtbldata = 0;
+
+ /* Retrieve the current complex Requester ID (Bus:Device:Function) */
+ reqid = idt_ntb_readfld_mem(cfg, IDT_NT_MTBL_REQID);
+
+ /* Set the corresponding NT Mapping table entry of port partition index
+ * with the data to perform the Request ID translation */
+ idt_ntb_writefld_var(&ntmtbldata, IDT_NT_MTBL_BDF, reqid);
+ idt_ntb_writefld_var(&ntmtbldata, IDT_NT_MTBL_PART, (u32)pdata->part);
+ idt_ntb_writefld_var(&ntmtbldata, IDT_NT_MTBL_VALID, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLADDR, (u32)pdata->part);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLDATA, ntmtbldata);
+
+ /* Enable the ID protection and Completion TLPs translation */
+ idt_ntb_writefld_var(&ntctl, IDT_NT_IDPROTDIS, OFF);
+ idt_ntb_writefld_var(&ntctl, IDT_NT_CPEN, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTCTL, ntctl);
+
+ /* Enable the bus mastering, which effectively enables the Request TLPs
+ * translation and MSI IRQs generation */
+ pci_set_master(pdata->pdev);
+
+ /* The ndevs->lnk_sts variable is going to change in the work thread */
+}
+
+/*
+ * Effectively disable the NTB link.
+ *
+ * From the moment of return from this function the inter-partition
+ * communications are disabled.
+ */
+static void idt_ntb_link_effective_disable(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+
+ /* Disable the bus mastering, which effectively stops translating the
+ * Request TLPs across the boundary of local partition */
+ pci_clear_master(pdata->pdev);
+
+ /* Disable Completion TLPs */
+ idt_ntb_writefld_mem(cfg, IDT_NT_CPEN, OFF);
+
+ /* Disable the corresponding NT Mapping table entry */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLADDR, (u32)pdata->part);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLDATA, (u32)OFF);
+
+ /* The ndevs->lnk_sts variable is going to change in the work thread */
+}
+
+/*
+ * Notify the peer device that the local side is ready.
+ *
+ * Since the Primary side can't enable/disable link by demand of the client
+ * driver, there should be some way to notify the opposite side, what the local
+ * client driver is installed and started working (by calling the
+ * ntb_enable_link method). So Global Signal register is used for that purpose.
+ */
+static void idt_ntb_link_notify(struct idt_ntb_dev *ndev)
+{
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Just write ON to the first bit of device NTGSIGNAL register
+ * It is available only using GASA* registers */
+ idt_ntb_writereg(cfg, portdata_tbl[ndev->port].ntgsignal, ON);
+}
+
+/*
+ * Clear the notification set before in the Global Signal Status register.
+ */
+static void idt_ntb_link_clear_notification(struct idt_ntb_dev *ndev)
+{
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Clear the Global Signal status bit of the device partition */
+ idt_ntb_writereg(cfg, IDT_SW_PCI_SEGSIGSTS, ((u32)1 << ndev->part));
+}
+
+/*
+ * Retrieve the current link status
+ */
+static int idt_ntb_link_status(struct idt_ntb_dev *ndev)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 localbme, peerbme, pciests, gsigsts;
+ unsigned int part;
+
+ /* Read the local Bus Master Enable status */
+ localbme = idt_ntb_readfld_mem(cfg, IDT_NT_BME);
+
+ /* Read the Global Signal Status bit related to the device partition */
+ gsigsts = idt_ntb_readreg(cfg, IDT_SW_PCI_SEGSIGSTS);
+ /* Retrieve the partition of the corresponding device */
+ part = (NTB_TOPO_PRI == pdata->role) ? ndev->part : pdata->part;
+ gsigsts = (gsigsts & ((u32)1 << part)) ? ON : OFF;
+
+ /* Read the peer Bus Master Enable status */
+ peerbme = idt_ntb_readreg(cfg, portdata_tbl[ndev->port].pcicmd);
+ peerbme = idt_ntb_readfld_var(peerbme, IDT_NT_BME);
+
+ /* Retrieve the peer port link status */
+ pciests = idt_ntb_readreg(cfg, portdata_tbl[ndev->port].sts);
+ pciests = idt_ntb_readfld_var(pciests, IDT_SW_PORT_LNKUP);
+
+ /* If Both BME fields are ON and PCIe data link is up then the NTB
+ * link is effectively up */
+ if (ON == pciests && ON == peerbme && ON == localbme && ON == gsigsts) {
+ return ON;
+ } /* else if (OFF == pciests || OFF == peerbme || Off == localbme ||
+ * OFF == gsigsts) {return OFF} */
+
+ return OFF;
+}
+
+/*
+ * Kernel thread polling the peer side link status by reading the corresponding
+ * PCIe link status register and NT Mapping table entry
+ */
+static void idt_ntb_poll_link_work(struct work_struct *work)
+{
+ struct idt_ntb_data *pdata = to_data_lnkwork(work);
+ struct idt_ntb_dev *ndev;
+ unsigned char id;
+ int curlnksts;
+
+ /* Walk through all available peers reading their status */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Get the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Retrieve the current link status */
+ curlnksts = idt_ntb_link_status(ndev);
+
+ /* If the link status has changed then call the event handler */
+ if (curlnksts != ndev->lnk_sts) {
+ ndev->lnk_sts = curlnksts;
+ ntb_link_event(&ndev->ntb);
+ }
+ }
+
+ /* Reschedule the work */
+ (void)queue_delayed_work(pdata->idt_wq, &pdata->lnk_work,
+ IDT_NTB_LNKPOLL_TOUT);
+}
+
+/*
+ * Initialize NTB link subsystem
+ *
+ * NOTE This function is not used by the client driver but just for
+ * initialization
+ */
+static void idt_ntb_init_link(struct idt_ntb_data *pdata)
+{
+ unsigned char id;
+
+ /* Initialize all the peers link status with OFF */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ pdata->ndevs[id].lnk_sts = OFF;
+ }
+
+ /* Enable the link if it's primary port */
+ if (NTB_TOPO_PRI == pdata->role) {
+ /* Clear all the Global Signal Status bits related to the
+ * locally available NTB device */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ idt_ntb_link_clear_notification(&pdata->ndevs[id]);
+ }
+ /* Next function enables the whole link no matter which NTB
+ * device it's */
+ idt_ntb_link_effective_enable(pdata);
+ }
+
+ /* Initialize the delayed kernel thread polling the link status */
+ INIT_DELAYED_WORK(&pdata->lnk_work, idt_ntb_poll_link_work);
+ (void)queue_delayed_work(pdata->idt_wq, &pdata->lnk_work,
+ IDT_NTB_LNKPOLL_TOUT);
+
+ dev_dbg_data(pdata, "IDT NTB peer device link polling started");
+}
+
+/*
+ * Clear the link polling subsystem
+ *
+ * NOTE This function is not used by the client driver but just for
+ * final deinitialization
+ */
+static void idt_ntb_clear_link(struct idt_ntb_data *pdata)
+{
+ unsigned char id;
+
+ /* Stop the link status polling thread */
+ cancel_delayed_work_sync(&pdata->lnk_work);
+
+ /* Disable the link */
+ idt_ntb_link_effective_disable(pdata);
+
+ /* Clear all the Global Signal Status bits related to the
+ * Primary port available NTB device */
+ if (NTB_TOPO_PRI == pdata->role) {
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ idt_ntb_link_clear_notification(&pdata->ndevs[id]);
+ }
+ }
+
+ dev_dbg_data(pdata, "IDT NTB peer device link polling stopped");
+}
+
+/*
+ * NTB bus callback - get the current ntb link state
+ */
+static int idt_ntb_link_is_up(struct ntb_dev *ntb, enum ntb_speed *speed,
+ enum ntb_width *width)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 pcielsts;
+ int lnksts;
+
+ /* Get the curret link status */
+ lnksts = idt_ntb_link_status(ndev);
+
+ /* Retrieve the PCIe data link parameters */
+ if (ON == lnksts) {
+ /* Read the PCIe link status */
+ pcielsts = idt_ntb_readreg(cfg,
+ portdata_tbl[ndev->port].pcielsts);
+ /* The register values numerically match the enum values */
+ if (speed) {
+ *speed = idt_ntb_readfld_var(pcielsts, IDT_NT_CURLNKSPD);
+ }
+ if (width) {
+ *width = idt_ntb_readfld_var(pcielsts, IDT_NT_CURLNKWDTH);
+ }
+ } else /* if (OFF == lnksts) */ {
+ if (speed) {
+ *speed = NTB_SPEED_NONE;
+ }
+ if (width) {
+ *width = NTB_WIDTH_NONE;
+ }
+ }
+
+ return lnksts;
+}
+
+/*
+ * NTB bus callback - enable the link on the secondary side of the ntb
+ *
+ * NOTE Since there can be more than one pair of NTB devices (we use shared
+ * Lookup table) on the Primary port, the link must be always enabled from that
+ * side. So the next function fully works from the Secondary side only.
+ */
+static int idt_ntb_link_enable(struct ntb_dev *ntb, enum ntb_speed speed,
+ enum ntb_width width)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = pdata->cfg_mmio;
+
+ /* Primary port driver enables the link in the initialization method */
+ if (NTB_TOPO_PRI == ntb->topo) {
+ /* Notify the opposite side, that the link is enabled */
+ idt_ntb_link_notify(ndev);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is virtually enabled");
+
+ return -EINVAL;
+ }
+
+ /* Secondary ports can effectively enable the link on the local side */
+ idt_ntb_link_effective_enable(pdata);
+
+ /* Enable the interrupts of message, doorbells, switch and temperature
+ * sensor events. This will generate all the pending interrupts after the
+ * link is effectively enabled */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, NTINT_UNMASK);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is enabled");
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - disable the link on the secondary side of the ntb
+ */
+static int idt_ntb_link_disable(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = pdata->cfg_mmio;
+
+ /* Primary port driver disables the link in the link clear method */
+ if (NTB_TOPO_PRI == ntb->topo) {
+ /* Notify the opposite side, that the link is disabled */
+ idt_ntb_link_clear_notification(ndev);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is virtually disabled");
+
+ return -EINVAL;
+ }
+
+ /* Disable the interrupts of message, doorbells, switch and temperature
+ * sensor events. This will stop generateing interrupts while link is
+ * down */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, NTINT_MASK);
+
+ /* Secondary ports can effectively disable the link on the local side */
+ idt_ntb_link_effective_disable(pdata);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is disabled");
+
+ return SUCCESS;
+}
+
+/*===========================================================================
+ * 4. Memory Window subsystem
+ *===========================================================================*/
+
+/*
+ * Find the Secondary port serial number (id) by the passed primary and
+ * secondary ports
+ */
+static inline unsigned char idt_ntb_findid(struct idt_ntb_topo *topo,
+ unsigned char pri, unsigned char sec)
+{
+ return hweight32(topo->secports[pri] & (((u32)1 << sec) - 1));
+}
+
+/*
+ * Initialize the PCI device BAR2(3:x64) setup register
+ */
+static int idt_ntb_setup_bar2(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ phys_addr_t limit;
+ int ret;
+
+ /* Request the PCI resources for the BAR2(3) */
+ ret = pci_request_region(pdata->pdev, BAR2, NTB_NAME);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Failed to request the PCI BAR2(3) resources");
+ return ret;
+ }
+
+ /* Retrieve the physical address of the mapped by the Lookup table
+ * shared memory - BAR2(3) */
+ pdata->mw_base = pci_resource_start(pdata->pdev, BAR2);
+
+ /* Limit the BAR2 address with resepect to the Lookup table boundary */
+ /* Calculate the size of just one Memory Window */
+ pdata->mw_size = pci_resource_len(pdata->pdev, BAR2)/32;
+
+ /* Find the limit address */
+ limit = pdata->mw_base + IDT_NTB_MW_CNT * pdata->mw_size - 1;
+
+ /* Set the BAR size limiting register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT2, (u32)limit);
+#ifdef CONFIG_64BIT
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT3, (u32)(limit >> 32));
+#endif /* CONFIG_64BIT */
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the PCI device BAR2(3:x64) setup register
+ */
+static void idt_ntb_clean_bar2(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 limit = -1;
+
+ /* Set the BAR size limiting register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT2, limit);
+#ifdef CONFIG_64BIT
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT3, limit);
+#endif /* CONFIG_64BIT */
+
+ /* Just write the disabled BARSETUP0 */
+ pci_release_region(pdata->pdev, BAR2);
+}
+
+/*
+ * Set the Memory Window translation address for the passed peer NTB device
+ */
+static int idt_ntb_setmw(struct idt_ntb_dev *ndev, const int mwindx,
+ const dma_addr_t addr)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 lut_indxbar = 0, lut_partval = 0;
+ unsigned long irqflags;
+
+ /* Return error if the passed memory window index is out of range */
+ if (mwindx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid Memory Window index specified to set");
+ return -EINVAL;
+ }
+
+ /* Return error if the passed address is not aligned with the four
+ * bytes */
+ if (!IS_ALIGNED(addr, IDT_NTB_TRANSALIGN)) {
+ dev_err_ndev(ndev, "Translated base address is not aligned");
+ return -EINVAL;
+ }
+
+ /* Collect the Lookup table offset */
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_INDEX,
+ ndev->mw_self_offset + mwindx);
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_BAR, BAR2);
+
+ /* Collect the Lookup table entry partition and valid bits */
+ idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_PART, ndev->part);
+ idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_VALID, ON);
+
+ /* Start critical section writing to the local port Lookup table */
+ spin_lock_irqsave(&pdata->lut_lock, irqflags);
+ /* Write the data to the Lookup table registers of the peer */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTOFFSET, lut_indxbar);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTLDATA, (u32)addr);
+#ifdef CONFIG_64BIT
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)(addr >> 32));
+#else
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)0);
+#endif /* !CONFIG_64BIT */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTUDATA, lut_partval);
+ /* Finally unlock the Lookup table */
+ spin_unlock_irqrestore(&pdata->lut_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * Set the Memory Window translation address for the passed peer NTB device
+ */
+static int idt_ntb_unsetmw(struct idt_ntb_dev *ndev, const int mwindx)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 lut_indxbar = 0, lut_partval = 0;
+ unsigned long irqflags;
+
+ /* Return Error if the passed Memory Window index is out of range */
+ if (mwindx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid Memory Window index specified to unset");
+ return -EINVAL;
+ }
+
+ /* Collect the Lookup table offset */
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_INDEX,
+ ndev->mw_self_offset + mwindx);
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_BAR, BAR2);
+
+ /* Collect the Lookup table entry partition and valid bits */
+ idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_VALID, OFF);
+
+ /* Start critical section writing to the Lookup table */
+ spin_lock_irqsave(&pdata->lut_lock, irqflags);
+ /* Write the data to the Lookup table registers of the peer */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTOFFSET, lut_indxbar);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTLDATA, (u32)0);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)0);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTUDATA, lut_partval);
+ /* Finally unlock the Lookup table */
+ spin_unlock_irqrestore(&pdata->lut_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * Cleanup the local Lookup table
+ */
+static int idt_ntb_cleanlut(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_dev *ndev;
+ unsigned char id, mw;
+ int ret;
+
+ /* Walk through all the available peers */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ ndev = &pdata->ndevs[id];
+
+ /* Unset all the local memory windows */
+ for (mw = 0; mw < ndev->mw_self_cnt; mw++) {
+ ret = idt_ntb_unsetmw(ndev, mw);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+ }
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Initialize the Memory Windows for the current NT-function with respect to the
+ * topologically predefined NTB pairs
+ *
+ * NOTE The first NTB pairs are lucky to have the extended set of Memory Windows
+ */
+static int idt_ntb_init_mws(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_topo *topo = &pdata->topo;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ unsigned char id, mwcnt, luckies, curoffset;
+ int ret;
+
+ /* Calculate the number of Memory Windows per NTB */
+ mwcnt = IDT_NTB_MW_CNT / topo->paircnt;
+ luckies = IDT_NTB_MW_CNT % topo->paircnt;
+
+ /* Find the memory windows local and peer parameters */
+ if (NTB_TOPO_PRI == pdata->role) {
+ /* Loop over all the locally available peers */
+ curoffset = 0;
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Find the memory windows offset and count */
+ ndevs[id].mw_self_offset = curoffset;
+ ndevs[id].mw_self_cnt = mwcnt + (luckies > id ? 1 : 0);
+ ndevs[id].mw_peer_cnt = IDT_NTB_MW_CNT;
+
+ /* Get the offset for the next Memory Windows */
+ curoffset += ndevs[id].mw_self_cnt;
+ }
+ } else /* if (NTB_TOPO_SEC == pdata->role) */ {
+ id = ndevs[0].pairid;
+ ndevs[0].mw_self_offset = 0;
+ ndevs[0].mw_self_cnt = IDT_NTB_MW_CNT;
+ ndevs[0].mw_peer_cnt = mwcnt + (luckies > id ? 1 : 0);
+ }
+
+ /* Initialize the BAR2(3) related registers and data fields */
+ ret = idt_ntb_setup_bar2(pdata);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+
+ /* Initialize the Lookup table spinlock*/
+ spin_lock_init(&pdata->lut_lock);
+
+ /* Cleanup the Lookup table */
+ (void)idt_ntb_cleanlut(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB device memory windows redistributed");
+
+ return SUCCESS;
+}
+
+/*
+ * Clean the Memory Windows initialized for the current NT-function
+ */
+static void idt_ntb_clean_mws(struct idt_ntb_data *pdata)
+{
+ /* Cleanup the peers Lookup tables */
+ (void)idt_ntb_cleanlut(pdata);
+
+ /* Clean the BAR2(3) */
+ idt_ntb_clean_bar2(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB function memory windows cleaned");
+}
+
+/*
+ * NTB bus callback - local memory windows count
+ */
+static int idt_ntb_mw_count(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Return the number of available local memory windows */
+ return ndev->mw_self_cnt;
+}
+
+/*
+ * NTB bus callback - get the map resource of a memory window
+ */
+static int idt_ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx, phys_addr_t *base,
+ resource_size_t *size)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+
+ /* It's error to pass the out of range Memory Window index */
+ if (idx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid memory window index passed to get map res");
+ return -EINVAL;
+ }
+
+ /* The base address is determined with respect to the Lookup table
+ * table offset */
+ if (base)
+ *base = pdata->mw_base +
+ (ndev->mw_self_offset + idx) * pdata->mw_size;
+ if (size)
+ *size = pdata->mw_size;
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - get the local memory windows alignments
+ */
+static int idt_ntb_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+
+ /* It's error to pass the out of range Memory Window index */
+ if (idx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid memory window index passed to get alignment");
+ return -EINVAL;
+ }
+
+ /* According to standard the address should be alignment within 4KB */
+ if (addr_align)
+ *addr_align = SZ_4K;
+ /* Size alignment and max size effectively make the size fixed to
+ * size_max */
+ if (size_align)
+ *size_align = pdata->mw_size;
+ if (size_max)
+ *size_max = pdata->mw_size;
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - set the translation of a Memory Window
+ */
+static int idt_ntb_mw_set_trans(struct ntb_dev *ntb, int idx, dma_addr_t addr,
+ resource_size_t size)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ int ret;
+
+ /* Although the passed size is not used anywhere, we need to make sure
+ * the size fits the memory window */
+ if (0 != size && size != pdata->mw_size) {
+ dev_err_ndev(ndev,
+ "Invalid translated address size was specified");
+ return -EINVAL;
+ }
+
+ /* Set the passed memory window or unset it if the size is zero */
+ if (0 != size) {
+ ret = idt_ntb_setmw(ndev, idx, addr);
+ } else /* if (0 == size) */ {
+ ret = idt_ntb_unsetmw(ndev, idx);
+ }
+
+ return ret;
+}
+
+/*
+ * NTB bus callback - peer memory windows count
+ */
+static int idt_ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Return the number of available peer memory windows */
+ return ndev->mw_peer_cnt;
+}
+
+/*
+ * NTB bus callback - get the peer memory windows alignments
+ */
+static int idt_ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+
+ /* It's error to pass the out of range Memory Window index */
+ if (idx >= ndev->mw_peer_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid memory window index passed to get "
+ "peer alignment");
+ return -EINVAL;
+ }
+
+ /* Although there are only two unmodifiable LS-bits in lookup table
+ * entries, according to standard the address should be aligned
+ * within 4KB */
+ if (addr_align)
+ *addr_align = SZ_4K;
+ /* Size alignment and max size effectively make the size fixed to
+ * size_max */
+ if (size_align)
+ *size_align = pdata->mw_size;
+ if (size_max)
+ *size_max = pdata->mw_size;
+
+ return SUCCESS;
+}
+
+/*===========================================================================
+ * 5. Doorbells subsystem
+ *===========================================================================*/
+
+static void idt_ntb_db_tasklet(unsigned long data);
+
+/*
+ * Initialize the Global Doorbell Mask
+ *
+ * NOTE Initialize the Inbound Doorbell mask so the local event can
+ * be rised by the self Doorbells bits only. The Outbound
+ * Doorbell is setup so the local port could set both self
+ * and peer Doorbells. Due to the self and peer masks swap
+ * the following loops should work well on the both sides
+ */
+static void idt_ntb_init_gdbellmsk(struct idt_ntb_data *pdata, unsigned char id)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ u32 selfpartbits, peerpartbits;
+ int setbit;
+
+ /* There is a bug if the passed id exceeds the total number of peers */
+ BUG_ON(id >= pdata->peer_cnt);
+
+ /* Get the self and peer partition masks */
+ selfpartbits = ~((u32)1 << pdata->part);
+ peerpartbits = ~((u32)1 << ndevs[id].part);
+
+ /* Init the self Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_self_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ selfpartbits);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ selfpartbits & peerpartbits);
+ }
+
+ /* Init the peer Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_peer_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ peerpartbits);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ selfpartbits & peerpartbits);
+ }
+}
+
+/*
+ * Deinitialize the Global Doorbell Mask
+ *
+ * Function is unused to make sure the NTB devices can be unloaded without
+ * any serious consequences for the peer device.
+ */
+static void __maybe_unused idt_ntb_clean_gdbellmsk(struct idt_ntb_data *pdata,
+ unsigned char id)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ int setbit;
+
+ /* There is a bug if the passed id exceeds the total number of peers */
+ BUG_ON(id >= pdata->peer_cnt);
+
+ /* Deinit the self Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_self_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ (u32)0);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ (u32)0);
+ }
+ /* Deinit the peer Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_peer_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ (u32)0);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ (u32)0);
+ }
+}
+
+/*
+ * Initialize the Doorbells for the current NT-function with respect to the
+ * topologically predefined NTB pairs
+ *
+ * NOTE The first NTB pairs are lucky to have the extended set of Doorbells
+ */
+static void idt_ntb_init_db(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_topo *topo = &pdata->topo;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ unsigned char id, dbcntstd, dbcntext, dbleft, luckies, pairid, dboffset;
+ u32 pridbmask, secdbmask;
+
+ /* Calculate the number of Doorbells per pair and the leftovers */
+ dbcntstd = IDT_NTB_DBELL_CNT / topo->paircnt;
+ dbleft = IDT_NTB_DBELL_CNT % topo->paircnt + (dbcntstd % 2) * topo->paircnt;
+ /* Alter the db count to be even */
+ dbcntstd = (dbcntstd / 2) * 2;
+ dbcntext = dbcntstd + 2;
+
+ /* Number of the lucky pairs having additional Doorbells */
+ luckies = dbleft / 2;
+
+ /* Loop over all the locally available peers */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Current pair ID */
+ pairid = ndevs[id].pairid;
+
+ /* Retrieve the doorbells count and the doorbells offset for the
+ * current pair ID (the first luckies have extended doorbells) */
+ if (luckies > pairid) {
+ ndevs[id].db_cnt = dbcntext / 2;
+ dboffset = dbcntext * pairid;
+ } else {
+ ndevs[id].db_cnt = dbcntstd / 2;
+ dboffset = dbcntext * luckies +
+ dbcntstd * (pairid - luckies);
+ }
+
+ /* Calculate the valid Doorbells mask for the corresponding
+ * ports */
+ ndevs[id].db_valid_mask = ((u32)1 << ndevs[id].db_cnt) - 1;
+ pridbmask = ndevs[id].db_valid_mask << dboffset;
+ secdbmask = pridbmask << ndevs[id].db_cnt;
+
+ /* Initialize the corresponding Device structure fields */
+ if (NTB_TOPO_PRI == pdata->role) {
+ ndevs[id].db_self_mask = pridbmask;
+ ndevs[id].db_self_offset = dboffset;
+ ndevs[id].db_peer_mask = secdbmask;
+ ndevs[id].db_peer_offset = dboffset + ndevs[id].db_cnt;
+ } else /* if (NTB_TOPO_SEC == pdata->role) */ {
+ ndevs[id].db_self_mask = secdbmask;
+ ndevs[id].db_self_offset = dboffset + ndevs[id].db_cnt;
+ ndevs[id].db_peer_mask = pridbmask;
+ ndevs[id].db_peer_offset = dboffset;
+ }
+
+ /* Initialize the corresponding Global Doorbell masks. It can be
+ * done by both Primary and Secondary ports */
+ idt_ntb_init_gdbellmsk(pdata, id);
+ }
+
+ /* Initialize the spin lock to sync access to the self doorbell status
+ * and mask variables */
+ pdata->db_sts = 0;
+ pdata->db_msk = (u32)-1;
+ /* In fact db_lock is used at most at tasklet so BH lock would be enough,
+ * but the critical section can be accessed in the db event handler,
+ * which is protected by the context irqsave spin lock. So calling BH
+ * spin locker/unlocker function would cause the OOPS Warning of
+ * local_bh_enable_ip method. Therefore the irqsave/irqrestore methods
+ * are used to synchronize access to the db_sts and db_msk fields*/
+ spin_lock_init(&pdata->db_lock);
+
+ /* Initialize the doorbells tasklet */
+ tasklet_init(&pdata->db_tasklet, idt_ntb_db_tasklet,
+ (unsigned long)pdata);
+
+ /* Unmask the inbound doorbell interrupts */
+ idt_ntb_writereg(pdata->cfg_mmio, IDT_NT_PCI_INDBELLMSK, INDB_UNMASK);
+
+ dev_dbg_data(pdata, "IDT NTB device doorbells initialized");
+}
+
+/*
+ * Clean the Doorbells initialized for the pairs of NT-functions
+ *
+ * It just makes all the NT-functions being able to use the self and peer
+ * Doorbells
+ */
+static void idt_ntb_clean_db(struct idt_ntb_data *pdata)
+{
+ /*unsigned char id;*/
+
+ /* Just kill the tasklet */
+ tasklet_kill(&pdata->db_tasklet);
+
+ /* Just clean the Doorbell masks for all the peers as they must have
+ * initially been. Do it by the Primary side only */
+ /*if (NTB_TOPO_PRI == pdata->role) {
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ idt_ntb_clean_gdbellmsk(pdata, id);
+ }
+ }*/
+
+ dev_dbg_data(pdata, "IDT NTB device doorbells deinitilized");
+}
+
+/*
+ * Doorbells event tasklet
+ */
+static void idt_ntb_db_tasklet(unsigned long data)
+{
+ struct idt_ntb_data *pdata = (struct idt_ntb_data *)data;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 db_sts, db_self, db_sts_prev;
+ unsigned long setbit, irqflags;
+ unsigned char id;
+
+ /* NOTE All doorbells are masked to generate the interrupt by the IRQ
+ * handler until the cause of the interrupt is handled */
+ db_sts = idt_ntb_readreg(cfg, IDT_NT_PCI_INDBELLSTS);
+ /* Clear all the retrieved doorbell bits */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_INDBELLSTS, db_sts);
+ /* Finally unmask the doorbells interrupt. The next action shall rise
+ * the interrupt if any doorbell bit was set after the register had
+ * been read and cleared */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_INDBELLMSK, INDB_UNMASK);
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Retrieve the current doorbell status bits */
+ db_sts_prev = pdata->db_sts;
+ /* Set the new doorbell status */
+ pdata->db_sts |= db_sts;
+ /* There are going to be handled only the doorbell bits, which have not
+ * been set before and also have not been masked */
+ db_sts &= ~db_sts_prev & ~pdata->db_msk;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* If the new doorbell status bits are masked then do nothing */
+ if (!db_sts) {
+ dev_dbg_data(pdata, "Got masked doorbell interrupt");
+ return;
+ }
+
+ /* Walk through all the peers looking for the relevant one to handle
+ * new doorbells */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Invoke the context callback if there are doorbells set for
+ * the current NTB device */
+ db_self = (db_sts & ndevs[id].db_self_mask);
+ db_self >>= ndevs[id].db_self_offset;
+ for_each_set_bit_u32(db_self, setbit) {
+ ntb_db_event(&ndevs[id].ntb, (int)setbit);
+ }
+ }
+}
+
+/*
+ * NTB bus callback - get a mask of doorbell bits supported by the ntb
+ */
+static u64 idt_ntb_db_valid_mask(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Return the valid doorbell bits mask */
+ return ndev->db_valid_mask;
+}
+
+/*
+ * NTB bus callback - get the number of doorbell interrupt vectors
+ */
+static int idt_ntb_db_vector_count(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Number of doorbell vectors equal to the doorbell bits count */
+ return ndev->db_cnt;
+}
+
+/*
+ * NTB bus callback - get a mask of doorbell bits serviced by a vector
+ */
+static u64 idt_ntb_db_vector_mask(struct ntb_dev *ntb, int db_vec)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ if (db_vec < 0 || ndev->db_cnt <= db_vec) {
+ return 0;
+ }
+
+ /* Each doorbell bit corresponds to the vector so the mask is just one
+ * shifted bit */
+ return ((u64)1 << db_vec);
+}
+
+/*
+ * NTB bus callback - read the local doorbell register
+ */
+static u64 idt_ntb_db_read(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+ u32 db_sts;
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Read the current doorbell status */
+ db_sts = pdata->db_sts;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* Return the accordingly shifted doorbell bits */
+ return (db_sts & ndev->db_self_mask) >> ndev->db_self_offset;
+}
+
+/*
+ * NTB bus callback - set bits in the local doorbell register
+ *
+ * NOTE It must be done using the doorbell register io to generate the
+ * interrupt and invoke the doorbell event handler set by the client
+ * driver
+ */
+static int idt_ntb_db_set(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid doorbell bits are passed to locally set");
+ return -EINVAL;
+ }
+
+ /* Set the corresponding bits in the doorbell register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_OUTDBELLSET,
+ ((u32)db_bits << ndev->db_self_offset));
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - clear bits in the local doorbell register
+ */
+static int idt_ntb_db_clear(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid doorbell bits are passed to locally clear");
+ return -EINVAL;
+ }
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Read the current doorbell status */
+ pdata->db_sts &= ~((u32)db_bits << ndev->db_self_offset);
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - read the local doorbell mask
+ */
+static u64 idt_ntb_db_read_mask(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+ u32 db_msk;
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Read the current doorbell mask */
+ db_msk = pdata->db_msk;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* Return the accordingly shifted doorbell bits */
+ return (db_msk & ndev->db_self_mask) >> ndev->db_self_offset;
+}
+
+/*
+ * NTB bus callback - set bits in the local doorbell mask
+ */
+static int idt_ntb_db_set_mask(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid field is passed to set the doorbell mask");
+ return -EINVAL;
+ }
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Set the corresponding bits in the local mask */
+ pdata->db_msk |= ((u32)db_bits << ndev->db_self_offset);
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - clear bits in the local doorbell mask
+ */
+static int idt_ntb_db_clear_mask(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ u32 db_sts, unmask_bits;
+ unsigned long setbit, irqflags;
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid field is passed to clear the doorbell mask");
+ return -EINVAL;
+ }
+
+ /* Calculate the unmaskable bits first */
+ unmask_bits = ((u32)db_bits << ndev->db_self_offset);
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Retrieve the doorbell status bits, which have been masked, but are
+ * going to be unmasked now */
+ db_sts = pdata->db_sts & pdata->db_msk & unmask_bits;
+ /* Clear the corresponding bits in the local mask */
+ pdata->db_msk &= ~unmask_bits;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* Invoke the context callback if there are set doorbells, which have
+ * just been unmasked */
+ db_sts = (db_sts & ndev->db_self_mask) >> ndev->db_self_offset;
+ for_each_set_bit_u32(db_sts, setbit) {
+ ntb_db_event(&ndev->ntb, (int)setbit);
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - set bits in the peer doorbell register
+ */
+static int idt_ntb_peer_db_set(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid doorbell bits are passed to remotely set");
+ return -EINVAL;
+ }
+
+ /* Set the corresponding bits in the doorbell register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_OUTDBELLSET,
+ ((u32)db_bits << ndev->db_peer_offset));
+
+ return SUCCESS;
+}
+
+/*===========================================================================
+ * 6. Messaging subsystem
+ *===========================================================================*/
+
+static void idt_ntb_inmsg_work(struct work_struct *work);
+
+static void idt_ntb_outmsg_work(struct work_struct *work);
+
+static void idt_ntb_msg_tasklet(unsigned long data);
+
+/*
+ * Constructor is used initialize the allocated message structure
+ */
+static inline void idt_ntb_msg_ctor(struct idt_ntb_msg *msg)
+{
+ /* Set initial message retry count */
+ msg->retry = IDT_NTB_SENDMSG_RETRY;
+
+ /* Init the queue entry */
+ INIT_LIST_HEAD(&msg->entry);
+}
+
+/*
+ * Initialize the messaging subsystem
+ */
+static int idt_ntb_init_msg(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndev;
+ unsigned char id;
+
+ /* Allocate the IDT messages cache without alignment and flags with no
+ * constructor */
+ pdata->msg_cache = kmem_cache_create(NTB_CACHENAME,
+ sizeof(struct idt_ntb_msg), 0, 0, NULL);
+ if (NULL == pdata->msg_cache) {
+ dev_err_data(pdata,
+ "IDT NTB failed to allocate the messages cache");
+ return -ENOMEM;
+ }
+
+ /* Init the messages routing spin lock */
+ spin_lock_init(&pdata->msg_lock);
+
+ /* Walk through all the device initializing the message related
+ * structures */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Get the current NTB device structure */
+ ndev = &pdata->ndevs[id];
+
+ /* Initialize the incoming messages queue */
+ atomic_queue_init(&ndev->qinmsg);
+ /* Setup the incoming message work thread (it's not delayed) */
+ INIT_WORK(&ndev->inmsg_work, idt_ntb_inmsg_work);
+
+ /* Initialize the outgoing messages queue */
+ atomic_queue_init(&ndev->qoutmsg);
+ /* Setup the outgoing message work thread (it can be
+ * delayed) */
+ INIT_DELAYED_WORK(&ndev->outmsg_work, idt_ntb_outmsg_work);
+ }
+
+ /* Setup the messages tasklet - bh handler of incoming messages */
+ tasklet_init(&pdata->msg_tasklet, idt_ntb_msg_tasklet,
+ (unsigned long)pdata);
+
+ /* Clear the outbound and inbound Messages status */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, MSG_MASK);
+
+ /* Unmask the message interrupts only for the first incoming message
+ * register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_UNMASK);
+
+ dev_dbg_data(pdata, "IDT NTB device messaging subsystem initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the messaging subsystem
+ */
+static void idt_ntb_deinit_msg(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndev;
+ struct list_head *entry;
+ unsigned char id;
+
+ /* Just kill the tasklet */
+ tasklet_kill(&pdata->db_tasklet);
+
+ /* Walk through all the devices deinitializing the message related
+ * structures */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Get the current NTB device structure */
+ ndev = &pdata->ndevs[id];
+
+ /* Stop the incoming message work thread */
+ cancel_work_sync(&ndev->inmsg_work);
+ /* Free all the allocated incoming message objects */
+ while (!atomic_queue_empty(&ndev->qinmsg)) {
+ entry = atomic_queue_get(&ndev->qinmsg);
+ kmem_cache_free(pdata->msg_cache,
+ to_msg_list_entry(entry));
+ }
+
+ /* Stop the outgoing message work thread */
+ cancel_delayed_work_sync(&ndev->outmsg_work);
+ /* Free all the allocated outgoing message objects */
+ while (!atomic_queue_empty(&ndev->qoutmsg)) {
+ entry = atomic_queue_get(&ndev->qoutmsg);
+ kmem_cache_free(pdata->msg_cache,
+ to_msg_list_entry(entry));
+ }
+ }
+
+ /* Mask the message interrupts */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_MASK);
+
+ /* Clear the outbound and inbound messages status */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, MSG_MASK);
+
+ /* Destroy the IDT messages cache */
+ kmem_cache_destroy(pdata->msg_cache);
+
+ dev_dbg_data(pdata,
+ "IDT NTB function messaging subsystem deinitialized");
+}
+
+/*
+ * Write message to the specified peer
+ */
+static int idt_ntb_writemsg(struct idt_ntb_dev *ndev, const struct ntb_msg *msg)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 stat, swpmsgctl[IDT_NTB_MSG_CNT];
+ int regid;
+
+ /* Initialize the message control register so the local outbound message
+ * registers would be connected with the peers inbound ones */
+ for (regid = 0; regid < IDT_NTB_MSG_CNT; regid++) {
+ /* Init switch partition message control registers variable */
+ swpmsgctl[regid] = 0;
+ idt_ntb_writefld_var(&swpmsgctl[regid], IDT_SW_MSGROUTE_REG,
+ regid);
+ idt_ntb_writefld_var(&swpmsgctl[regid], IDT_SW_MSGROUTE_PART,
+ ndev->part);
+ }
+
+ /* Use spin lock to synchronize just thirteen IO operations. It's used
+ * just among the kernel threads so we don't need to disable IRQs/bh */
+ spin_lock(&pdata->msg_lock);
+ /* Route to the local outbound message to the inbound one of the peer
+ * and send the data to there starting from the data because the
+ * interrupts are enabled for the first message register only */
+ for (regid = (IDT_NTB_MSG_CNT - 1); 0 <= regid; regid--) {
+ /* Set the route and send the data */
+ idt_ntb_writereg(cfg, partdata_tbl[pdata->part].msgctl[regid],
+ swpmsgctl[regid]);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_OUTMSG0 + regid,
+ msg->data[regid]);
+ /* Read the status of the previous operation */
+ stat = idt_ntb_readfld_mem(cfg, IDT_NT_OUTMSGSTS);
+ if (SUCCESS != stat) {
+ dev_dbg_ndev(ndev,
+ "Failed to send message to peer %hhd", regid);
+ break;
+ }
+ }
+ /* Immedietly clear the outbound message status if it has been set */
+ if (SUCCESS != stat) {
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, OUTMSG_MASK);
+ }
+ /* Finally unlock the message routing subsystem */
+ spin_unlock(&pdata->msg_lock);
+
+ /* If the write operation was not successful then the peer inbound
+ * register must be full so return -EBUSY error */
+ if (SUCCESS != stat) {
+ return -EBUSY;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Read the message
+ */
+static int idt_ntb_readmsg(struct idt_ntb_data *pdata, unsigned char *part,
+ struct ntb_msg *msg)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 msgsts, msgsrc;
+ unsigned char regid;
+
+ /* Read the inbound messages status */
+ msgsts = idt_ntb_readfld_mem(cfg, IDT_NT_INMSGSTS);
+ if (INMSG_STS != msgsts) {
+ dev_err_data(pdata, "Invalid status %#80x to read msg", msgsts);
+ BUG();
+ return -EINVAL;
+ }
+
+ /* Read data from the inbound message registers. It doesn't need to be
+ * synchronized since the read operation is performed from the tasklet
+ * only, that is non-reentrant */
+ *part = idt_ntb_readreg(cfg, IDT_NT_PCI_INMSGSRC0);
+ for (regid = 0; regid < IDT_NTB_MSG_CNT; regid++) {
+ msg->data[regid] =
+ idt_ntb_readreg(cfg, IDT_NT_PCI_INMSG0 + regid);
+ /* Read the source of the message checking whether the message
+ * data has come from the same partition */
+ msgsrc = idt_ntb_readreg(cfg, IDT_NT_PCI_INMSGSRC0 + regid);
+ if (msgsrc != *part) {
+ dev_err_data(pdata,
+ "Message data is inconsistent, src: %u != %u",
+ *part, msgsrc);
+ BUG();
+ return -EINVAL;
+ }
+ }
+
+ /* Clear the inbound message status */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, INMSG_MASK);
+
+ return SUCCESS;
+}
+
+/*
+ * Work thread handling the inbound messages events
+ */
+static void idt_ntb_inmsg_work(struct work_struct *work)
+{
+ struct idt_ntb_dev *ndev = to_ndev_inmsg_work(work);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ struct list_head *entry;
+ struct idt_ntb_msg *msgwrap;
+
+ /* Retrieve the last received message. It's bug to have inbound message
+ * queue empty at this point since the tasklet has just added one in
+ * there */
+ entry = atomic_queue_get(&ndev->qinmsg);
+ BUG_ON(NULL == entry);
+ msgwrap = to_msg_list_entry(entry);
+
+ /* Call the client driver message event handler */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_NEW, &msgwrap->msg);
+
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+}
+
+/*
+ * Work thread handling the outgoing messages
+ */
+static void idt_ntb_outmsg_work(struct work_struct *work)
+{
+ struct idt_ntb_dev *ndev = to_ndev_outmsg_work(work);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ struct list_head *entry;
+ struct idt_ntb_msg *msgwrap;
+ int ret;
+
+ /* Retrieve a message from the top of the queue. It's bug to have
+ * inbound message queue empty at this point since the client driver
+ * has just added one in there */
+ entry = atomic_queue_get(&ndev->qoutmsg);
+ BUG_ON(NULL == entry);
+ msgwrap = to_msg_list_entry(entry);
+
+ /* If link is not up it is useless to send any data */
+ if (OFF == idt_ntb_link_status(ndev)) {
+ dev_dbg_ndev(ndev,
+ "Link got suddenly down while sending a message");
+ /* Link got down so rise the fail event */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_FAIL, &msgwrap->msg);
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+ /* If some messages are left then reschedule the worker */
+ goto outmsg_work_requeue;
+ } /* else of (ON == idt_ntb_link_status(ndev)) */
+
+ /* Try to send the message */
+ ret = idt_ntb_writemsg(ndev, &msgwrap->msg);
+ if (SUCCESS == ret) {
+ /* The message has been successfully sent so rise the event */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_SENT, &msgwrap->msg);
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+ /* May need to reschedule the worker */
+ goto outmsg_work_requeue;
+ } /* else if (SUCCESS != ret) {} */
+
+ /* Could not send message. Rise the error if it has been the last
+ * attempt. If it hasn't get the message back into the queue and
+ * restart the worker */
+ msgwrap->retry--;
+ if (likely(0 != msgwrap->retry)) {
+ atomic_queue_add(&ndev->qoutmsg, &msgwrap->entry);
+ } else /* if (0 == msgwrap->retry) */ {
+ dev_err_ndev(ndev, "Run out of attempt to send a message");
+ /* Rise the error in this case */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_FAIL, &msgwrap->msg);
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+ }
+
+ /* If there is something left to send then queue the handler again */
+outmsg_work_requeue:
+ if (!atomic_queue_empty(&ndev->qoutmsg)) {
+ (void)queue_delayed_work(pdata->idt_wq, &ndev->outmsg_work,
+ IDT_NTB_SENDMSG_TOUT);
+ }
+}
+
+/*
+ * Tasklet handling inbound messages
+ */
+static void idt_ntb_msg_tasklet(unsigned long data)
+{
+ struct idt_ntb_data *pdata = (struct idt_ntb_data *)data;
+ struct idt_ntb_dev *ndev, *tndev = NULL;
+ struct idt_ntb_msg *msgwrap;
+ void __iomem *cfg = pdata->cfg_mmio;
+ unsigned char part, id;
+
+ /* Allocate the memory for the new message */
+ msgwrap = kmem_cache_alloc(pdata->msg_cache, GFP_KERNEL);
+ if (NULL == msgwrap) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for incoming message");
+ return;
+ }
+ /* Initializet the allocated message wrap structure although it's not
+ * necessary here */
+ idt_ntb_msg_ctor(msgwrap);
+
+ /* Read the message from the inbound registers. Don't need to check
+ * the return value since error would be asserted anyway */
+ (void)idt_ntb_readmsg(pdata, &part, &msgwrap->msg);
+
+ /* Finally unmask the message IRQs so the next message can be
+ * retrieved */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_UNMASK);
+
+ /* Find device the message has been sent to */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Retrieve the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Break the loop if target device is found */
+ if (ndev->part == part) {
+ tndev = ndev;
+ break;
+ }
+ }
+ /* Assert bug if message was received from invalid partition */
+ BUG_ON(NULL == tndev);
+
+ /* Add the new message to the tail of incoming queue of the target
+ * device */
+ atomic_queue_add_tail(&ndev->qinmsg, &msgwrap->entry);
+
+ /* Schedule the inbound message worker straight away */
+ (void)queue_work(pdata->idt_wq, &ndev->inmsg_work);
+}
+
+/*
+ * NTB bus callback - post the message to the peer
+ */
+static int idt_ntb_msg_post(struct ntb_dev *ntb, struct ntb_msg *msg)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ struct idt_ntb_msg *msgwrap;
+ unsigned char idx;
+
+ /* If the link is down then don't post any message */
+ if (OFF == idt_ntb_link_status(ndev)) {
+ dev_dbg_ndev(ndev,
+ "Can't post a message since link is down");
+ return -EINVAL;
+ }
+
+ /* Allocate memory for message wrap structure */
+ msgwrap = kmem_cache_alloc(pdata->msg_cache, GFP_KERNEL);
+ if (NULL == msgwrap) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for outgoing message");
+ return -ENOMEM;
+ }
+ /* Initializet the allocated message wrap structure */
+ idt_ntb_msg_ctor(msgwrap);
+
+ /* Fill in the message wrapper with data */
+ for (idx = 0; idx < IDT_NTB_MSG_CNT; idx++) {
+ msgwrap->msg.data[idx] = msg->data[idx];
+ }
+
+ /* Add the initialized wrap to the queue of outgoing messages */
+ atomic_queue_add_tail(&ndev->qoutmsg, &msgwrap->entry);
+
+ /* Start the outgoing messages worker with no timeout */
+ (void)queue_delayed_work(pdata->idt_wq, &ndev->outmsg_work, 0);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - size of the message data
+ */
+static int idt_ntb_msg_size(struct ntb_dev *ntb)
+{
+ /* Just return the number of messages registers */
+ return IDT_NTB_MSG_CNT;
+}
+
+/*===========================================================================
+ * 7. IRQ-related functions
+ *===========================================================================*/
+
+static irqreturn_t idt_ntb_isr(int irq, void *dev);
+
+/*
+ * Convert the temperature field to the value and fraction
+ */
+static inline void idt_ntb_convert_temp(const u32 temp,
+ unsigned char *val, unsigned char *frac)
+{
+ *val = temp >> 1;
+ *frac = ((temp & 0x1) ? 5 : 0);
+}
+
+/*
+ * Initialize the IDT IRQ sources
+ */
+static void idt_ntb_init_irqsrc(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 tempctl = 0;
+
+ /* Set the temperature sensor alarms */
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_LTH, IDT_NTB_TEMP_LTH << 1);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_HTH, IDT_NTB_TEMP_HTH << 1);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_BLTH_EN, ON);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_AHTH_EN, ON);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_PDOWN, OFF);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_TMPCTL, tempctl);
+
+ /* Interrupts are enabled by default only for Primary side since there
+ * can be more than one device */
+ if (NTB_TOPO_PRI == pdata->role) {
+ /* Enable the interrupts of message, doorbells, switch and
+ * temperature sensor events. This will generate all the
+ * pending interrupts after the link is effectively enabled */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, NTINT_UNMASK);
+ } else /* if (NTB_TOPO_SEC == pdata->role) */ {
+ /* Disable all the interrupts. NTB device enable callback will
+ * enable the necessary message, doorbells, switch and
+ * temperature sensor events */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, ALLINT_MASK);
+ }
+}
+
+/*
+ * Clear the IDT IRQs
+ */
+static void idt_ntb_clear_irqsrc(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 tempctl = 0;
+
+ /* Unset the temperature sensor alarm and disable the sensor */
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_BLTH_EN, OFF);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_AHTH_EN, OFF);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_PDOWN, ON);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_TMPCTL, tempctl);
+
+ /* Mask all the interrupts */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, ALLINT_MASK);
+}
+
+/*
+ * Initialize the PCIe interrupt handler
+ *
+ * NOTE The code is gotoed a bit, but still it's pretty obvious. First
+ * we try to enable MSI interrupt. If it fails we initiate the INTx interrupt.
+ * In any successful case the IDT NTB interrupts need to be enabled.
+ */
+static int idt_ntb_init_isr(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+ int ret;
+
+ /* Enable the MSI interrupts */
+ ret = pci_enable_msi(pdev);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "IDT failed to enable MSI interrupt");
+ goto err_try_intx;
+ }
+
+ /* Request correspondig IRQ number */
+ ret = request_irq(pdev->irq, idt_ntb_isr, 0, NTB_IRQNAME, pdata);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "IDT failed to set MSI IRQ handler");
+ goto err_disable_msi;
+ }
+
+ /* From now on the MSI interrupt is used */
+ dev_dbg_data(pdata, "IDT NTB is using MSI interrupts");
+
+ /* Need to enable the corresponding IDT NTB interrupts */
+ goto idt_init_irqs;
+
+err_disable_msi:
+ pci_disable_msi(pdev);
+
+err_try_intx:
+ /* Enable INTx interrutps since MSI can't be used */
+ pci_intx(pdev, ON);
+
+ ret = request_irq(pdev->irq, idt_ntb_isr, IRQF_SHARED,
+ NTB_IRQNAME, pdata);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "IDT failed to enable INTx interrupt");
+ goto err_pci_indx;
+ }
+
+ /* From now on the INTx interrupt is used */
+ dev_dbg_data(pdata, "IDT NTB is using INTx interrupts");
+
+ /* Need to enable the corresponding IDT NTB interrupts */
+idt_init_irqs:
+ idt_ntb_init_irqsrc(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB function IRQs initilized");
+
+ return SUCCESS;
+
+err_pci_indx:
+ pci_intx(pdev, OFF);
+
+ return ret;
+}
+
+/*
+ * Deinitialize the PCIe interrupt handler
+ */
+static void idt_ntb_clear_isr(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+
+ /* Clear the IDT NTB interrupt sources by masking them */
+ idt_ntb_clear_irqsrc(pdata);
+
+ /* Stop the interrupt handling */
+ free_irq(pdev->irq, pdata);
+ if (pci_dev_msi_enabled(pdev)) {
+ pci_disable_msi(pdev);
+ } else /* if (!pci_dev_msi_enabled(pdev)) */ {
+ pci_intx(pdev, OFF);
+ }
+
+ dev_dbg_data(pdata, "IDT NTB function interrupts are disabled");
+}
+
+/*
+ * Switch events ISR
+ */
+static void idt_ntb_se_isr(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntintsts = 0, sests;
+
+ /* Clean the corresponding interrupt bit */
+ idt_ntb_writefld_var(&ntintsts, IDT_NT_SEINT_STS, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTSTS, ntintsts);
+
+ /* Just print we got the switch event */
+ sests = idt_ntb_readreg(cfg, IDT_SW_PCI_SESTS);
+ dev_dbg_data(pdata, "Got switch event IRQ %#08x", sests);
+}
+
+/*
+ * Temperature sensor event ISR
+ */
+static void idt_ntb_temp_isr(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntintsts = 0, curtemp;
+ unsigned char val, frac;
+
+ /* Clean the corresponding interrupt bit */
+ idt_ntb_writefld_var(&ntintsts, IDT_NT_TMPINT_STS, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTSTS, ntintsts);
+
+ /* Read the temperature status */
+ curtemp = idt_ntb_readfld_mem(cfg, IDT_SW_TMP_CURTEMP);
+ idt_ntb_convert_temp(curtemp, &val, &frac);
+
+ /* Print the current temperature */
+ dev_warn_data(pdata,
+ "IDT temperature sensor alarm: %hhu.%hhu, valid space [%d;%d]",
+ val, frac, IDT_NTB_TEMP_LTH, IDT_NTB_TEMP_HTH);
+
+ /* Read the temperature alarm to clear the value out */
+ (void)idt_ntb_readreg(cfg, IDT_SW_PCI_TMPALARM);
+}
+
+/*
+ * IDT PCIe-swtich NTB-function interrupts handler
+ */
+static irqreturn_t idt_ntb_isr(int irq, void *dev)
+{
+ struct idt_ntb_data *pdata = dev;
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntintsts;
+ unsigned long setbit;
+ irqreturn_t status = IRQ_NONE;
+
+ /* Read the NTINTSTS register to determine the source of the
+ * interrupt.
+ * NOTE In order to make sure the deferred handlers are executed
+ * only when the corresponding interrupt really happens, the
+ * message/boorbell interrupt is temporarily masked. Additionally
+ * the interrupts status register must be filtered with the interrupts
+ * mask since the correposnding bit may be set even when the interrupt
+ * is masked */
+ ntintsts = idt_ntb_readreg(cfg, IDT_NT_PCI_NTINTSTS) &
+ ~idt_ntb_readreg(cfg, IDT_NT_PCI_NTINTMSK);
+ for_each_set_bit_u32(ntintsts, setbit) {
+ /* Handle the cause of the interrupt */
+ switch (setbit) {
+ case MSGINT_BIT:
+ /* Mask the message IRQs until the data is handled. It
+ * must be unmasked within the tasklet right after the
+ * data is read so the next message can be retrieved */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_MASK);
+ /* Schedule the tasklet to handle the new message */
+ tasklet_schedule(&pdata->msg_tasklet);
+ break;
+ case DBINT_BIT:
+ /* Mask the doorbell IRQs until the data is handled. It
+ * must be unmasked within the tasklet right after the
+ * doorbell status bits are read and clear so the next
+ * doorbell event can be raised */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_INDBELLMSK, INDB_MASK);
+ /* Schedule the tasklet to handle the set doorbell bits */
+ tasklet_schedule(&pdata->db_tasklet);
+ break;
+ case SEINT_BIT:
+ /* Just call the switch event handler. It doesn't do
+ * much work */
+ idt_ntb_se_isr(pdata);
+ break;
+ case TEMPINT_BIT:
+ /* Just call the temperature sensor event handler.
+ * It doesn't do much work */
+ idt_ntb_temp_isr(pdata);
+ break;
+ default:
+ dev_err_data(pdata,
+ "Invalid IDT IQR status bit is set");
+ break;
+ }
+ /* If there is any interrupt bit is set then we handle it */
+ status = IRQ_HANDLED;
+ }
+
+ return status;
+}
+
+/*===========================================================================
+ * 8. NTB bus initialization
+ *===========================================================================*/
+
+/*
+ * NTB KAPI operations
+ *
+ * NOTE This driver implements the synchronous interface only.
+ */
+static const struct ntb_dev_ops idt_ntb_ops = {
+ .link_is_up = idt_ntb_link_is_up,
+ .link_enable = idt_ntb_link_enable,
+ .link_disable = idt_ntb_link_disable,
+ .mw_count = idt_ntb_mw_count,
+ .mw_get_maprsc = idt_ntb_mw_get_maprsc,
+ .mw_get_align = idt_ntb_mw_get_align,
+ .mw_set_trans = idt_ntb_mw_set_trans,
+ .peer_mw_count = idt_ntb_peer_mw_count,
+ .peer_mw_get_align = idt_ntb_peer_mw_get_align,
+ .db_valid_mask = idt_ntb_db_valid_mask,
+ .db_vector_count = idt_ntb_db_vector_count,
+ .db_vector_mask = idt_ntb_db_vector_mask,
+ .db_read = idt_ntb_db_read,
+ .db_set = idt_ntb_db_set,
+ .db_clear = idt_ntb_db_clear,
+ .db_read_mask = idt_ntb_db_read_mask,
+ .db_set_mask = idt_ntb_db_set_mask,
+ .db_clear_mask = idt_ntb_db_clear_mask,
+ .peer_db_set = idt_ntb_peer_db_set,
+ .msg_post = idt_ntb_msg_post,
+ .msg_size = idt_ntb_msg_size
+};
+
+/*
+ * NTB devices registration function
+ */
+static int idt_ntb_register_devs(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_dev *ndev;
+ int id, ret;
+
+ /* Loop over all the NTB devices initializing the necessary fields */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Retrieve the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Set the device operation callbacks */
+ ndev->ntb.ops = &idt_ntb_ops;
+
+ /* Register the device */
+ ret = ntb_register_device(&ndev->ntb);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to register NTB device");
+ goto err_unregister_device;
+ }
+ }
+
+ dev_dbg_data(pdata, "IDT NTB device(s) successfully registered");
+
+ return SUCCESS;
+
+err_unregister_device:
+ for (id--; 0 <= id; id--) {
+ ndev = &pdata->ndevs[id];
+ ntb_unregister_device(&ndev->ntb);
+ }
+
+ return ret;
+}
+
+/*
+ * NTB devices unregistration function
+ */
+static void idt_ntb_unregister_devs(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_dev *ndev;
+ int id;
+
+ /* Loop over all the NTB devices initializing the necessary fields */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Retrieve the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Just unregister the device */
+ ntb_unregister_device(&ndev->ntb);
+ }
+
+ dev_dbg_data(pdata, "IDT NTB devices are practically unregistered");
+}
+
+/*===========================================================================
+ * 9. IDT NT-functions topology
+ *===========================================================================*/
+
+/*
+ * Add the NT-function pair of Primary and Secondary ports to the topology
+ */
+static inline void idt_ntb_addntb(struct idt_ntb_topo *topo,
+ const unsigned char pri,
+ const unsigned char sec)
+{
+ topo->priports |= ((u32)1 << pri);
+ topo->secports[pri] |= ((u32)1 << sec);
+}
+
+/*
+ * Retrieve the port role
+ */
+static inline enum ntb_topo idt_ntb_portrole(const struct idt_ntb_topo *topo,
+ const unsigned char port)
+{
+ return ((topo->priports & ((u32)1 << port)) ?
+ NTB_TOPO_PRI : NTB_TOPO_SEC);
+}
+
+/*
+ * Function first checks whether the port can have an NT-function then whether
+ * the function is activated on the port
+ */
+static int idt_ntb_checkport(const struct idt_ntb_data *pdata,
+ const unsigned char port)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ unsigned char pid;
+ u32 sts, mode;
+ int stat = -EINVAL;
+
+ /* Check whether the port can have the NT-function */
+ for (pid = 0; pid < pdata->swcfg->port_cnt; pid++) {
+ if (pdata->swcfg->ports[pid] == port) {
+ stat = SUCCESS;
+ break;
+ }
+ }
+ /* Return -EINVAL if it can't */
+ if (SUCCESS != stat) {
+ return -EINVAL;
+ }
+
+ /* Get the port status so to determine the port mode */
+ sts = idt_ntb_readreg(cfg, portdata_tbl[port].sts);
+ mode = idt_ntb_readfld_var(sts, IDT_SW_PORT_MODE);
+
+ /* Check whther the port has the NT-function */
+ if (PORTMODE_NT != mode && PORTMODE_USNT != mode &&
+ PORTMODE_USNTDMA != mode) {
+ return -EINVAL;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Scan the IDT NT-function topology by reading the NTSDATA register
+ * That register is initialized with the Primary port number of the
+ * corresponding secondary ports. Of course the algorithm doesn't permit the
+ * two Primary ports pointing to each other.
+ */
+static int idt_ntb_scantopo(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_topo *topo = &pdata->topo;
+ unsigned char pid;
+ unsigned long port;
+ u32 priport;
+ int ret;
+
+ /* Clean the topo structure */
+ memset(topo, 0, sizeof(*topo));
+
+ /* Walk through all the available ports checking whether the
+ * NT-function enabled on them. If so retrieve its Primary side port */
+ for (pid = 0; pid < pdata->swcfg->port_cnt; pid++) {
+ /* Retrieve the port number */
+ port = pdata->swcfg->ports[pid];
+
+ /* Check whether the port has the NT-function
+ * NOTE Within this loop we are sure it can */
+ if (SUCCESS == idt_ntb_checkport(pdata, port)) {
+ /* If it does then read it's NTSDATA interpreting its
+ * value as the Primary port number */
+ priport = idt_ntb_readreg(cfg,
+ portdata_tbl[port].ntsdata);
+
+ /* Add the NTB to the topology only if the retrieved
+ * primary port can have NT-function and have it
+ * activated */
+ ret = idt_ntb_checkport(pdata, priport);
+ if (SUCCESS == ret && port != priport) {
+ idt_ntb_addntb(topo, priport, port);
+ /* Increment the number of NTB pairs */
+ topo->paircnt++;
+ }
+
+ /* If the retrieved port either can't have the
+ * NT-function or doesn't have NT-function activated
+ * then the topology is corrupted */
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Invalid primary NT port %u was read",
+ priport);
+ return -EINVAL;
+ }
+ } /* else { just skip it }*/
+ }
+
+ /* Check the topology consistency to make sure it is just downwards
+ * directional tree graph with two levels: one primary root and
+ * a number of secondary lists (can be none) */
+ for_each_set_bit_u32(topo->priports, port) {
+ /* Check whether there is no any Primary port amongst the
+ * Secondary ports */
+ if (topo->secports[port] & topo->priports) {
+ dev_err_data(pdata,
+ "Port %lu has Primary and Secondary roles,"
+ "IDT NTB topology is inconsistent", port);
+ return -EINVAL;
+ }
+ }
+
+ dev_dbg_data(pdata, "IDT NTB functions topology has been scanned");
+
+ return SUCCESS;
+}
+
+/*
+ * Create set of Secondary sided peer devices of the topology
+ * The function is used by the Primary side of the topology
+ */
+static int idt_ntb_secpeers(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_topo *topo = &pdata->topo;
+ u32 secports, portsts;
+ unsigned char id = 0;
+ unsigned long port;
+ int node;
+
+ /* Get the set of the Secondary ports of the current Primary port */
+ secports = topo->secports[pdata->port];
+
+ /* Calculate the number of peers */
+ pdata->peer_cnt = hweight32(secports);
+
+ /* Allocate the memory for all the peers IDT NTB device structures */
+ node = dev_to_node(to_dev_data(pdata));
+ pdata->ndevs = kzalloc_node(pdata->peer_cnt*sizeof(*pdata->ndevs),
+ GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(pdata->ndevs)) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for Secondary peer devices");
+ return -ENOMEM;
+ }
+
+ /* Walk through all the secondary ports initializing the
+ * corresponding NTB device and data fields */
+ for_each_set_bit_u32(secports, port) {
+ /* Read the port status register to retrieve the partition */
+ portsts = idt_ntb_readreg(cfg, portdata_tbl[port].sts);
+
+ /* Save the port and partition numbers */
+ pdata->ndevs[id].port = port;
+ pdata->ndevs[id].part =
+ idt_ntb_readfld_var(portsts, IDT_SW_PORT_SWPART);
+
+ /* Initialize the local topology and PCI device fields */
+ pdata->ndevs[id].ntb.topo = pdata->role;
+ pdata->ndevs[id].ntb.pdev = pdata->pdev;
+
+ /* Increment the device id number */
+ id++;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Create Primary sided peer device of the topology
+ * The function is used by the Secondary side of the topology
+ */
+static int idt_ntb_pripeer(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 priport, portsts;
+ int node;
+
+ /* Get the Primary port of the current port */
+ priport = idt_ntb_readreg(cfg, portdata_tbl[pdata->port].ntsdata);
+
+ /* There is going to be just one peer */
+ pdata->peer_cnt = 1;
+
+ /* Allocate the memory for IDT NTB device structure of just one peer */
+ node = dev_to_node(to_dev_data(pdata));
+ pdata->ndevs = kzalloc_node(sizeof(*pdata->ndevs), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(pdata->ndevs)) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for Primary peer device");
+ return -ENOMEM;
+ }
+
+ /* Read the port status register to retrieve the partition */
+ portsts = idt_ntb_readreg(cfg, portdata_tbl[priport].sts);
+
+ /* Save the peer id, port and partition numbers */
+ pdata->ndevs->port = priport;
+ pdata->ndevs->part = idt_ntb_readfld_var(portsts, IDT_SW_PORT_SWPART);
+
+ /* Initialize the local topology and PCI device fields */
+ pdata->ndevs->ntb.topo = pdata->role;
+ pdata->ndevs->ntb.pdev = pdata->pdev;
+
+ return SUCCESS;
+}
+
+/*
+ * Enumerate the peer pairs
+ *
+ * Basically the pairid is just the order number of the corresponding
+ * Secondary side port. So the function just loop over the Primary ports.
+ * If the local port is Primary then just linearly enumerate its peers
+ * starting from the corresponding number.
+ * If the local port is Secondary then the function walks through
+ * all the Secondary port of the corresponding Primary port looking
+ * for the current one to assign the simultaniously incremented id.
+ */
+static void idt_ntb_enumpairs(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_topo *topo = &pdata->topo;
+ unsigned char id, pairid = 0;
+ unsigned long priport, secport;
+ u32 secports;
+
+ /* Loop over all the Primary ports calculating the pairids */
+ for_each_set_bit_u32(topo->priports, priport) {
+ /* Retrieve the Secondary ports connected to the current
+ * Primary one */
+ secports = topo->secports[priport];
+
+ /* Enumerate the current port related pairs */
+ /* If current port is Primary then enumerate its peers */
+ if (NTB_TOPO_PRI == pdata->role && priport == pdata->port) {
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ pdata->ndevs[id].pairid = pairid + id;
+ }
+ /* Stop looping, the job is done */
+ break;
+ }
+ /* If the current port is Secondary then retrieve its peer id
+ * within the corresponding Primary port */
+ else if (NTB_TOPO_SEC == pdata->role &&
+ priport == pdata->ndevs[0].port) {
+ id = 0;
+ for_each_set_bit_u32(secports, secport) {
+ if (secport == pdata->port) {
+ pdata->ndevs[0].pairid = pairid + id;
+ break;
+ }
+ id++;
+ }
+ /* Stop looping, the job is done */
+ break;
+ }
+
+ /* Increment the pairid with the number of the related Secondary
+ * ports */
+ pairid += hweight32(secports);
+ }
+}
+
+/*
+ * Create the NTB devices with respect to the topology
+ */
+static int idt_ntb_addpeers(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_topo *topo = &pdata->topo;
+ u32 portsts;
+ int ret;
+
+ /* Retrieve the current port number */
+ pdata->port = idt_ntb_readfld_mem(cfg, IDT_NT_PORTNUM);
+
+ /* Read the current port partition number */
+ portsts = idt_ntb_readreg(cfg, portdata_tbl[pdata->port].sts);
+ pdata->part = idt_ntb_readfld_var(portsts, IDT_SW_PORT_SWPART);
+
+ /* Check whether the current port role is Primary or Secondary */
+ pdata->role = idt_ntb_portrole(topo, pdata->port);
+
+ /* Create either the Primary or Secondary side peers set */
+ ret = (NTB_TOPO_PRI == pdata->role) ?
+ idt_ntb_secpeers(pdata) : idt_ntb_pripeer(pdata);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+
+ /* Enumerate all the NTB connected pairs */
+ idt_ntb_enumpairs(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB peer devices created");
+
+ return SUCCESS;
+}
+
+/*
+ * Remove the peer NTB devices added to the data structure
+ */
+static void idt_ntb_delpeers(struct idt_ntb_data *pdata)
+{
+ /* Release the memory occupied by the */
+ kfree(pdata->ndevs);
+
+ dev_dbg_data(pdata, "IDT NTB peer devices discarded");
+}
+
+/*===========================================================================
+ * 10. Basic initialization functions
+ *===========================================================================*/
+
+/*
+ * Check whether the device is properly pre-initialized
+ */
+static int idt_ntb_check_quirks(struct pci_dev *pdev)
+{
+ u32 data, fld;
+ int ret;
+
+ /* Read the BARSETUP0 */
+ ret = pci_read_config_dword(pdev, BARSETUP0_OFF, &data);
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to read BARSETUP0 configuration register");
+ return ret;
+ }
+
+ /* Check whether the BAR0 register is enabled */
+ if (OFF == idt_ntb_readfld_var(data, IDT_NT_BARSTP_EN)) {
+ dev_err(&pdev->dev,
+ "BAR0 isn't enabled");
+ return -EINVAL;
+ }
+
+ /* Check whether the BAR0 maps the registers configuration space */
+ fld = idt_ntb_readfld_var(data, IDT_NT_BARSTP_MODE);
+ if (BARSTP_MODE_CFGSPC != fld) {
+ dev_err(&pdev->dev,
+ "BAR0 isn't configured to map the configuration space");
+ return -EINVAL;
+ }
+
+ /* Read the BARSETUP2 */
+ ret = pci_read_config_dword(pdev, BARSETUP2_OFF, &data);
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to read BARSETUP2 configuration register");
+ return ret;
+ }
+
+ /* Check whether the BAR2 register is enabled */
+ if (OFF == idt_ntb_readfld_var(data, IDT_NT_BARSTP_EN)) {
+ dev_err(&pdev->dev,
+ "BAR2 isn't enabled");
+ return -EINVAL;
+ }
+
+ /* Check whether the BAR2 maps memory windows */
+ fld = idt_ntb_readfld_var(data, IDT_NT_BARSTP_MODE);
+ if (BARSTP_MODE_WNDW != fld) {
+ dev_err(&pdev->dev,
+ "BAR2 isn't configured to map memory windows");
+ return -EINVAL;
+ }
+
+ /* Check whether the BAR2 maps the 24-entries lookup table */
+ fld = idt_ntb_readfld_var(data, IDT_NT_BARSTP_ATRAN);
+ if (BARSTP_ATRAN_LU24 != fld) {
+ dev_err(&pdev->dev,
+ "BAR2 isn't configured to map 24-entries lookup table");
+ return -EINVAL;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Create the IDT PCIe-swtich driver data structure performing the basic
+ * initialization
+ */
+static struct idt_ntb_data *idt_ntb_create_data(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct idt_ntb_data *pdata;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&pdev->dev);
+ pdata = kzalloc_node(sizeof(*pdata), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(pdata)) {
+ dev_err(&pdev->dev,
+ "Failed to allocate memory for IDT NTB driver data");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Create the workqueue used by the driver */
+ pdata->idt_wq = create_workqueue(NTB_WQNAME);
+ if (IS_ERR_OR_NULL(pdata->idt_wq)) {
+ dev_err(&pdev->dev, "Failed to create workqueue");
+ goto err_kfree;
+ }
+
+ /* Put the IDT driver data pointer to the PCI-device private pointer */
+ pci_set_drvdata(pdev, pdata);
+ /* Save the PCI-device pointer inside the data structure */
+ pdata->pdev = pdev;
+ /* Save the IDT PCIe-switch ports configuration */
+ pdata->swcfg = (struct idt_89hpes_pdata *)id->driver_data;
+
+ dev_dbg_data(pdata, "IDT NTB device data created");
+
+ return pdata;
+
+err_kfree:
+ kfree(pdata);
+
+ return NULL;
+}
+
+/*
+ * Free the IDT PCie-swtich driver data structure
+ */
+static void idt_ntb_free_data(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+
+ /* Flush and destroy the workqueue */
+ flush_workqueue(pdata->idt_wq);
+ destroy_workqueue(pdata->idt_wq);
+
+ /* Clean the private data pointer of the PCI-device structure */
+ pci_set_drvdata(pdev, NULL);
+
+ /* Free the memory allocated for the IDT NTB driver data */
+ kfree(pdata);
+
+ dev_dbg(&pdev->dev, "IDT NTB device data discarded");
+}
+
+/*
+ * Initialize the basic PCI-related subsystem
+ */
+static int idt_ntb_init_pci(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+ int ret;
+
+ /* Enable the device advanced error reporting. Don't check the return
+ * value since the service might be disabled from the kernel */
+ ret = pci_enable_pcie_error_reporting(pdev);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to enable AER capability of IDT NTB");
+ }
+ /* Cleanup the uncorrectable error status before starting the rest of
+ * initialization */
+ pci_cleanup_aer_uncorrect_error_status(pdev);
+
+ /* First enable the PCI device */
+ ret = pci_enable_device(pdev);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to enable the PCI device");
+ goto err_disable_aer;
+ }
+
+ /* Reguest the PCI device resources like the BAR memory mapping, etc
+ * It's done for BAR0 for now */
+ ret = pci_request_region(pdev, BAR0, NTB_NAME);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Failed to request the PCI BAR0 resources");
+ goto err_disable_device;
+ }
+
+ /* Initialize the bit mask of DMA although I don't see where it can be
+ * used for now */
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (SUCCESS != ret) {
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to set any DMA bit mask\n");
+ goto err_release_region;
+ }
+ dev_warn_data(pdata, "Cannot set the DMA highmem bit mask\n");
+ }
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (SUCCESS != ret) {
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Failed to set any consistent DMA bit mask\n");
+ goto err_release_region;
+ }
+ dev_warn_data(pdata,
+ "Cannot set the consistent DMA highmem bit mask\n");
+ }
+
+ /* Retrieve the virtual address of the PCI configuration space */
+ pdata->cfg_mmio = pci_iomap(pdev, BAR0, 0);
+ if (IS_ERR_OR_NULL(pdata->cfg_mmio)) {
+ dev_err_data(pdata,
+ "Failed to map the IDT NT-function config space\n");
+ ret = -EIO;
+ goto err_release_region;
+ }
+
+ dev_dbg_data(pdata, "IDT NTB function PCI interface was initialized");
+
+ return SUCCESS;
+
+err_disable_aer:
+ (void)pci_disable_pcie_error_reporting(pdev);
+err_release_region:
+ pci_release_region(pdev, BAR0);
+err_disable_device:
+ pci_disable_device(pdev);
+
+ return ret;
+}
+
+/*
+ * Deinitialize the basic PCI-related subsystem
+ */
+static void idt_ntb_deinit_pci(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+
+ /* Disable the AER capability */
+ (void)pci_disable_pcie_error_reporting(pdev);
+
+ /* Unmap the IDT PCIe-switch configuration space */
+ pci_iounmap(pdev, pdata->cfg_mmio);
+
+ /* Release the PCI-device BAR0 resources */
+ pci_release_region(pdev, BAR0);
+
+ /* Finally disable the PCI device */
+ pci_disable_device(pdev);
+
+ dev_dbg_data(pdata, "IDT NTB function PCI interface was cleaned");
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t idt_ntb_dbgfs_info_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+static ssize_t idt_ntb_dbgfs_ntregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+static ssize_t idt_ntb_dbgfs_swregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+/*
+ * Driver DebugFS info file operations
+ */
+static const struct file_operations idt_ntb_dbgfs_info_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = idt_ntb_dbgfs_info_read
+};
+
+/*
+ * Driver DebugFS NT registers file operations
+ */
+static const struct file_operations idt_ntb_dbgfs_ntregs_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = idt_ntb_dbgfs_ntregs_read
+};
+
+/*
+ * Driver DebugFS IDT PCIe-swtich global registers file operations
+ */
+static const struct file_operations idt_ntb_dbgfs_swregs_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = idt_ntb_dbgfs_swregs_read
+};
+
+/*
+ * DebugFS read info node callback
+ */
+static ssize_t idt_ntb_dbgfs_info_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ struct idt_ntb_data *pdata = filp->private_data;
+ void __iomem *cfg = pdata->cfg_mmio;
+ enum ntb_speed speed;
+ enum ntb_width width;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+ u32 var;
+ int id, sts, part, bdf, port;
+ unsigned char temp, frac;
+
+ /* Lets limit the buffer size the way the Intel/AMD drivers do */
+ size = min_t(size_t, count, 0x1000U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tIDT PCIe-switch NT-function Information:\n\n");
+
+ /* General device configurations */
+ off += scnprintf(strbuf + off, size - off,
+ "Switch port\t\t\t- %hhu\n", pdata->port);
+ off += scnprintf(strbuf + off, size - off,
+ "Port partition\t\t\t- %hhu\n", pdata->part);
+ off += scnprintf(strbuf + off, size - off,
+ "Number of peers\t\t\t- %hhu\n", pdata->peer_cnt);
+
+ /* Local switch NT-function role topology and available port to
+ * communicate to */
+ off += scnprintf(strbuf + off, size - off,
+ "NT-function role\t\t- %s\n", ntb_topo_string(pdata->role));
+ off += scnprintf(strbuf + off, size - off,
+ "Peer Port:Partition available\t- ");
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "%hhd:%hhd ",
+ pdata->ndevs[id].port, pdata->ndevs[id].part);
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+ /* Links status */
+ var = idt_ntb_readreg(cfg, portdata_tbl[pdata->port].sts);
+ if (idt_ntb_readfld_var(var, IDT_SW_PORT_LNKUP)) {
+ off += scnprintf(strbuf + off, size - off,
+ "Local Port Link status\t\t- ");
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_PCIELSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "PCIe Gen %u ",
+ idt_ntb_readfld_var(var, IDT_NT_CURLNKSPD));
+ off += scnprintf(strbuf + off, size - off,
+ "x%u lanes\n",
+ idt_ntb_readfld_var(var, IDT_NT_CURLNKWDTH));
+ } else {
+ off += scnprintf(strbuf + off, size - off,
+ "Local port link status\t\t- Down (Weird)\n");
+ }
+ off += scnprintf(strbuf + off, size - off,
+ "Peer ports link status\t\t- ");
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ sts = idt_ntb_link_is_up(&pdata->ndevs[id].ntb, &speed, &width);
+ if (ON == sts) {
+ off += scnprintf(strbuf + off, size - off,
+ "%hhd:Gen %u x%u, ", pdata->ndevs[id].port, speed, width);
+ } else /* if (OFF == sts) */ {
+ off += scnprintf(strbuf + off, size - off,
+ "%hhd:Down, ", pdata->ndevs[id].port);
+ }
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+ /* General resources information */
+ off += scnprintf(strbuf + off, size - off,
+ "Total doorbells count\t\t- %u\n", IDT_NTB_DBELL_CNT);
+ off += scnprintf(strbuf + off, size - off,
+ "Total memory windows count\t- %u\n", IDT_NTB_MW_CNT);
+ off += scnprintf(strbuf + off, size - off,
+ "Total message registers count\t- %u\n", IDT_NTB_MSG_CNT);
+
+ /* Common resources state */
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_GDBELLSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Global doorbells status\t\t- %#010x\n", var);
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_INDBELLSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Local doorbells status\t\t- %#010x\n", var);
+ off += scnprintf(strbuf + off, size - off,
+ "Mirror doorbells value\t\t- %#010x\n",
+ pdata->db_sts);
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_INDBELLMSK);
+ off += scnprintf(strbuf + off, size - off,
+ "Local doorbells mask\t\t- %#010x\n", var);
+ off += scnprintf(strbuf + off, size - off,
+ "Mirror doorbells mask value\t- %#010x\n",
+ pdata->db_msk);
+
+ /* Per-device resources */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "Port %hhd (pair id %hhd)\n",
+ pdata->ndevs[id].port, pdata->ndevs[id].pairid);
+ off += scnprintf(strbuf + off, size - off,
+ "\tDoorbells share\t- "
+ "local %#010x offset %hhu, peer %#010x offset %hhu\n",
+ pdata->ndevs[id].db_self_mask,
+ pdata->ndevs[id].db_self_offset,
+ pdata->ndevs[id].db_peer_mask,
+ pdata->ndevs[id].db_peer_offset);
+ off += scnprintf(strbuf + off, size - off,
+ "\tDoorbells\t- count %hhu, valid mask: %#010x,\n",
+ pdata->ndevs[id].db_cnt,
+ pdata->ndevs[id].db_valid_mask);
+ off += scnprintf(strbuf + off, size - off,
+ "\tMemory windows\t- local/peer count %hhu/%hhu, "
+ "size %u bytes, local offset: %hhu\n",
+ pdata->ndevs[id].mw_self_cnt,
+ pdata->ndevs[id].mw_peer_cnt,
+ (unsigned int)pdata->mw_size,
+ pdata->ndevs[id].mw_self_offset);
+ }
+
+ /* Doorbells mapping */
+ off += scnprintf(strbuf + off, size - off,
+ "\nInbound db:part mapping\n\t");
+ for (id = 0; id < IDT_NTB_DBELL_CNT; id++) {
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_GIDBELLMSK0 + id);
+ off += scnprintf(strbuf + off, size - off, "%02d:", id);
+ for_each_set_bit_u32(~var & 0xFF, part) {
+ off += scnprintf(strbuf + off, size - off, "%d,", part);
+ }
+ off += scnprintf(strbuf + off, size - off, "\b; ");
+ if (0 == ((id + 1) % 10)) {
+ off += scnprintf(strbuf + off, size - off, "\n\t");
+ }
+ }
+ off += scnprintf(strbuf + off, size - off,
+ "\nOutbound db:part mapping\n\t");
+ for (id = 0; id < IDT_NTB_DBELL_CNT; id++) {
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_GODBELLMSK0 + id);
+ off += scnprintf(strbuf + off, size - off, "%02d:", id);
+ for_each_set_bit_u32(~var & 0xFF, part) {
+ off += scnprintf(strbuf + off, size - off,
+ "%d,", part);
+ }
+ off += scnprintf(strbuf + off, size - off, "\b; ");
+ if (0 == ((id + 1) % 10)) {
+ off += scnprintf(strbuf + off, size - off, "\n\t");
+ }
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+ /* NTB control register */
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_NTCTL);
+ off += scnprintf(strbuf + off, size - off,
+ "\nNTB control register\t- %#010x\n", var);
+
+ /* NTB Mapping table */
+ off += scnprintf(strbuf + off, size - off,
+ "NTB mapping table\n");
+ for (id = 0; id < IDT_NTB_MTBL_ENTRY_CNT; id++) {
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLADDR, (u32)id);
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_NTMTBLDATA);
+ if (ON == idt_ntb_readfld_var(var, IDT_NT_MTBL_VALID)) {
+ bdf = idt_ntb_readfld_var(var, IDT_NT_MTBL_BDF);
+ off += scnprintf(strbuf + off, size - off,
+ "\t%02d: part %d, bus %d, dev %d, func %d\n",
+ id, idt_ntb_readfld_var(var, IDT_NT_MTBL_PART),
+ (bdf >> 8) & 0xFF, (bdf >> 3) & 0x1F, bdf & 7);
+ }
+ }
+
+ /* Currently enabled IRQs */
+ off += scnprintf(strbuf + off, size - off, "\nNTB interrupts status\n");
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_NTINTMSK);
+ for_each_set_bit_u32(ALLINT_MASK, id) {
+ switch (id) {
+ case MSGINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tMessage interrupts\t\t\t\t- ");
+ break;
+ case DBINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tDoorbell interrupts\t\t\t\t- ");
+ break;
+ case SEINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tSwitch event interrupts\t\t\t\t- ");
+ break;
+ case FMCI_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tFailover mode change initiated IRQ\t\t- ");
+ break;
+ case FMCC_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tFailover mode change completed IRQ\t\t- ");
+ break;
+ case TEMPINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tTemperature sensor IRQ (T < %d || %d < T)\t- ",
+ IDT_NTB_TEMP_LTH, IDT_NTB_TEMP_HTH);
+ break;
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "\tWarning! Invalid bit is set in the NTINTMSK register\n");
+ break;
+ }
+
+ if (0x0 == (var & BIT_MASK(id))) {
+ off += scnprintf(strbuf + off, size - off,
+ "enabled\n");
+ } else {
+ off += scnprintf(strbuf + off, size - off,
+ "disabled\n");
+ }
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tIDT PCIe-switch general configuration:\n\n");
+
+ /* Boot configuration vector status */
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_BCVSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Switch boot mode\n\t");
+ switch (idt_ntb_readfld_var(var, IDT_SW_SWMODE)) {
+ case (0x0):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition\n");
+ break;
+ case (0x1):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition with Serial EEPROM\n");
+ break;
+ case (0x2):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition with Serial EEPROM Jump 0 "
+ "Initialization\n");
+ break;
+ case (0x3):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition with Serial EEPROM Jump 1 "
+ "Initialization\n");
+ break;
+ case (0x8):
+ off += scnprintf(strbuf + off, size - off,
+ "Single partition with reduced latency\n");
+ break;
+ case (0x9):
+ off += scnprintf(strbuf + off, size - off,
+ "Single partition with Serial EEPROM initialization "
+ "and reduced latency\n");
+ break;
+ case (0xA):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports\n");
+ break;
+ case (0xB):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports and i2c Reset\n");
+ break;
+ case (0xC):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports and Serial EEPROM "
+ "initialization\n");
+ break;
+ case (0xD):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports with i2c Reset "
+ "and Serial EEPROM initialization\n");
+ break;
+ case (0xE):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Disabled ports\n");
+ break;
+ case (0xF):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Disabled ports and Serial EEPROM "
+ "initialization\n");
+ break;
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "Unknown\n");
+ break;
+ }
+ off += scnprintf(strbuf + off, size - off,
+ "Switch boot clock mode\n\t");
+ switch (idt_ntb_readfld_var(var, IDT_SW_CLKMODE)) {
+ case (0x0):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- non-common global clocked\n"
+ "\tOther ports\t- non-common global clocked\n");
+ break;
+ case (0x1):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- Common global clocked\n"
+ "\tOther ports\t- non-common global clocked\n");
+ break;
+ case (0x2):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- non-common global clocked\n"
+ "\tOther ports\t- common global clocked\n");
+ break;
+ case (0x3):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- common global clocked\n"
+ "\tOther ports\t- common global clocked\n");
+ break;
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "Unknown\n");
+ break;
+ }
+
+ /* Per-port link status and clock configuration */
+ off += scnprintf(strbuf + off, size - off,
+ "Ports clocking status\n");
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_PCLKMODE);
+ for (id = 0; id < pdata->swcfg->port_cnt; id++) {
+ port = pdata->swcfg->ports[id];
+ sts = idt_ntb_readreg(cfg, portdata_tbl[port].pcielsts);
+ off += scnprintf(strbuf + off, size - off,
+ "\tPort %d\t- %s %s mode\n", port,
+ idt_ntb_readfld_var(sts, IDT_NT_SCLK) ?
+ "common" : "non-common",
+ idt_ntb_readfld_var(var, IDT_SW_P0CLKMODE + id) ?
+ "local" : "global");
+ }
+
+ /* SMBus configuration */
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_SMBUSSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Slave SMBus address\t- %#04x\n",
+ idt_ntb_readfld_var(var, IDT_SW_SSMBADDR));
+ off += scnprintf(strbuf + off, size - off,
+ "Master SMBus address\t- %#04x\n",
+ idt_ntb_readfld_var(var, IDT_SW_MSMBADDR));
+
+ /* Current temperature */
+ var = idt_ntb_readfld_mem(cfg, IDT_SW_TMP_CURTEMP);
+ idt_ntb_convert_temp(var, &temp, &frac);
+ off += scnprintf(strbuf + off, size - off,
+ "Switch temperature\t- %d.%dC\n", temp, (0 != frac) ? 5 : 0);
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, count, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * Read passed set of registers method for DebugFS nodes
+ */
+static ssize_t idt_ntb_dbgfs_regs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp,
+ enum idt_ntb_cfgreg start,
+ enum idt_ntb_cfgreg end,
+ const char *title)
+{
+ struct idt_ntb_data *pdata = filp->private_data;
+ void __iomem *cfg = pdata->cfg_mmio;
+ enum idt_ntb_cfgreg reg;
+ enum idt_ntb_regtype regtype;
+ ptrdiff_t regoffset;
+ enum idt_ntb_regsize regsize;
+ const char *regdesc;
+ u32 data;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+
+ /* Lets limit the buffer size the way the Intel/AMD drivers do */
+ size = min_t(size_t, count, 0x4000U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ return -ENOMEM;
+ }
+
+ /* Put the title first */
+ off += scnprintf(strbuf + off, size - off, "\n\t\t%s\n\n", title);
+
+ /* Print the header of the registers */
+ off += scnprintf(strbuf + off, size - off, " 03 02 01 00\n");
+
+ /* Scan through the whole passed range reading the addresses, values
+ * and description and printing it to the buffer */
+ for (reg = start; reg < end; reg++) {
+ /* Retrieve the register type, offset, size and description */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, ®desc);
+
+ /* Read the value of the corresponding register */
+ data = idt_ntb_readreg(cfg, reg);
+
+ /* Print the register offset */
+ off += scnprintf(strbuf + off, size - off,
+ "0x%05lX: ", (unsigned long)regoffset);
+
+ /* Then print the value of the register in compliance with the
+ * register size */
+ switch (regsize) {
+ case REGBYTE:
+ off += scnprintf(strbuf + off, size - off,
+ " %02hhX", data);
+ break;
+ case REGWORD:
+ off += scnprintf(strbuf + off, size - off,
+ " %02hhX %02hhX", (data >> 8), data);
+ break;
+ case REGDWORD:
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "%02hhX %02hhX %02hhX %02hhX",
+ (data >> 24), (data >> 16), (data >> 8), data);
+ break;
+ }
+
+
+ /* Then description if going to be the last */
+ off += scnprintf(strbuf + off, size - off,
+ " - %s\n", regdesc);
+ }
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, count, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS read NT-function registers node callback
+ */
+static ssize_t idt_ntb_dbgfs_ntregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ ssize_t size;
+
+ /* Read the values of the NT-related registers */
+ size = idt_ntb_dbgfs_regs_read(filp, ubuf, count, offp,
+ 0, IDT_NTB_CFGREGS_SPLIT, "NT-function registers raw values");
+
+ return size;
+}
+
+/*
+ * DebugFS read IDT PCIe-switch registers node info callback
+ */
+static ssize_t idt_ntb_dbgfs_swregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ ssize_t size;
+
+ /* Read the values of the IDT PCIe-swtich global registers */
+ size = idt_ntb_dbgfs_regs_read(filp, ubuf, count, offp,
+ IDT_NTB_CFGREGS_SPLIT + 1, IDT_NTB_CFGREGS_END,
+ "IDT PCIe-switch global registers raw values");
+
+ return size;
+}
+
+/*
+ * Driver DebugFS initialization function
+ */
+static int idt_ntb_init_dbgfs(struct idt_ntb_data *pdata)
+{
+ const char *devname;
+ struct dentry *dbgfs_info, *dbgfs_ntregs, *dbgfs_swregs;
+ int ret = 0;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_topdir)) {
+ dev_info_data(pdata,
+ "Top DebugFS directory has not been created for "
+ NTB_NAME);
+ return PTR_ERR(dbgfs_topdir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(to_dev_data(pdata));
+
+ /* Create the top directory of the device */
+ pdata->dbgfs_dir = debugfs_create_dir(devname, dbgfs_topdir);
+ if (IS_ERR(pdata->dbgfs_dir)) {
+ dev_dbg_data(pdata, "Could not create the DebugFS dir %s for %s",
+ devname, NTB_NAME);
+ return PTR_ERR(pdata->dbgfs_dir);
+ }
+
+ /* Create the info file node */
+ dbgfs_info = debugfs_create_file("info", S_IRUSR,
+ pdata->dbgfs_dir, pdata, &idt_ntb_dbgfs_info_ops);
+ if (IS_ERR(dbgfs_info)) {
+ dev_dbg_data(pdata, "Could not create the DebugFS info node");
+ ret = PTR_ERR(dbgfs_info);
+ goto err_rm_dir;
+ }
+
+ /* Create the NT-registers file node */
+ dbgfs_ntregs = debugfs_create_file("ntregs", S_IRUSR,
+ pdata->dbgfs_dir, pdata, &idt_ntb_dbgfs_ntregs_ops);
+ if (IS_ERR(dbgfs_ntregs)) {
+ dev_dbg_data(pdata,
+ "Could not create the DebugFS NT-registers node");
+ ret = PTR_ERR(dbgfs_ntregs);
+ goto err_rm_dir;
+ }
+
+ /* Create the NT-registers file node */
+ dbgfs_swregs = debugfs_create_file("swregs", S_IRUSR,
+ pdata->dbgfs_dir, pdata, &idt_ntb_dbgfs_swregs_ops);
+ if (IS_ERR(dbgfs_swregs)) {
+ dev_dbg_data(pdata,
+ "Could not create the DebugFS global registers node");
+ ret = PTR_ERR(dbgfs_swregs);
+ goto err_rm_dir;
+ }
+
+ dev_dbg_data(pdata, "IDT NTB device DebugFS nodes created");
+
+ return SUCCESS;
+
+ /* Following call will remove all the subfiles in the directory */
+err_rm_dir:
+ debugfs_remove_recursive(pdata->dbgfs_dir);
+
+ return ret;
+}
+
+/*
+ * Driver DebugFS deinitialization function
+ */
+static void idt_ntb_deinit_dbgfs(struct idt_ntb_data *pdata)
+{
+ debugfs_remove_recursive(pdata->dbgfs_dir);
+
+ dev_dbg_data(pdata, "IDT NTB device DebugFS nodes discarded");
+}
+
+/*===========================================================================
+ * 12. PCI bus callback functions
+ *===========================================================================*/
+
+/*
+ * PCI device probe() callback function
+ */
+static int idt_ntb_pci_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct idt_ntb_data *pdata;
+ int ret;
+
+ /* Check whether the kernel has properly fixed the IDT NTB
+ * function up */
+ ret = idt_ntb_check_quirks(pdev);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+
+ /* Allocate the memory for the IDT PCIe-swtich NTB driver data */
+ pdata = idt_ntb_create_data(pdev, id);
+ if (IS_ERR_OR_NULL(pdata)) {
+ return PTR_ERR(pdata);
+ }
+
+ /* Initialize the basic PCI subsystem of the device */
+ ret = idt_ntb_init_pci(pdata);
+ if (SUCCESS != ret) {
+ goto err_free_data;
+ }
+
+ /* Determine the ports NT-functions predefined topology */
+ ret = idt_ntb_scantopo(pdata);
+ if (SUCCESS != ret) {
+ goto err_deinit_pci;
+ }
+
+ /* Add all the peers */
+ ret = idt_ntb_addpeers(pdata);
+ if (SUCCESS != ret) {
+ goto err_deinit_pci;
+ }
+
+ /* Initialize the doorbells */
+ idt_ntb_init_db(pdata);
+
+ /* Allocate the Memory Window resources */
+ ret = idt_ntb_init_mws(pdata);
+ if (SUCCESS != ret) {
+ goto err_freedb;
+ }
+
+ /* Init messaging subsystem */
+ ret = idt_ntb_init_msg(pdata);
+ if (SUCCESS != ret) {
+ goto err_clean_mws;
+ }
+
+ /* Start the link polling subsystem */
+ idt_ntb_init_link(pdata);
+
+ /* Initialize the PCIe interrupts */
+ ret = idt_ntb_init_isr(pdata);
+ if (SUCCESS != ret) {
+ goto err_clear_link;
+ }
+
+ /* Register all the devices on the NTB bus */
+ ret = idt_ntb_register_devs(pdata);
+ if (SUCCESS != ret) {
+ goto err_clear_isr;
+ }
+
+ /* Initialize the DebugFS node of the IDT PCIe-switch NTB driver.
+ * Don't pay much attention to this even if it failed */
+ (void)idt_ntb_init_dbgfs(pdata);
+
+ /* IDT PCIe-switch NTB driver is finally initialized */
+ dev_info_data(pdata, "IDT PCIe-swtich NTB devices are ready");
+
+ /* May the force be with us... */
+ return SUCCESS;
+
+err_clear_isr:
+ idt_ntb_clear_isr(pdata);
+err_clear_link:
+ idt_ntb_clear_link(pdata);
+/*err_deinit_msg:*/
+ idt_ntb_deinit_msg(pdata);
+err_clean_mws:
+ idt_ntb_clean_mws(pdata);
+err_freedb:
+ idt_ntb_clean_db(pdata);
+/*err_delpeers:*/
+ idt_ntb_delpeers(pdata);
+err_deinit_pci:
+ idt_ntb_deinit_pci(pdata);
+err_free_data:
+ idt_ntb_free_data(pdata);
+
+ return ret;
+}
+
+/*
+ * PCI device remove() callback function
+ */
+static void idt_ntb_pci_remove(struct pci_dev *pdev)
+{
+ struct idt_ntb_data *pdata = pci_get_drvdata(pdev);
+
+ /* Deinit the DebugFS node */
+ idt_ntb_deinit_dbgfs(pdata);
+
+ /* Unregister the devices from the NTB bus */
+ idt_ntb_unregister_devs(pdata);
+
+ /* Stop the interrupt handler */
+ idt_ntb_clear_isr(pdata);
+
+ /* Stop the link polling subsystem */
+ idt_ntb_clear_link(pdata);
+
+ /* Deinitialize the messaging subsystem */
+ idt_ntb_deinit_msg(pdata);
+
+ /* Clear the memory windows */
+ idt_ntb_clean_mws(pdata);
+
+ /* Free the allocated Doorbells */
+ idt_ntb_clean_db(pdata);
+
+ /* Delete the added peer devices */
+ idt_ntb_delpeers(pdata);
+
+ /* Deinit the basic PCI subsystem */
+ idt_ntb_deinit_pci(pdata);
+
+ /* Free the memory occupied by the data */
+ idt_ntb_free_data(pdata);
+
+ /* IDT PCIe-switch NTB driver is finally initialized */
+ dev_info(&pdev->dev, "IDT PCIe-swtich NTB devices are unloaded");
+
+ /* Sayonara... */
+}
+
+/*
+ * IDT PCIe-switch models ports configuration structures
+ */
+static struct idt_89hpes_pdata idt_89hpes24nt6ag2_config = {
+ .port_cnt = 6, .ports = {0, 2, 4, 6, 8, 12}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt8ag2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt8bg2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes12nt12g2_config = {
+ .port_cnt = 3, .ports = {0, 8, 16}
+};
+static struct idt_89hpes_pdata idt_89hpes16nt16g2_config = {
+ .port_cnt = 4, .ports = {0, 8, 12, 16}
+};
+static struct idt_89hpes_pdata idt_89hpes24nt24g2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt24ag2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt24bg2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+
+/*
+ * PCI-ids table of the supported IDT PCIe-switch devices
+ */
+static const struct pci_device_id idt_ntb_pci_tbl[] = {
+ {IDT_PCI_DEVICE_IDS(89HPES24NT6AG2, idt_89hpes24nt6ag2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT8AG2, idt_89hpes32nt8ag2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT8BG2, idt_89hpes32nt8bg2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES12NT12G2, idt_89hpes12nt12g2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES16NT16G2, idt_89hpes16nt16g2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES24NT24G2, idt_89hpes24nt24g2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT24AG2, idt_89hpes32nt24ag2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT24BG2, idt_89hpes32nt24bg2_config)},
+ {0}
+};
+MODULE_DEVICE_TABLE(pci, idt_ntb_pci_tbl);
+
+/*
+ * IDT PCIe-switch NT-function device driver structure definition
+ */
+static struct pci_driver idt_ntb_pci_driver = {
+ .name = KBUILD_MODNAME,
+ .probe = idt_ntb_pci_probe,
+ .remove = idt_ntb_pci_remove,
+ .id_table = idt_ntb_pci_tbl,
+};
+
+static int __init idt_ntb_pci_driver_init(void)
+{
+ pr_info("%s %s\n", NTB_DESC, NTB_VER);
+
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_topdir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Register the NTB hardware driver to handle the PCI device */
+ return pci_register_driver(&idt_ntb_pci_driver);
+}
+module_init(idt_ntb_pci_driver_init);
+
+static void __exit idt_ntb_pci_driver_exit(void)
+{
+ /* Unregister the NTB hardware driver */
+ pci_unregister_driver(&idt_ntb_pci_driver);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_topdir);
+}
+module_exit(idt_ntb_pci_driver_exit);
+
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt.h b/drivers/ntb/hw/idt/ntb_hw_idt.h
new file mode 100644
index 0000000..ffe3327
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt.h
@@ -0,0 +1,390 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+#ifndef NTB_HW_IDT_H
+#define NTB_HW_IDT_H
+
+#include <linux/types.h>
+#include <linux/ntb.h>
+#include <linux/pci.h>
+#include <linux/pci_ids.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+
+#include "ntb_hw_idt_regmap.h"
+
+/*
+ * Macro is used to create the struct pci_device_id that matches
+ * the supported IDT PCIe-switches
+ * @devname: Capitalized name of the particular device
+ * @data: Variable passed to the driver of the particular device
+ */
+#define IDT_PCI_DEVICE_IDS(devname, data) \
+ .vendor = PCI_VENDOR_ID_IDT, .device = PCI_DEVICE_ID_IDT_##devname, \
+ .subvendor = PCI_ANY_ID, .subdevice = PCI_ANY_ID, \
+ .class = (PCI_CLASS_BRIDGE_OTHER << 8), .class_mask = (0xFFFF00), \
+ .driver_data = (kernel_ulong_t)&data
+
+/*
+ * IDT PCIe-switches device IDs
+ */
+#define PCI_DEVICE_ID_IDT_89HPES24NT6AG2 0x8091
+#define PCI_DEVICE_ID_IDT_89HPES32NT8AG2 0x808F
+#define PCI_DEVICE_ID_IDT_89HPES32NT8BG2 0x8088
+#define PCI_DEVICE_ID_IDT_89HPES12NT12G2 0x8092
+#define PCI_DEVICE_ID_IDT_89HPES16NT16G2 0x8090
+#define PCI_DEVICE_ID_IDT_89HPES24NT24G2 0x808E
+#define PCI_DEVICE_ID_IDT_89HPES32NT24AG2 0x808C
+#define PCI_DEVICE_ID_IDT_89HPES32NT24BG2 0x808A
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ * @BAR0: Operation with BAR0
+ * @BAR2: Operation with BAR2
+ * @BAR4: Operation with BAR4
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+#define BAR0 0
+#define BAR2 2
+#define BAR4 4
+
+/*
+ * Inline helper function to perform the for each set bit looping.
+ *
+ * NOTE We don't use the standard for_each_set_bit because it's unsigned
+ * long aligned, but our registers are u32 sized.
+ */
+static __always_inline int next_bit(u32 var, int bit)
+{
+ int pos;
+
+ pos = ffs(var & ~(((u32)1 << bit) - 1));
+ return (0 == pos || 32 <= bit) ? 32 : (pos - 1);
+}
+
+/*
+ * Perform loop for each set bit of a u32 variable.
+ *
+ * NOTE Size of integer is supposed to be 32-bits or greater so this
+ * "for each"-macro would work.
+ */
+#define for_each_set_bit_u32(var, bit) \
+ for ((bit) = next_bit(var, 0); \
+ bit < 32; \
+ (bit) = next_bit(var, (bit) + 1))
+
+/*
+ * Create a contiguous bitmask starting at bit position @l and ending at
+ * position @h. For example
+ * GENMASK_ULL(39, 21) gives us the 64bit vector 0x000000ffffe00000.
+ */
+#ifndef GENMASK
+#define GENMASK(h, l) \
+ (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+#endif /* !GENMASK */
+
+/*
+ * Number of NTB resource like Doorbell bits, Memory windows
+ * and Message registers
+ */
+#define IDT_NTB_DBELL_CNT 32
+#define IDT_NTB_MW_CNT 24
+#define IDT_NTB_MSG_CNT 4
+#define IDT_NTB_MTBL_ENTRY_CNT 64
+
+/*
+ * General IDT PCIe-switch constant
+ * @IDT_NTB_MAXPORTS_CNT: Maximum number of ports per IDT PCIe-switch
+ * @IDT_NTB_MAXPARTS_CNT: Maximum number of partitions per IDT PCIe-switch
+ * @IDT_PCIE_REGSIZE: Size of the registers in bytes
+ * @IDT_NTB_TRANSALIGN: Alignment of the translated base address
+ * @IDT_NTB_LNKPOLL_TOUT: Timeout of the link polling kernel thread
+ * @IDT_NTB_SENDMSG_TOUT: Timeout of sending the next message to a peer
+ * @IDT_NTB_TEMP_LTH: Lower threshold of the IDT temperature sensor
+ * @IDT_NTB_TEMP_HTH: Higher threshold of the IDT temperature sensor
+ */
+#define IDT_NTB_MAXPORTS_CNT 24
+#define IDT_NTB_MAXPARTS_CNT 8
+#define IDT_PCIE_REGSIZE 4
+#define IDT_NTB_TRANSALIGN 4
+#define IDT_NTB_LNKPOLL_TOUT msecs_to_jiffies(1000)
+#define IDT_NTB_SENDMSG_TOUT msecs_to_jiffies(100)
+#define IDT_NTB_SENDMSG_RETRY 50
+#define IDT_NTB_TEMP_LTH (u32)10
+#define IDT_NTB_TEMP_HTH (u32)85
+
+/*
+ * u32 data atomic structure
+ */
+typedef struct {
+ spinlock_t lock;
+ u32 data;
+} atomic_u32_t;
+
+/*
+ * Queue head with atomic access
+ */
+typedef struct {
+ struct list_head head;
+ spinlock_t lock;
+} queue_atomic_t;
+
+/*
+ * Messages list container
+ * @msg: Message structure
+ * @retry: Number of retries left
+ * @entry: Queue entry
+ */
+struct idt_ntb_msg {
+ struct ntb_msg msg;
+ int retry;
+ struct list_head entry;
+};
+#define to_msg_list_entry(pentry) \
+ (list_entry(pentry, struct idt_ntb_msg, entry))
+
+/*
+ * IDT PCIe-switch model private data
+ * @port_cnt: Total number of NT endpoint ports
+ * @ports: Port ids
+ */
+struct idt_89hpes_pdata {
+ unsigned char port_cnt;
+ unsigned char ports[];
+};
+
+/*
+ * NTB-bus device structure
+ * @ntb: NTB-bus device related structure
+ *
+ * @port: Remote NT-function port
+ * @part: Remote NT-function partition
+ *
+ * @pairid: Global Identifier of Primary-Secondary ports pair
+ *
+ * @lnk_sts: Peer side link status
+ *
+ * @mw_self_cnt: Number of memory windows locally available
+ * @mw_self_offset: Offset of the first memory window in the Lookup table
+ * @mw_peer_cnt: Number of peer memory windows
+ *
+ * @db_cnt: Number of Doorbells for communications with the
+ * peer NT-function
+ * @db_self_offset: Bits offset of the self Doorbells
+ * @db_peer_offset: Bits offset of the peer Doorbells
+ * @db_valid_mask: Doorbell valid mask
+ * @db_self_mask: Mask of the shifted by self_offset doorbells
+ * @db_peer_mask: Mask of the shifted by peer_offset doorbells
+ *
+ * @qinmsg: Queue of inbound messages received from the peer
+ * @inmsg_work: Work thread rising event of new message arrival
+ * @qoutmsg: Queue of outbound messages posted to send to the peer
+ * @outmsg_work: Work thread sending messages
+ */
+struct idt_ntb_dev {
+ struct ntb_dev ntb;
+
+ unsigned char port;
+ unsigned char part;
+
+ unsigned char pairid;
+
+ u32 lnk_sts;
+
+ unsigned char mw_self_cnt;
+ unsigned char mw_self_offset;
+ unsigned char mw_peer_cnt;
+
+ unsigned char db_cnt;
+ unsigned char db_self_offset;
+ unsigned char db_peer_offset;
+ u32 db_valid_mask;
+ u32 db_self_mask;
+ u32 db_peer_mask;
+
+ queue_atomic_t qinmsg;
+ struct work_struct inmsg_work;
+ queue_atomic_t qoutmsg;
+ struct delayed_work outmsg_work;
+};
+#define to_ndev_ntb(pntb) container_of(pntb, struct idt_ntb_dev, ntb)
+#define to_pdev_ndev(ndev) ((ndev)->ntb.pdev)
+#define to_dev_ndev(ndev) (&ndev->ntb.dev)
+#define to_data_ndev(ndev) \
+ ((struct idt_ntb_data *)(pci_get_drvdata(to_pdev_ndev(ndev))))
+#define to_cfg_ndev(ndev) (to_data_ndev(ndev)->cfg_mmio)
+#define to_ndev_inmsg_work(work) \
+ container_of(work, struct idt_ntb_dev, inmsg_work)
+#define to_ndev_outmsg_work(work) \
+ container_of(to_delayed_work(work), struct idt_ntb_dev, outmsg_work)
+
+/*
+ * IDT PCIe-switch NTB bus topology structure
+ * @paircnt: Total number of the NTB pair in the current topology
+ * (it's just the number of Secondary ports)
+ *
+ * @priports: Bitset of Primary ports
+ * @secports: Array of Secondary ports bitsets related to the corresponding
+ * Primary ports
+ */
+struct idt_ntb_topo {
+ unsigned char paircnt;
+
+ u32 priports;
+ u32 secports[IDT_NTB_MAXPORTS_CNT];
+};
+
+/*
+ * Structure related to the local IDT PCIe-switch NT-function
+ * @pdev: Pointer to the PCI-bus device
+ * @swcfg: Pointer to the struct idt_89hpes_pdata related to the current
+ * IDT PCIe-switch
+ *
+ * @port: Local NT-function port
+ * @part: Local NT-function partition
+ *
+ * @topo: Topology of the NT-function ports
+ * @role: Local port role in the IDT topology
+ *
+ * @peer_cnt: Number of possible remote peers
+ * @ndevs: Array of the device-related structures
+ *
+ * @cfg_mmio: Virtual address of the memory mapped configuration space
+ * of the NT-function
+ *
+ * @idt_wq: IDT driver workqueue to setup the link poll and messages
+ * delivery operations
+ *
+ * @lnk_work: Link status polling kernel thread
+ *
+ * @mw_base: Physical address of the memory mapped base address of the
+ * Memory Windows
+ * @mw_size: Size of one Memory Window
+ * @lut_lock: Lookup table access spin lock
+ *
+ * @db_sts: Doorbell status atomic variable
+ * @db_msk: Doorbell mask atomic variable
+ * @db_lock: Doorbell status and mask spin lock
+ * @db_tasklet: Tasklet to handle the doorbell events
+ *
+ * @msg_lock: Messages routing table lock
+ * @msg_cache: Slab cache of the message structures
+ * @msg_tasklet: Tasklet - handler of the incoming messages
+ *
+ * @dbgfs_dir: DebugFS directory to place the driver debug file
+ */
+struct idt_ntb_data {
+ struct pci_dev *pdev;
+ struct idt_89hpes_pdata *swcfg;
+
+ unsigned char port;
+ unsigned char part;
+
+ struct idt_ntb_topo topo;
+ enum ntb_topo role;
+
+ unsigned char peer_cnt;
+ struct idt_ntb_dev *ndevs;
+
+ void __iomem *cfg_mmio;
+
+ struct workqueue_struct *idt_wq;
+
+ struct delayed_work lnk_work;
+
+ phys_addr_t mw_base;
+ resource_size_t mw_size;
+ spinlock_t lut_lock;
+
+ u32 db_sts;
+ u32 db_msk;
+ spinlock_t db_lock;
+ struct tasklet_struct db_tasklet;
+
+ spinlock_t msg_lock;
+ struct kmem_cache *msg_cache;
+ struct tasklet_struct msg_tasklet;
+
+ struct dentry *dbgfs_dir;
+};
+#define to_dev_data(data) (&(data->pdev->dev))
+#define to_data_lnkwork(work) \
+ container_of(to_delayed_work(work), struct idt_ntb_data, lnk_work)
+
+/*
+ * Descriptor of the IDT PCIe-switch port specific parameters in the
+ * Global Configuration Space
+ * @pcicmd: PCI command register
+ * @pcielsts: PCIe link status
+ * @ntsdata: NT signal data
+ * @ntgsignal: NT global signal
+ *
+ * @ctl: Port control register
+ * @sts: Port status register
+ */
+struct idt_ntb_port {
+ enum idt_ntb_cfgreg pcicmd;
+ enum idt_ntb_cfgreg pcielsts;
+ enum idt_ntb_cfgreg ntsdata;
+ enum idt_ntb_cfgreg ntgsignal;
+
+ enum idt_ntb_cfgreg ctl;
+ enum idt_ntb_cfgreg sts;
+};
+
+/*
+ * Descriptor of the IDT PCIe-switch partition specific parameters.
+ * @ctl: Partition control register in the Global Address Space
+ * @sts: Partition status register in the Global Address Space
+ */
+struct idt_ntb_part {
+ enum idt_ntb_cfgreg ctl;
+ enum idt_ntb_cfgreg sts;
+ enum idt_ntb_cfgreg msgctl[IDT_NTB_MSG_CNT];
+};
+
+#endif /* NTB_HW_IDT_H */
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt_quirks.c b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
new file mode 100644
index 0000000..57c7ccf
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
@@ -0,0 +1,163 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*#define DEBUG*/
+
+#include <linux/kernel.h>
+#include <linux/moduleparam.h>
+#include <linux/pci.h>
+
+#include "ntb_hw_idt.h"
+#include "ntb_hw_idt_quirks.h"
+
+/*
+ * Module parameters:
+ * @mw_aprt: Memory Windows aperture (x86: 9 - 26, x64: 9 - 32)
+ */
+static unsigned char mw_aprt = DEFAULT_MW_APRT;
+module_param(mw_aprt, byte, 0000);
+MODULE_PARM_DESC(mw_aprt,
+ "IDT NTB memory windows aperture. The actual memory windows size is "
+ "limited with 2^mw_aprt. It is initially set to 20 so the upper "
+ "boundary of the memory windows size would be 1 MB."
+ "Both sides, local node and peer MUST set the same value!");
+
+/*
+ * Alter the passed driver paremeters
+ */
+static void idt_ntb_alter_params(struct pci_dev *pdev)
+{
+ unsigned char mw_aprt_bak = mw_aprt;
+
+ /* Clamp the memory windows aperture parameter */
+#ifdef CONFIG_64BIT
+ mw_aprt = clamp(mw_aprt, MIN_MW_APRT, MAX_X64_MW_APRT);
+#else
+ mw_aprt = clamp(mw_aprt, MIN_MW_APRT, MAX_X86_MW_APRT);
+#endif /* !CONFIG_64BIT */
+ if (mw_aprt_bak != mw_aprt) {
+ dev_warn(&pdev->dev,
+ "IDT NTB memory windows aperture has been clamped "
+ "from %hhu to %hhu", mw_aprt_bak, mw_aprt);
+ }
+
+ dev_dbg(&pdev->dev, "IDT NTB HW-driver parameter has been verified");
+}
+
+/*
+ * IDT PCIe-swtich NTB function BARs pre-initializer
+ */
+static void idt_ntb_quirks(struct pci_dev *pdev)
+{
+ int ret;
+ u32 lubar_aprt = 0, dirbar_aprt = 0;
+
+ /* Alter the memory windows aperture parameter first */
+ idt_ntb_alter_params(pdev);
+
+ /* Calculate memory windows related BAR aperture */
+ lubar_aprt = (mw_aprt + MWLUTBL_APRT) << MWAPRT_OFF;
+ dirbar_aprt = mw_aprt << MWAPRT_OFF;
+
+ /* Pre-initialize the maximum possible BAR's so don't worry about them
+ * anymore */
+ /* BAR0 - Memory mapped Configuration space - x32 Non-prefetchable
+ * memory mapped space. Since it is the registers space then it must be
+ * non-prefetchable, which permits the 32-bits address only according
+ * to the PCI specs. Even though PCIe bridges doesn't do any prefetching
+ * whether prefetch bit is set or not, We'll set the bit as a matter of
+ * legacy */
+ ret = pci_write_config_dword(pdev, BARSETUP0_OFF, BARSETUP_CFG_32BIT);
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to activate registers configuration space (BAR0)");
+ return;
+ }
+
+ /* BAR2(+ x64:3) - Memory mapped shared memory with address translation
+ * based on lookup table - x32/x64 Non-prefetchable/prefetchable memory
+ * mapped space with aperture of 2^(mw_aprt + MWLUTBL_APRT), which
+ * effectively gives 2^mw_aprt bytes of memory space per each memory
+ * window */
+#ifdef CONFIG_64BIT
+ ret = pci_write_config_dword(pdev, BARSETUP2_OFF,
+ BARSETUP_24LUMW_64BIT | lubar_aprt);
+
+#else
+ ret = pci_write_config_dword(pdev, BARSETUP2_OFF,
+ BARSETUP_24LUMW_32BIT | lubar_aprt);
+
+#endif /* !CONFIG_64BIT */
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to activate lookup table based memory window (BAR2)");
+ return;
+ }
+
+ /* BAR4(+ x64:5) - Memory mapped shared memory with direct address
+ * translation - x32/x64 Non-prefetchable/prefetchable memory
+ * mapped space with aperture of 2^(mw_aprt + MWLUTBL_APRT) */
+#ifdef CONFIG_64BIT
+ ret = pci_write_config_dword(pdev, BARSETUP4_OFF,
+ BARSETUP_DIRMW_64BIT | dirbar_aprt);
+
+#else
+ ret = pci_write_config_dword(pdev, BARSETUP4_OFF,
+ BARSETUP_DIRMW_32BIT | dirbar_aprt);
+
+#endif /* !CONFIG_64BIT */
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to activate directly mapped memory window (BAR4)");
+ return;
+ }
+
+ dev_dbg(&pdev->dev, "IDT NTB BAR's enabled");
+}
+IDT_NTB_PCI_FIXUP_EARLY(89HPES24NT6AG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT8AG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT8BG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES12NT12G2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES16NT16G2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES24NT24G2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT24AG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT24BG2, idt_ntb_quirks);
+
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt_quirks.h b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
new file mode 100644
index 0000000..e633b7c
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
@@ -0,0 +1,114 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+#ifndef NTB_HW_IDT_QUIRKS_H
+#define NTB_HW_IDT_QUIRKS_H
+
+#include <linux/pci.h>
+#include <linux/pci_ids.h>
+
+#include "ntb_hw_idt.h"
+
+/*
+ * Macro is used to create the struct pci_fixup that matches the supported
+ * IDT PCIe-switches
+ * @devname: Capitalized name of the particular device
+ * @hook: Fixup hook function name
+ */
+#define IDT_NTB_PCI_FIXUP_EARLY(devname, hook) \
+ DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_IDT, \
+ PCI_DEVICE_ID_IDT_##devname, PCI_CLASS_BRIDGE_OTHER, 8U, hook)
+
+/*
+ * IDT PCIe-switch NTB function BAR setup parameters:
+ * @BARSETUP{N}_OFF: BAR{N} setup register offset
+ * @BARSETUP_CFG_32BIT: 32-bits addressable non-prefetchable memory
+ * mapped registers configuration space
+ * @BARSETUP_CFG_64BIT: 64-bits addressable prefetchable memory
+ * mapped registers configuration space
+ * @BARSETUP_DIRMW_32BIT: 32-bits addresable non-prefetchable direct
+ * address translatable memory window
+ * @BARSETUP_DIRMW_64BIT: 64-bits addresable prefetchable direct
+ * address translatable memory window
+ * @BARSETUP_12LUMW_32BIT: 32-bits addresable non-prefetchable 12-entries
+ * lookup table memory window
+ * @BARSETUP_12LUMW_64BIT: 64-bits addresable prefetchable 12-entries
+ * lookup table memory window
+ * @BARSETUP_24LUMW_32BIT: 32-bits addresable non-prefetchable 24-entries
+ * lookup table memory window
+ * @BARSETUP_24LUMW_64BIT: 64-bits addresable prefetchable 24-entries
+ * lookup table memory window
+ *
+ */
+#define BARSETUP0_OFF 0x00470
+#define BARSETUP1_OFF 0x00480
+#define BARSETUP2_OFF 0x00490
+#define BARSETUP3_OFF 0x004A0
+#define BARSETUP4_OFF 0x004B0
+#define BARSETUP5_OFF 0x004C0
+#define BARSETUP_CFG_32BIT ((u32)0x800004C0U)
+#define BARSETUP_CFG_64BIT ((u32)0x800004CCU)
+#define BARSETUP_DIRMW_32BIT ((u32)0x80000000U)
+#define BARSETUP_DIRMW_64BIT ((u32)0x8000000CU)
+#define BARSETUP_12LUMW_32BIT ((u32)0x80000800U)
+#define BARSETUP_12LUMW_64BIT ((u32)0x8000080CU)
+#define BARSETUP_24LUMW_32BIT ((u32)0x80001000U)
+#define BARSETUP_24LUMW_64BIT ((u32)0x8000100CU)
+#define MWAPRT_OFF 4
+
+/*
+ * IDT PCIe-switch NTB function related parameters:
+ * @DEFAULT_MW_APRT: Default aperture of the memory windows (that is
+ * maximum size of the memory windows)
+ * @MIN_MW_APRT: Minimum possible aperture of the memory windows
+ * @MAX_X86_MW_APRT: Maximum aperture for x86 architecture
+ * @MAX_X64_MW_APRT: Maximum aperture for x64 architecture
+ * @MWLUTBL_APRT: Additional value to translate the per memory
+ * windows specific aperture to the aperture of
+ * the whole lookup table
+ */
+#define DEFAULT_MW_APRT (unsigned char)20
+#define MIN_MW_APRT (unsigned char)9
+#define MAX_X86_MW_APRT (unsigned char)26
+#define MAX_X64_MW_APRT (unsigned char)32
+#define MWLUTBL_APRT (unsigned char)5
+
+#endif /* NTB_HW_IDT_QUIRKS_H */
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt_regmap.h b/drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
new file mode 100644
index 0000000..fa9aaf2
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
@@ -0,0 +1,877 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+#ifndef NTB_HW_IDT_REGMAP_H
+#define NTB_HW_IDT_REGMAP_H
+
+#include <linux/compiler.h>
+#include <linux/spinlock.h>
+
+/*
+ * Helper macros to enumerate the the registers and fields tables
+ * identifications.
+ * It's used in conjunction with the IDT_NT_REGFLDS(), IDT_SW_REGFLDS(),
+ * IDT_NT_CFGREGS() and IDT_SW_CFGREGS() macroses
+ */
+#define PAIR_ID_ENUM(ID, reg, mask, offset) ID,
+
+/*
+ * Helper macros to pair the Field identification with the corresponding
+ * register, mask and the offset
+ * It's used in conjunction with the IDT_NT_REGFLDS() and
+ * IDT_SW_REGFLDS() macroses
+ */
+#define PAIR_FLDID_ACCESS(ID, reg, mask, offset, retreg, retmask, retoffset) \
+ case ID: \
+ retreg = reg; \
+ retmask = mask; \
+ retoffset = offset; \
+ break;
+
+/*
+ * Helper macros to pair the registers identifications with the corresponding
+ * offset and the size
+ * It's used in conjunction with the IDT_NT_CFGREGS() and
+ * IDT_SW_CFGREGS() macroses
+ */
+#define PAIR_REGID_ACCESS(ID, addr, size, desc, retaddr, retsize, retdesc) \
+ case ID: \
+ retaddr = addr; \
+ retsize = size; \
+ retdesc = desc; \
+ break;
+
+/*
+ * IDT PCIe-swtich registers related constants:
+ *
+ * @BARSTP_MEMMAP: Memory mapped BAR select
+ * @BARSTP_TYPE_32: 32-bits addressable BAR select
+ * @BARSTP_TYPE_64: 64-bits addressable BAR select
+ * @BARSTP_NONPREF: Non-prefetchable memory
+ * @BARSTP_PREF: Prefetchable memory
+ * @BARSTP_MINSIZE: Minimum BAR aperture (2^SIZE) for Lookup table
+ * @BARSTP_MAXSIZE_32: Maximum BAR aperture for Lookup table and x86 CPU
+ * @BARSTP_MAXSIZE_64: Maximum BAR aperture for Lookup table and x64 CPU
+ * @BARSTP_MODE_WNDW: Memory Window mode of BAR
+ * @BARSTP_MODE_CFGSPC: Configuration space mode of BAR
+ * @BARSTP_ATRAN_DRCT: Direct address translation
+ * @BARSTP_ATRAN_LU12: 12-entries Lookup table for address translation
+ * @BARSTP_ATRAN_LU24: 24-entries Lookup table for address translation
+ *
+ * @GASAADDR_OFFSET: GASAADDR register offset in the NT-function config space
+ * @GASADATA_OFFSET: GASADATA register offset in the NT-function config space
+ *
+ * @PORTMODE_NT: Port mode - just one NT function
+ * @PORTMODE_USNT: Port mode - upstream switch port with NT function
+ * @PORTMODE_USNTDMA: Port mode - upstream switch port with NT and DMA
+ * functions
+ *
+ * @NTINT_MASK: Mask the NT interrupts (Msg, DB, SE, Temp)
+ * @NTINT_UNMASK: Unmask the NT interrupts (Msg, DB, SE, Temp)
+ * @ALLINT_MASK: Mask all the available interrupts
+ * @ALLINT_UNMASK: Unmask all the available unterrupts
+ * @MSGINT_BIT: Message interrupt serial bit
+ * @DBINT_BIT: Doorbell interrupt serial bit
+ * @SEINT_BIT: Switch events interrupt serial bit
+ * @FMCI_BIT: Failover mode change initiated
+ * @FMCC_BIT: Failover mode change completed
+ * @TEMPINT_BIT: Temperature sensor interrupt serial bit
+ *
+ * @INDB_MASK: Mask the inbound doorbells interrupt
+ * @INDB_UNMASK: Unmask the inbound doorbells interrupt
+ * @OUTMSG_MASK: Mask all the outbound message bits
+ * @INMSG_MASK: Mask all the inbound message bits
+ * @INMSG_STS: Valid inbound message status
+ * @MSG_MASK: Mask all the message interrupts
+ * @MSG_UNMASK: Unmask the first inbound message register interrupt
+ */
+#define BARSTP_MEMMAP ((u32)0x0)
+#define BARSTP_TYPE_32 ((u32)0x0)
+#define BARSTP_TYPE_64 ((u32)0x2)
+#define BARSTP_NONPREF ((u32)0x0)
+#define BARSTP_PREF ((u32)0x1)
+#define BARSTP_MINSIZE ((u32)14)
+#define BARSTP_MAXSIZE_32 ((u32)16)
+#define BARSTP_MAXSIZE_64 ((u32)37)
+#define BARSTP_MODE_WNDW ((u32)0x0)
+#define BARSTP_MODE_CFGSPC ((u32)0x1)
+#define BARSTP_ATRAN_DRCT ((u32)0x0)
+#define BARSTP_ATRAN_LU12 ((u32)0x1)
+#define BARSTP_ATRAN_LU24 ((u32)0x2)
+#define GASAADDR_OFFSET ((ptrdiff_t)0x00FF8)
+#define GASADATA_OFFSET ((ptrdiff_t)0x00FFC)
+#define PORTMODE_NT ((u32)0x3)
+#define PORTMODE_USNT ((u32)0x4)
+#define PORTMODE_USNTDMA ((u32)0x7)
+#define NTINT_MASK ((u32)0x8B)
+#define NTINT_UNMASK (~NTINT_MASK)
+#define ALLINT_MASK ((u32)0xBB)
+#define ALLINT_UNMASK (~ALLINT_MASK)
+#define MSGINT_BIT ((u32)0)
+#define DBINT_BIT ((u32)1)
+#define SEINT_BIT ((u32)3)
+#define FMCI_BIT ((u32)4)
+#define FMCC_BIT ((u32)5)
+#define TEMPINT_BIT ((u32)7)
+#define INDB_MASK ((u32)-1)
+#define INDB_UNMASK ((u32)0x0)
+#define OUTMSG_MASK ((u32)0xF)
+#define INMSG_MASK ((u32)0xF0000)
+#define INMSG_STS ((u32)0xF)
+#define MSG_MASK ((u32)0xF000F)
+#define MSG_UNMASK ((u32)0xE000F)
+
+/*
+ * Table of the register fields accessed over either the NT-function
+ * memory mapped registers or IDT PCIe-switch Global registers.
+ * This table is then translated into the switch-case statement
+ * so to get the proper "Name"->{reg addr, mask, fld offset}
+ * pairs.
+ */
+#define IDT_NT_REGFLDS(X, args...) \
+ /* PCI command register */ \
+ X(IDT_NT_IOAE, IDT_NT_PCI_CMD, 0x1, 0, ## args) \
+ X(IDT_NT_MAE, IDT_NT_PCI_CMD, 0x1, 1, ## args) \
+ X(IDT_NT_BME, IDT_NT_PCI_CMD, 0x1, 2, ## args) \
+ /* Link status registers */ \
+ X(IDT_NT_MAXLNKSPD, IDT_NT_PCI_PCIELCAP, 0xF, 0, ## args) \
+ X(IDT_NT_MAXLNKWDTH, IDT_NT_PCI_PCIELCAP, 0x3F, 4, ## args) \
+ X(IDT_NT_PORTNUM, IDT_NT_PCI_PCIELCAP, 0xFF, 24, ## args) \
+ X(IDT_NT_CURLNKSPD, IDT_NT_PCI_PCIELSTS, 0xF, 0, ## args) \
+ X(IDT_NT_CURLNKWDTH, IDT_NT_PCI_PCIELSTS, 0x3F, 4, ## args) \
+ X(IDT_NT_SCLK, IDT_NT_PCI_PCIELSTS, 0x1, 12, ## args) \
+ /* SSVID/SSID registers */\
+ X(IDT_NT_SSVID, IDT_NT_PCI_SSIDSSVID, 0xFFFF, 0, ## args) \
+ X(IDT_NT_SSID, IDT_NT_PCI_SSIDSSVID, 0xFFFF, 16, ## args) \
+ /* General NT-function registers */\
+ X(IDT_NT_IDPROTDIS, IDT_NT_PCI_NTCTL, 0x1, 0, ## args) \
+ X(IDT_NT_CPEN, IDT_NT_PCI_NTCTL, 0x1, 1, ## args) \
+ /* NT interrupts related registers */ \
+ X(IDT_NT_INTSTS, IDT_NT_PCI_NTINTSTS, 0xBB, 0, ## args) \
+ X(IDT_NT_MSGINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 0, ## args) \
+ X(IDT_NT_DBINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 1, ## args) \
+ X(IDT_NT_SEINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 3, ## args) \
+ X(IDT_NT_FMCIINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 4, ## args) \
+ X(IDT_NT_FMCCINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 5, ## args) \
+ X(IDT_NT_TMPINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 7, ## args) \
+ X(IDT_NT_INTMSK, IDT_NT_PCI_NTINTMSK, 0xBB, 0, ## args) \
+ X(IDT_NT_MSGINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 0, ## args) \
+ X(IDT_NT_DBINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 1, ## args) \
+ X(IDT_NT_SEINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 3, ## args) \
+ X(IDT_NT_FMCIINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 4, ## args) \
+ X(IDT_NT_FMCCINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 5, ## args) \
+ X(IDT_NT_TMPINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 7, ## args) \
+ X(IDT_NT_GSIGNAL, IDT_NT_PCI_NTGSIGNAL, 0x1, 0, ## args) \
+ /* Message registers status and masks */ \
+ X(IDT_NT_OUTMSGSTS, IDT_NT_PCI_MSGSTS, 0xF, 0, ## args) \
+ X(IDT_NT_INMSGSTS, IDT_NT_PCI_MSGSTS, 0xF, 16, ## args) \
+ X(IDT_NT_OUTMSG0STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 0, ## args) \
+ X(IDT_NT_OUTMSG1STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 1, ## args) \
+ X(IDT_NT_OUTMSG2STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 2, ## args) \
+ X(IDT_NT_OUTMSG3STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 3, ## args) \
+ X(IDT_NT_INMSG0STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 16, ## args) \
+ X(IDT_NT_INMSG1STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 17, ## args) \
+ X(IDT_NT_INMSG2STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 18, ## args) \
+ X(IDT_NT_INMSG3STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 19, ## args) \
+ /* BARSETUPx register (default BARSETUP0) */ \
+ X(IDT_NT_BARSTP_MEMSI, IDT_NT_PCI_BARSETUP0, 0x1, 0, ## args) \
+ X(IDT_NT_BARSTP_TYPE, IDT_NT_PCI_BARSETUP0, 0x3, 1, ## args) \
+ X(IDT_NT_BARSTP_PREF, IDT_NT_PCI_BARSETUP0, 0x1, 3, ## args) \
+ X(IDT_NT_BARSTP_SIZE, IDT_NT_PCI_BARSETUP0, 0x3F, 4, ## args) \
+ X(IDT_NT_BARSTP_MODE, IDT_NT_PCI_BARSETUP0, 0x1, 10, ## args) \
+ X(IDT_NT_BARSTP_ATRAN, IDT_NT_PCI_BARSETUP0, 0x3, 11, ## args) \
+ X(IDT_NT_BARSTP_TPART, IDT_NT_PCI_BARSETUP0, 0x7, 13, ## args) \
+ X(IDT_NT_BARSTP_EN, IDT_NT_PCI_BARSETUP0, 0x1, 31, ## args) \
+ /* NT mapping table registers */ \
+ X(IDT_NT_MTBL_ADDR, IDT_NT_PCI_NTMTBLADDR, 0x7F, 0, ## args) \
+ X(IDT_NT_MTBL_ERR, IDT_NT_PCI_NTMTBLSTS, 0x1, 0, ## args) \
+ X(IDT_NT_MTBL_VALID, IDT_NT_PCI_NTMTBLDATA, 0x1, 0, ## args) \
+ X(IDT_NT_MTBL_BDF, IDT_NT_PCI_NTMTBLDATA, 0xFFFF, 1, ## args) \
+ X(IDT_NT_MTBL_PART, IDT_NT_PCI_NTMTBLDATA, 0x7, 17, ## args) \
+ X(IDT_NT_MTBL_ATP, IDT_NT_PCI_NTMTBLDATA, 0x1, 29, ## args) \
+ X(IDT_NT_MTBL_CNS, IDT_NT_PCI_NTMTBLDATA, 0x1, 30, ## args) \
+ X(IDT_NT_MTBL_RNS, IDT_NT_PCI_NTMTBLDATA, 0x1, 31, ## args) \
+ X(IDT_NT_MTBL_REQID, IDT_NT_PCI_REQIDCAP, 0xFFFF, 0, ## args) \
+ /* Lookup table registers */ \
+ X(IDT_NT_LUT_INDEX, IDT_NT_PCI_LUTOFFSET, 0x1F, 0, ## args) \
+ X(IDT_NT_LUT_BAR, IDT_NT_PCI_LUTOFFSET, 0x7, 8, ## args) \
+ X(IDT_NT_LUT_PART, IDT_NT_PCI_LUTUDATA, 0xF, 0, ## args) \
+ X(IDT_NT_LUT_VALID, IDT_NT_PCI_LUTUDATA, 0x1, 31, ## args)
+
+/*
+ * Table of the fields accessed over the global switch registers
+ * This table is then translated into the switch-case statement
+ * so to get the proper "Name"->{reg addr, mask, fld offset}
+ * pairs.
+ */
+#define IDT_SW_REGFLDS(X, args...) \
+ /* Boot configuration vector status */ \
+ X(IDT_SW_SWMODE, IDT_SW_PCI_BCVSTS, 0xF, 0, ## args) \
+ X(IDT_SW_GCLKFSEL, IDT_SW_PCI_BCVSTS, 0x1, 5, ## args) \
+ X(IDT_SW_SSMB_ADDRSET, IDT_SW_PCI_BCVSTS, 0x3, 7, ## args) \
+ X(IDT_SW_CLKMODE, IDT_SW_PCI_BCVSTS, 0x3, 14, ## args) \
+ /* Ports clocking mode */ \
+ X(IDT_SW_P0CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 0, ## args) \
+ X(IDT_SW_P2CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 2, ## args) \
+ X(IDT_SW_P4CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 4, ## args) \
+ X(IDT_SW_P6CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 6, ## args) \
+ X(IDT_SW_P8CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 8, ## args) \
+ X(IDT_SW_P12CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 10, ## args) \
+ X(IDT_SW_P16CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 12, ## args) \
+ X(IDT_SW_P20CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 14, ## args) \
+ /* Switch Ports Status register (default, Port 0) */ \
+ X(IDT_SW_PORT_LNKUP, IDT_SW_PCI_SWPORT0STS, 0x1, 4, ## args) \
+ X(IDT_SW_PORT_LNKMODE, IDT_SW_PCI_SWPORT0STS, 0x1, 5, ## args) \
+ X(IDT_SW_PORT_MODE, IDT_SW_PCI_SWPORT0STS, 0xF, 6, ## args) \
+ X(IDT_SW_PORT_SWPART, IDT_SW_PCI_SWPORT0STS, 0x7, 10, ## args) \
+ /* Switch Event registers */ \
+ X(IDT_SW_LNKUP_GSTS, IDT_SW_PCI_SESTS, 0x1, 0, ## args) \
+ X(IDT_SW_LNKDN_GSTS, IDT_SW_PCI_SESTS, 0x1, 1, ## args) \
+ X(IDT_SW_FRST_GSTS, IDT_SW_PCI_SESTS, 0x1, 2, ## args) \
+ X(IDT_SW_HRST_GSTS, IDT_SW_PCI_SESTS, 0x1, 3, ## args) \
+ X(IDT_SW_FOVER_GSTS, IDT_SW_PCI_SESTS, 0x1, 4, ## args) \
+ X(IDT_SW_GSIG_GSTS, IDT_SW_PCI_SESTS, 0x1, 5, ## args) \
+ X(IDT_SW_LNKUP_GMSK, IDT_SW_PCI_SEMSK, 0x1, 0, ## args) \
+ X(IDT_SW_LNKDN_GMSK, IDT_SW_PCI_SEMSK, 0x1, 1, ## args) \
+ X(IDT_SW_FRST_GMSK, IDT_SW_PCI_SEMSK, 0x1, 2, ## args) \
+ X(IDT_SW_HRST_GMSK, IDT_SW_PCI_SEMSK, 0x1, 3, ## args) \
+ X(IDT_SW_FOVER_GMSK, IDT_SW_PCI_SEMSK, 0x1, 4, ## args) \
+ X(IDT_SW_GSIG_GMSK, IDT_SW_PCI_SEMSK, 0x1, 5, ## args) \
+ X(IDT_SW_SEPART_GMSK, IDT_SW_PCI_SEPMSK, 0xFF, 0, ## args) \
+ X(IDT_SW_PORTLNKUP_STS, IDT_SW_PCI_SELINKUPSTS, 0xFFF, 0, ## args) \
+ X(IDT_SW_PORTLNKUP_MSK, IDT_SW_PCI_SELINKUPMSK, 0xFFF, 0, ## args) \
+ X(IDT_SW_PORTLNKDN_STS, IDT_SW_PCI_SELINKDNSTS, 0xFFF, 0, ## args) \
+ X(IDT_SW_PORTLNKDN_MSK, IDT_SW_PCI_SELINKDNMSK, 0xFFF, 0, ## args) \
+ X(IDT_SW_PARTFRST_STS, IDT_SW_PCI_SEFRSTSTS, 0xF, 0, ## args) \
+ X(IDT_SW_PARTFRST_MSK, IDT_SW_PCI_SEFRSTMSK, 0xF, 0, ## args) \
+ X(IDT_SW_PARTHRST_STS, IDT_SW_PCI_SEHRSTSTS, 0xF, 0, ## args) \
+ X(IDT_SW_PARTHRST_MSK, IDT_SW_PCI_SEHRSTMSK, 0xF, 0, ## args) \
+ X(IDT_SW_PARTGSIG_STS, IDT_SW_PCI_SEGSIGSTS, 0xF, 0, ## args) \
+ X(IDT_SW_PARTGSIG_MSK, IDT_SW_PCI_SEGSIGMSK, 0xF, 0, ## args) \
+ /* Global DoorBell registers (default, Doorbell 0) */ \
+ X(IDT_SW_PART_GODBELLMSK, IDT_SW_PCI_GODBELLMSK0, 0xF, 0, ## args) \
+ X(IDT_SW_PART_GIDBELLMSK, IDT_SW_PCI_GIDBELLMSK0, 0xF, 0, ## args) \
+ /* Message register (default, Partition 0 Message Control 0) */ \
+ X(IDT_SW_MSGROUTE_REG, IDT_SW_PCI_SWP0MSGCTL0, 0x3, 0, ## args) \
+ X(IDT_SW_MSGROUTE_PART, IDT_SW_PCI_SWP0MSGCTL0, 0x7, 4, ## args) \
+ /* SMBUS status */ \
+ X(IDT_SW_SSMBADDR, IDT_SW_PCI_SMBUSSTS, 0x7F, 1, ## args) \
+ X(IDT_SW_MSMBADDR, IDT_SW_PCI_SMBUSSTS, 0x7F, 9, ## args) \
+ /* Temperature sensor register */ \
+ X(IDT_SW_TMP_LTH, IDT_SW_PCI_TMPCTL, 0xFF, 0, ## args) \
+ X(IDT_SW_TMP_HTH, IDT_SW_PCI_TMPCTL, 0xFF, 16, ## args) \
+ X(IDT_SW_TMP_BLTH_EN, IDT_SW_PCI_TMPCTL, 0x1, 26, ## args) \
+ X(IDT_SW_TMP_AHTH_EN, IDT_SW_PCI_TMPCTL, 0x1, 29, ## args) \
+ X(IDT_SW_TMP_PDOWN, IDT_SW_PCI_TMPCTL, 0x1, 31, ## args) \
+ X(IDT_SW_TMP_CURTEMP, IDT_SW_PCI_TMPSTS, 0xFF, 0, ## args) \
+ X(IDT_SW_TMP_BLTH_STS, IDT_SW_PCI_TMPSTS, 0x1, 24, ## args) \
+ X(IDT_SW_TMP_AHTH_STS, IDT_SW_PCI_TMPSTS, 0x1, 29, ## args) \
+ X(IDT_SW_TMP_BLTH_CLR, IDT_SW_PCI_TMPALARM, 0x1, 24, ## args) \
+ X(IDT_SW_TMP_AHTH_CLR, IDT_SW_PCI_TMPALARM, 0x1, 29, ## args)
+
+/*
+ * Enumeration of the IDT PCIe-switch registers access fields
+ */
+enum idt_ntb_regfld {
+ IDT_NT_REGFLDS(PAIR_ID_ENUM)
+ IDT_NTB_REGFLDS_SPLIT,
+ IDT_SW_REGFLDS(PAIR_ID_ENUM)
+ IDT_NTB_REGFLDS_END
+};
+
+/*
+ * Enumeration of the possible registers size
+ */
+enum idt_ntb_regsize {
+ REGBYTE = 1,
+ REGWORD = 2,
+ REGDWORD = 4
+};
+
+/*
+ * Enumeration of the NT-function Configuration Space registers
+ * NOTE 1) The IDT PCIe-switch internal data is littel-endian
+ * so it must be taken into account in the driver
+ * internals.
+ * 2) Additionally the registers should be accessed either
+ * with byte-enables corresponding to their native size or
+ * the size of one DWORD
+ */
+#define IDT_NT_CFGREGS(X, args...) \
+ /* PCI Express Configuration Space */ \
+ /* Type 0 configuration header */ \
+ X(IDT_NT_PCI_VID, 0x00000, REGWORD, "Vendor Identification", ## args) \
+ X(IDT_NT_PCI_DID, 0x00002, REGWORD, "Device Identification", ## args) \
+ X(IDT_NT_PCI_CMD, 0x00004, REGWORD, "PCI Command", ## args) \
+ X(IDT_NT_PCI_STS, 0x00006, REGWORD, "PCI Status", ## args) \
+ X(IDT_NT_PCI_RID, 0x00008, REGBYTE, "Revision Identification", ## args) \
+ X(IDT_NT_PCI_PROGIF, 0x00009, REGBYTE, "Program Interface", ## args) \
+ X(IDT_NT_PCI_SCCLASS, 0x0000A, REGBYTE, "Sub Class Code", ## args) \
+ X(IDT_NT_PCI_CLASS, 0x0000B, REGBYTE, "Class Code", ## args) \
+ X(IDT_NT_PCI_CLS, 0x0000C, REGBYTE, "Cache Line Size", ## args) \
+ X(IDT_NT_PCI_LTIMER, 0x0000D, REGBYTE, "Latency Time", ## args) \
+ X(IDT_NT_PCI_HDR, 0x0000E, REGBYTE, "Header Type", ## args) \
+ X(IDT_NT_PCI_BIST, 0x0000F, REGBYTE, "Built-in Self Test Register", ## args) \
+ X(IDT_NT_PCI_BAR0, 0x00010, REGDWORD, "Base Address Register 0", ## args) \
+ X(IDT_NT_PCI_BAR1, 0x00014, REGDWORD, "Base Address Register 1", ## args) \
+ X(IDT_NT_PCI_BAR2, 0x00018, REGDWORD, "Base Address Register 2", ## args) \
+ X(IDT_NT_PCI_BAR3, 0x0001C, REGDWORD, "Base Address Register 3", ## args) \
+ X(IDT_NT_PCI_BAR4, 0x00020, REGDWORD, "Base Address Register 4", ## args) \
+ X(IDT_NT_PCI_BAR5, 0x00024, REGDWORD, "Base Address Register 5", ## args) \
+ X(IDT_NT_PCI_CCISPTR, 0x00028, REGDWORD, "CardBus CIS Pointer", ## args) \
+ X(IDT_NT_PCI_SUBVID, 0x0002C, REGWORD, "Subsystem Vendor ID Pointer", ## args) \
+ X(IDT_NT_PCI_SUBID, 0x0002E, REGWORD, "Subsystem ID Pointer", ## args) \
+ X(IDT_NT_PCI_EROMBASE, 0x00030, REGWORD, "Expansion ROM Base", ## args) \
+ X(IDT_NT_PCI_CAPPTR, 0x00034, REGBYTE, "Capabilities Pointer", ## args) \
+ X(IDT_NT_PCI_INTRLINE, 0x0003C, REGBYTE, "Interrupt Line", ## args) \
+ X(IDT_NT_PCI_INTRPIN, 0x0003D, REGBYTE, "Interrupt PIN", ## args) \
+ X(IDT_NT_PCI_MINGNT, 0x0003E, REGBYTE, "Minimum Grant", ## args) \
+ X(IDT_NT_PCI_MAXLAT, 0x0003F, REGBYTE, "Maximum Latency", ## args) \
+ /* PCI Express capablity structure */ \
+ X(IDT_NT_PCI_PCIECAP, 0x00040, REGDWORD, "PCI Express Capability", ## args) \
+ X(IDT_NT_PCI_PCIEDCAP, 0x00044, REGDWORD, "PCI Express Device Capabilities", ## args) \
+ X(IDT_NT_PCI_PCIEDCTL, 0x00048, REGWORD, "PCI Express Device Control", ## args) \
+ X(IDT_NT_PCI_PCIEDSTS, 0x0004A, REGWORD, "PCI Express Device Status", ## args) \
+ X(IDT_NT_PCI_PCIELCAP, 0x0004C, REGDWORD, "PCI Express Link Capabilities", ## args) \
+ X(IDT_NT_PCI_PCIELCTL, 0x00050, REGWORD, "PCI Express Link Control", ## args) \
+ X(IDT_NT_PCI_PCIELSTS, 0x00052, REGWORD, "PCI Express Link Status", ## args) \
+ X(IDT_NT_PCI_PCIEDCAP2, 0x00064, REGDWORD, "PCI Express Device Capabilities 2", ## args) \
+ X(IDT_NT_PCI_PCIEDCTL2, 0x00068, REGWORD, "PCI Express Device Control 2", ## args) \
+ X(IDT_NT_PCI_PCIEDSTS2, 0x0006A, REGWORD, "PCI Express Device Status 2", ## args) \
+ X(IDT_NT_PCI_PCIELCAP2, 0x0006C, REGDWORD, "PCI Express Link Capabilities 2", ## args) \
+ X(IDT_NT_PCI_PCIELCTL2, 0x00070, REGWORD, "PCI Express Link Control 2", ## args) \
+ X(IDT_NT_PCI_PCIELSTS2, 0x00072, REGWORD, "PCI Express Link Status 2", ## args) \
+ /* PCI Power Management capability structure */ \
+ X(IDT_NT_PCI_PMCAP, 0x000C0, REGDWORD, "PCI Power Management Capabilities", ## args) \
+ X(IDT_NT_PCI_PMCSR, 0x000C4, REGDWORD, "PCI Power Management Control and Status", ## args) \
+ /* MSI Capability structure */ \
+ X(IDT_NT_PCI_MSICAP, 0x000D0, REGDWORD, "Message Signaled Interrupt Capability and Control", ## args) \
+ X(IDT_NT_PCI_MSIADDR, 0x000D4, REGDWORD, "Message Signaled Interrupt Address", ## args) \
+ X(IDT_NT_PCI_MSIUADDR, 0x000D8, REGDWORD, "Message Signaled Interrupt Upper Address", ## args) \
+ X(IDT_NT_PCI_MSIMDATA, 0x000DC, REGDWORD, "Message Signaled Interrupt Message Data", ## args) \
+ /* SSID/SSVID capability structure */ \
+ X(IDT_NT_PCI_SSIDSSVIDCAP, 0x000F0, REGDWORD, "Subsystem ID and Subsystem Vendor ID Capability", ## args) \
+ X(IDT_NT_PCI_SSIDSSVID, 0x000F4, REGDWORD, "Subsystem ID and Subsystem Vendor ID", ## args) \
+ /* Extended access registers */ \
+ X(IDT_NT_PCI_ECFGADDR, 0x000F8, REGDWORD, "Extended Configuration Space Access Address", ## args) \
+ X(IDT_NT_PCI_ECFGDATA, 0x000FC, REGDWORD, "Extended Configuration Space Access Data", ## args) \
+ /*==============64 REGDWORDs ================*/ \
+ /* PCI Express Extended Configuration Space */ \
+ /* Advanced Error Reporting enhanced capability */ \
+ X(IDT_NT_PCI_AERCAP, 0x00100, REGDWORD, "AER Capabilities ", ## args) \
+ X(IDT_NT_PCI_AERUES, 0x00104, REGDWORD, "AER Uncorrectable Error Status", ## args) \
+ X(IDT_NT_PCI_AERUEM, 0x00108, REGDWORD, "AER Uncorrectable Error Mask ", ## args) \
+ X(IDT_NT_PCI_AERUESV, 0x0010C, REGDWORD, "AER Uncorrectable Error Severity ", ## args) \
+ X(IDT_NT_PCI_AERCES, 0x00110, REGDWORD, "AER Correctable Error Status ", ## args) \
+ X(IDT_NT_PCI_AERCEM, 0x00114, REGDWORD, "AER Correctable Error Mask", ## args) \
+ X(IDT_NT_PCI_AERCTL, 0x00118, REGDWORD, "AER Control", ## args) \
+ X(IDT_NT_PCI_AERHL1DW, 0x0011C, REGDWORD, "AER Header Log 1st Doubleword", ## args) \
+ X(IDT_NT_PCI_AERHL2DW, 0x00120, REGDWORD, "AER Header Log 2nd Doubleword", ## args) \
+ X(IDT_NT_PCI_AERHL3DW, 0x00124, REGDWORD, "AER Header Log 3rd Doubleword", ## args) \
+ X(IDT_NT_PCI_AERHL4DW, 0x00128, REGDWORD, "AER Header Log 4th Doubleword", ## args) \
+ /* Device Serial Number enhanced capability */ \
+ X(IDT_NT_PCI_SNUMCAP, 0x00180, REGDWORD, "Serial Number Capabilities", ## args) \
+ X(IDT_NT_PCI_SNUMLDW, 0x00184, REGDWORD, "Serial Number Lower Doubleword", ## args) \
+ X(IDT_NT_PCI_SNUMUDW, 0x00188, REGDWORD, "Serial Number Upper Doubleword", ## args) \
+ /* PCIe Virtual Channel enhanced capability */ \
+ X(IDT_NT_PCI_PCIEVCECAP, 0x00200, REGDWORD, "PCI Express VC Extended Capability Header", ## args) \
+ X(IDT_NT_PCI_PVCCAP1, 0x00204, REGDWORD, "Port VC Capability 1", ## args) \
+ X(IDT_NT_PCI_PVCCAP2, 0x00208, REGDWORD, "Port VC Capability 2", ## args) \
+ X(IDT_NT_PCI_PVCCTL, 0x0020C, REGDWORD, "Port VC Control", ## args) \
+ X(IDT_NT_PCI_PVCSTS, 0x0020E, REGDWORD, "Port VC Status ", ## args) \
+ X(IDT_NT_PCI_VCR0CAP, 0x00210, REGDWORD, "VC Resource 0 Capability", ## args) \
+ X(IDT_NT_PCI_VCR0CTL, 0x00214, REGDWORD, "VC Resource 0 Control", ## args) \
+ X(IDT_NT_PCI_VCR0STS, 0x00218, REGDWORD, "VC Resource 0 Status", ## args) \
+ /* ACS enhanced capability */ \
+ X(IDT_NT_PCI_ACSECAPH, 0x00320, REGDWORD, "ACS Extended Capability Header", ## args) \
+ X(IDT_NT_PCI_ACSCAP, 0x00324, REGWORD, "ACS Capability", ## args) \
+ X(IDT_NT_PCI_ACSCTL, 0x00326, REGWORD, "ACS Control", ## args) \
+ X(IDT_NT_PCI_MCCAPH, 0x00330, REGDWORD, "Multicast Extended Capability Header", ## args) \
+ X(IDT_NT_PCI_MCCAP, 0x00334, REGWORD, "Multicast Capability", ## args) \
+ X(IDT_NT_PCI_MCCTL, 0x00336, REGWORD, "Multicast Control", ## args) \
+ X(IDT_NT_PCI_MCBARL, 0x00338, REGDWORD, "Multicast Base Address Low", ## args) \
+ X(IDT_NT_PCI_MCBARH, 0x0033C, REGDWORD, "Multicast Base Address High", ## args) \
+ X(IDT_NT_PCI_MCRCVL, 0x00340, REGDWORD, "Multicast Receive Low", ## args) \
+ X(IDT_NT_PCI_MCRCVH, 0x00344, REGDWORD, "Multicast Receive High", ## args) \
+ X(IDT_NT_PCI_MCBLKALLL, 0x00348, REGDWORD, "Multicast Block All Low", ## args) \
+ X(IDT_NT_PCI_MCBLKALLH, 0x0034C, REGDWORD, "Multicast Block All High", ## args) \
+ X(IDT_NT_PCI_MCBLKUTL, 0x00350, REGDWORD, "Multicast Block Untranslated Low", ## args) \
+ X(IDT_NT_PCI_MCBLKUTH, 0x00354, REGDWORD, "Multicast Block Untranslated High", ## args) \
+ /*==========================================*/ \
+ /* IDT Proprietary NT-port-specific registers */ \
+ /* NT-function main control registers */ \
+ X(IDT_NT_PCI_NTCTL, 0x00400, REGDWORD, "NT Endpoint Control", ## args) \
+ X(IDT_NT_PCI_NTINTSTS, 0x00404, REGDWORD, "NT Endpoint Interrupt Status", ## args) \
+ X(IDT_NT_PCI_NTINTMSK, 0x00408, REGDWORD, "NT Endpoint Interrupt Mask", ## args) \
+ X(IDT_NT_PCI_NTSDATA, 0x0040C, REGDWORD, "NT Endpoint Signal Data", ## args) \
+ X(IDT_NT_PCI_NTGSIGNAL, 0x00410, REGDWORD, "NT Endpoint Global Signal", ## args) \
+ X(IDT_NT_PCI_NTIERRORMSK0, 0x00414, REGDWORD, "Internal Error Reporting Mask 0", ## args) \
+ X(IDT_NT_PCI_NTIERRORMSK1, 0x00418, REGDWORD, "Internal Error Reporting Mask 1", ## args) \
+ /* Doorbel registers */ \
+ X(IDT_NT_PCI_OUTDBELLSET, 0x00420, REGDWORD, "NT Outbound Doorbell Set", ## args) \
+ X(IDT_NT_PCI_INDBELLSTS, 0x00428, REGDWORD, "NT Inbound Doorbell Status", ## args) \
+ X(IDT_NT_PCI_INDBELLMSK, 0x0042C, REGDWORD, "NT Inbound Doorbell Mask", ## args) \
+ /* Message registers */ \
+ X(IDT_NT_PCI_OUTMSG0, 0x00430, REGDWORD, "Outbound Message 0", ## args) \
+ X(IDT_NT_PCI_OUTMSG1, 0x00434, REGDWORD, "Outbound Message 1", ## args) \
+ X(IDT_NT_PCI_OUTMSG2, 0x00438, REGDWORD, "Outbound Message 2", ## args) \
+ X(IDT_NT_PCI_OUTMSG3, 0x0043C, REGDWORD, "Outbound Message 3", ## args) \
+ X(IDT_NT_PCI_INMSG0, 0x00440, REGDWORD, "Inbound Message 0", ## args) \
+ X(IDT_NT_PCI_INMSG1, 0x00444, REGDWORD, "Inbound Message 1", ## args) \
+ X(IDT_NT_PCI_INMSG2, 0x00448, REGDWORD, "Inbound Message 2", ## args) \
+ X(IDT_NT_PCI_INMSG3, 0x0044C, REGDWORD, "Inbound Message 3", ## args) \
+ X(IDT_NT_PCI_INMSGSRC0, 0x00450, REGDWORD, "Inbound Message Source 0", ## args) \
+ X(IDT_NT_PCI_INMSGSRC1, 0x00454, REGDWORD, "Inbound Message Source 1", ## args) \
+ X(IDT_NT_PCI_INMSGSRC2, 0x00458, REGDWORD, "Inbound Message Source 2", ## args) \
+ X(IDT_NT_PCI_INMSGSRC3, 0x0045C, REGDWORD, "Inbound Message Source 3", ## args) \
+ X(IDT_NT_PCI_MSGSTS, 0x00460, REGDWORD, "Message Status", ## args) \
+ X(IDT_NT_PCI_MSGSTSMSK, 0x00464, REGDWORD, "Message Status Mask", ## args) \
+ /* BAR-setup registers */ \
+ X(IDT_NT_PCI_BARSETUP0, 0x00470, REGDWORD, "BAR 0 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT0, 0x00474, REGDWORD, "BAR 0 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE0, 0x00478, REGDWORD, "BAR 0 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE0, 0x0047C, REGDWORD, "BAR 0 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP1, 0x00480, REGDWORD, "BAR 1 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT1, 0x00484, REGDWORD, "BAR 1 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE1, 0x00488, REGDWORD, "BAR 1 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE1, 0x0048C, REGDWORD, "BAR 1 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP2, 0x00490, REGDWORD, "BAR 2 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT2, 0x00494, REGDWORD, "BAR 2 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE2, 0x00498, REGDWORD, "BAR 2 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE2, 0x0049C, REGDWORD, "BAR 2 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP3, 0x004A0, REGDWORD, "BAR 3 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT3, 0x004A4, REGDWORD, "BAR 3 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE3, 0x004A8, REGDWORD, "BAR 3 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE3, 0x004AC, REGDWORD, "BAR 3 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP4, 0x004B0, REGDWORD, "BAR 4 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT4, 0x004B4, REGDWORD, "BAR 4 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE4, 0x004B8, REGDWORD, "BAR 4 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE4, 0x004BC, REGDWORD, "BAR 4 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP5, 0x004C0, REGDWORD, "BAR 5 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT5, 0x004C4, REGDWORD, "BAR 5 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE5, 0x004C8, REGDWORD, "BAR 5 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE5, 0x004CC, REGDWORD, "BAR 5 Upper Translated Base Address", ## args) \
+ /* NT mapping table registers */ \
+ X(IDT_NT_PCI_NTMTBLADDR, 0x004D0, REGDWORD, "NT Mapping Table Address", ## args) \
+ X(IDT_NT_PCI_NTMTBLSTS, 0x004D4, REGDWORD, "NT Mapping Table Status", ## args) \
+ X(IDT_NT_PCI_NTMTBLDATA, 0x004D8, REGDWORD, "NT Mapping Table Data", ## args) \
+ X(IDT_NT_PCI_REQIDCAP, 0x004DC, REGDWORD, "Requester ID Capture", ## args) \
+ /* Memory Windows Lookup table registers */ \
+ X(IDT_NT_PCI_LUTOFFSET, 0x004E0, REGDWORD, "Lookup Table Offset", ## args) \
+ X(IDT_NT_PCI_LUTLDATA, 0x004E4, REGDWORD, "Lookup Table Lower Data", ## args) \
+ X(IDT_NT_PCI_LUTMDATA, 0x004E8, REGDWORD, "Lookup Table Middle Data", ## args) \
+ X(IDT_NT_PCI_LUTUDATA, 0x004EC, REGDWORD, "Lookup Table Upper Data", ## args) \
+ /* NT Endpoint Errors Emulation registers */ \
+ X(IDT_NT_PCI_NTUEEM, 0x004F0, REGDWORD, "NT Endpoint Uncorrectable Error Emulation", ## args) \
+ X(IDT_NT_PCI_NTCEEM, 0x004F4, REGDWORD, "NT Endpoint Correctable Error Emulation", ## args) \
+ /* Punch-through registers */ \
+ X(IDT_NT_PCI_PTCCTL0, 0x00510, REGDWORD, "Punch-Through Configuration Control 0", ## args) \
+ X(IDT_NT_PCI_PTCCTL1, 0x00514, REGDWORD, "Punch-Through Configuration Control 1", ## args) \
+ X(IDT_NT_PCI_PTCDATA, 0x00518, REGDWORD, "Punch-Through Data", ## args) \
+ X(IDT_NT_PCI_PTCSTS, 0x0051C, REGDWORD, "Punch-Through Status", ## args) \
+ /* NT Multicast Group x Port association */ \
+ X(IDT_NT_PCI_NTMCG0PA, 0x00600, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ X(IDT_NT_PCI_NTMCG1PA, 0x00604, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ X(IDT_NT_PCI_NTMCG2PA, 0x00608, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ X(IDT_NT_PCI_NTMCG3PA, 0x0060C, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ /* Global Address Space Access registers */ \
+ /*X(IDT_NT_PCI_GASAADDR, 0x00FF8, REGDWORD, "Global Address Space Access Address", ## args) \
+ *X(IDT_NT_PCI_GASADATA, 0x00FFC, REGDWORD, "Global Address Space Access Data", ## args)*/
+
+/*
+ * Table of the IDT PCIe-switch Global Configuration and Status
+ * registers, corresponding size and the string name
+ */
+#define IDT_SW_CFGREGS(X, args...) \
+ /* Basic NT-function globally accessed registers */ \
+ /* Port 0 */ \
+ X(IDT_SW_PCI_NTP0_CMD, 0x01004, REGWORD, "Port 0 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP0_PCIELSTS, 0x01052, REGWORD, "Port 0 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP0_NTSDATA, 0x0140C, REGDWORD, "Port 0 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP0_NTGSIGNAL, 0x01410, REGDWORD, "Port 0 NT Global Signal", ## args) \
+ /* Port 2 */ \
+ X(IDT_SW_PCI_NTP2_CMD, 0x05004, REGWORD, "Port 2 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP2_PCIELSTS, 0x05052, REGWORD, "Port 2 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP2_NTSDATA, 0x0540C, REGDWORD, "Port 2 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP2_NTGSIGNAL, 0x05410, REGDWORD, "Port 2 NT Global Signal", ## args) \
+ /* Port 4 */ \
+ X(IDT_SW_PCI_NTP4_CMD, 0x09004, REGWORD, "Port 4 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP4_PCIELSTS, 0x09052, REGWORD, "Port 4 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP4_NTSDATA, 0x0940C, REGDWORD, "Port 4 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP4_NTGSIGNAL, 0x09410, REGDWORD, "Port 4 NT Global Signal", ## args) \
+ /* Port 6 */ \
+ X(IDT_SW_PCI_NTP6_CMD, 0x0D004, REGWORD, "Port 6 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP6_PCIELSTS, 0x0D052, REGWORD, "Port 6 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP6_NTSDATA, 0x0D40C, REGDWORD, "Port 6 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP6_NTGSIGNAL, 0x0D410, REGDWORD, "Port 6 NT Global Signal", ## args) \
+ /* Port 8 */ \
+ X(IDT_SW_PCI_NTP8_CMD, 0x11004, REGWORD, "Port 8 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP8_PCIELSTS, 0x11052, REGWORD, "Port 8 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP8_NTSDATA, 0x1140C, REGDWORD, "Port 8 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP8_NTGSIGNAL, 0x11410, REGDWORD, "Port 8 NT Global Signal", ## args) \
+ /* Port 12 */ \
+ X(IDT_SW_PCI_NTP12_CMD, 0x19004, REGWORD, "Port 12 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP12_PCIELSTS, 0x19052, REGWORD, "Port 12 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP12_NTSDATA, 0x1940C, REGDWORD, "Port 12 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP12_NTGSIGNAL, 0x19410, REGDWORD, "Port 12 NT Global Signal", ## args) \
+ /* Port 16 */ \
+ X(IDT_SW_PCI_NTP16_CMD, 0x21004, REGWORD, "Port 16 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP16_PCIELSTS, 0x21052, REGWORD, "Port 16 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP16_NTSDATA, 0x2140C, REGDWORD, "Port 16 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP16_NTGSIGNAL, 0x21410, REGDWORD, "Port 16 NT Global Signal", ## args) \
+ /* Port 20 */ \
+ X(IDT_SW_PCI_NTP20_CMD, 0x29004, REGWORD, "Port 20 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP20_PCIELSTS, 0x29052, REGWORD, "Port 20 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP20_NTSDATA, 0x2940C, REGDWORD, "Port 20 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP20_NTGSIGNAL, 0x29410, REGDWORD, "Port 20 NT Global Signal", ## args) \
+ /* IDT PCIe-switch control registers */ \
+ X(IDT_SW_PCI_SWCTL, 0x3E000, REGDWORD, "Switch Control", ## args) \
+ X(IDT_SW_PCI_BCVSTS, 0x3E004, REGDWORD, "Boot Configuration Vector Status", ## args) \
+ X(IDT_SW_PCI_PCLKMODE, 0x3E008, REGDWORD, "Port Clocking Mode", ## args) \
+ X(IDT_SW_PCI_STK0CFG, 0x3E010, REGDWORD, "Stack 0 Configuration", ## args) \
+ X(IDT_SW_PCI_STK1CFG, 0x3E014, REGDWORD, "Stack 1 Configuration", ## args) \
+ X(IDT_SW_PCI_STK2CFG, 0x3E018, REGDWORD, "Stack 2 Configuration", ## args) \
+ X(IDT_SW_PCI_STK3CFG, 0x3E01C, REGDWORD, "Stack 3 Configuration", ## args) \
+ /* Switch initialization delays */ \
+ X(IDT_SW_PCI_RDRAINDELAY, 0x3E080, REGDWORD, "Reset Drain Delay ", ## args) \
+ X(IDT_SW_PCI_POMCDELAY, 0x3E084, REGDWORD, "Port Operating Mode Change Drain Delay", ## args) \
+ X(IDT_SW_PCI_SEDELAY, 0x3E088, REGDWORD, "Side Effect Delay", ## args) \
+ X(IDT_SW_PCI_USSBRDELAY, 0x3E08C, REGDWORD, "Upstream Secondary Bus Reset Delay", ## args) \
+ /* Switch Partitions control and status registers */ \
+ X(IDT_SW_PCI_SWPART0CTL, 0x3E100, REGDWORD, "Switch Partition 0 Control", ## args) \
+ X(IDT_SW_PCI_SWPART0STS, 0x3E104, REGDWORD, "Switch Partition 0 Status", ## args) \
+ X(IDT_SW_PCI_SWPART0FCTL, 0x3E108, REGDWORD, "Switch Partition 0 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART1CTL, 0x3E120, REGDWORD, "Switch Partition 1 Control", ## args) \
+ X(IDT_SW_PCI_SWPART1STS, 0x3E124, REGDWORD, "Switch Partition 1 Status", ## args) \
+ X(IDT_SW_PCI_SWPART1FCTL, 0x3E128, REGDWORD, "Switch Partition 1 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART2CTL, 0x3E140, REGDWORD, "Switch Partition 2 Control", ## args) \
+ X(IDT_SW_PCI_SWPART2STS, 0x3E144, REGDWORD, "Switch Partition 2 Status", ## args) \
+ X(IDT_SW_PCI_SWPART2FCTL, 0x3E148, REGDWORD, "Switch Partition 2 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART3CTL, 0x3E160, REGDWORD, "Switch Partition 3 Control", ## args) \
+ X(IDT_SW_PCI_SWPART3STS, 0x3E164, REGDWORD, "Switch Partition 3 Status", ## args) \
+ X(IDT_SW_PCI_SWPART3FCTL, 0x3E168, REGDWORD, "Switch Partition 3 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART4CTL, 0x3E180, REGDWORD, "Switch Partition 4 Control", ## args) \
+ X(IDT_SW_PCI_SWPART4STS, 0x3E184, REGDWORD, "Switch Partition 4 Status", ## args) \
+ X(IDT_SW_PCI_SWPART4FCTL, 0x3E188, REGDWORD, "Switch Partition 4 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART5CTL, 0x3E1A0, REGDWORD, "Switch Partition 5 Control", ## args) \
+ X(IDT_SW_PCI_SWPART5STS, 0x3E1A4, REGDWORD, "Switch Partition 5 Status", ## args) \
+ X(IDT_SW_PCI_SWPART5FCTL, 0x3E1A8, REGDWORD, "Switch Partition 5 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART6CTL, 0x3E1C0, REGDWORD, "Switch Partition 6 Control", ## args) \
+ X(IDT_SW_PCI_SWPART6STS, 0x3E1C4, REGDWORD, "Switch Partition 6 Status", ## args) \
+ X(IDT_SW_PCI_SWPART6FCTL, 0x3E1C8, REGDWORD, "Switch Partition 6 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART7CTL, 0x3E1E0, REGDWORD, "Switch Partition 7 Control", ## args) \
+ X(IDT_SW_PCI_SWPART7STS, 0x3E1E4, REGDWORD, "Switch Partition 7 Status", ## args) \
+ X(IDT_SW_PCI_SWPART7FCTL, 0x3E1E8, REGDWORD, "Switch Partition 7 Failover Control", ## args) \
+ /* Switch Ports control and status registers */ \
+ X(IDT_SW_PCI_SWPORT0CTL, 0x3E200, REGDWORD, "Switch Port 0 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT0STS, 0x3E204, REGDWORD, "Switch Port 0 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT0FCTL, 0x3E208, REGDWORD, "Switch Port 0 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT2CTL, 0x3E240, REGDWORD, "Switch Port 2 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT2STS, 0x3E244, REGDWORD, "Switch Port 2 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT2FCTL, 0x3E248, REGDWORD, "Switch Port 2 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT4CTL, 0x3E280, REGDWORD, "Switch Port 4 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT4STS, 0x3E284, REGDWORD, "Switch Port 4 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT4FCTL, 0x3E288, REGDWORD, "Switch Port 4 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT6CTL, 0x3E2C0, REGDWORD, "Switch Port 6 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT6STS, 0x3E2C4, REGDWORD, "Switch Port 6 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT6FCTL, 0x3E2C8, REGDWORD, "Switch Port 6 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT8CTL, 0x3E300, REGDWORD, "Switch Port 8 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT8STS, 0x3E304, REGDWORD, "Switch Port 8 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT8FCTL, 0x3E308, REGDWORD, "Switch Port 8 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT12CTL, 0x3E380, REGDWORD, "Switch Port 12 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT12STS, 0x3E384, REGDWORD, "Switch Port 12 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT12FCTL, 0x3E388, REGDWORD, "Switch Port 12 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT16CTL, 0x3E400, REGDWORD, "Switch Port 16 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT16STS, 0x3E404, REGDWORD, "Switch Port 16 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT16FCTL, 0x3E408, REGDWORD, "Switch Port 16 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT20CTL, 0x3E480, REGDWORD, "Switch Port 20 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT20STS, 0x3E484, REGDWORD, "Switch Port 20 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT20FCTL, 0x3E488, REGDWORD, "Switch Port 20 Failover Control", ## args) \
+ /* Failover capability control and status registers */ \
+ X(IDT_SW_PCI_FCAP0CTL, 0x3E500, REGDWORD, "Failover Capability 0 Control", ## args) \
+ X(IDT_SW_PCI_FCAP0STS, 0x3E504, REGDWORD, "Failover Capability 0 Status", ## args) \
+ X(IDT_SW_PCI_FCAP0TIMER, 0x3E508, REGDWORD, "Failover Capability 0 Watchdog Timer", ## args) \
+ X(IDT_SW_PCI_FCAP1CTL, 0x3E520, REGDWORD, "Failover Capability 1 Control", ## args) \
+ X(IDT_SW_PCI_FCAP1STS, 0x3E524, REGDWORD, "Failover Capability 1 Status", ## args) \
+ X(IDT_SW_PCI_FCAP1TIMER, 0x3E528, REGDWORD, "Failover Capability 1 Watchdog Timer", ## args) \
+ X(IDT_SW_PCI_FCAP2CTL, 0x3E540, REGDWORD, "Failover Capability 2 Control", ## args) \
+ X(IDT_SW_PCI_FCAP2STS, 0x3E544, REGDWORD, "Failover Capability 2 Status", ## args) \
+ X(IDT_SW_PCI_FCAP2TIMER, 0x3E548, REGDWORD, "Failover Capability 2 Watchdog Timer", ## args) \
+ X(IDT_SW_PCI_FCAP3CTL, 0x3E560, REGDWORD, "Failover Capability 3 Control", ## args) \
+ X(IDT_SW_PCI_FCAP3STS, 0x3E564, REGDWORD, "Failover Capability 3 Status", ## args) \
+ X(IDT_SW_PCI_FCAP3TIMER, 0x3E568, REGDWORD, "Failover Capability 3 Watchdog Timer", ## args) \
+ /* Protection registers */ \
+ X(IDT_SW_PCI_GASAPROT, 0x3E700, REGDWORD, "Global Address Space Access Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT0, 0x3E710, REGDWORD, "Partition 0 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT1, 0x3E714, REGDWORD, "Partition 1 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT2, 0x3E718, REGDWORD, "Partition 2 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT3, 0x3E71C, REGDWORD, "Partition 3 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT4, 0x3E720, REGDWORD, "Partition 4 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT5, 0x3E724, REGDWORD, "Partition 5 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT6, 0x3E728, REGDWORD, "Partition 6 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT7, 0x3E72C, REGDWORD, "Partition 7 NT Mapping Table Protection", ## args) \
+ /* Switch Event registers */ \
+ X(IDT_SW_PCI_SESTS, 0x3EC00, REGDWORD, "Switch Event Status", ## args) \
+ X(IDT_SW_PCI_SEMSK, 0x3EC04, REGDWORD, "Switch Event Mask", ## args) \
+ X(IDT_SW_PCI_SEPMSK, 0x3EC08, REGDWORD, "Switch Event Partition Mask", ## args) \
+ X(IDT_SW_PCI_SELINKUPSTS, 0x3EC0C, REGDWORD, "Switch Event Link Up Status", ## args) \
+ X(IDT_SW_PCI_SELINKUPMSK, 0x3EC10, REGDWORD, "Switch Event Link Up Mask", ## args) \
+ X(IDT_SW_PCI_SELINKDNSTS, 0x3EC14, REGDWORD, "Switch Event Link Down Status", ## args) \
+ X(IDT_SW_PCI_SELINKDNMSK, 0x3EC18, REGDWORD, "Switch Event Link Down Mask", ## args) \
+ X(IDT_SW_PCI_SEFRSTSTS, 0x3EC1C, REGDWORD, "Switch Event Fundamental Reset Status", ## args) \
+ X(IDT_SW_PCI_SEFRSTMSK, 0x3EC20, REGDWORD, "Switch Event Fundamental Reset Mask", ## args) \
+ X(IDT_SW_PCI_SEHRSTSTS, 0x3EC24, REGDWORD, "Switch Event Hot Reset Status", ## args) \
+ X(IDT_SW_PCI_SEHRSTMSK, 0x3EC28, REGDWORD, "Switch Event Hot Reset Mask", ## args) \
+ X(IDT_SW_PCI_SEFOVRMSK, 0x3EC2C, REGDWORD, "Switch Event Failover Mask", ## args) \
+ X(IDT_SW_PCI_SEGSIGSTS, 0x3EC30, REGDWORD, "Switch Event Global Signal Status", ## args) \
+ X(IDT_SW_PCI_SEGSIGMSK, 0x3EC34, REGDWORD, "Switch Event Global Signal Mask", ## args) \
+ /* Global Doorbell configuration registers */ \
+ X(IDT_SW_PCI_GDBELLSTS, 0x3EC3C, REGDWORD, "NT Global Doorbell Status", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK0, 0x3ED00, REGDWORD, "NT Global Outbound Doorbell 0 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK1, 0x3ED04, REGDWORD, "NT Global Outbound Doorbell 1 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK2, 0x3ED08, REGDWORD, "NT Global Outbound Doorbell 2 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK3, 0x3ED0C, REGDWORD, "NT Global Outbound Doorbell 3 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK4, 0x3ED10, REGDWORD, "NT Global Outbound Doorbell 4 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK5, 0x3ED14, REGDWORD, "NT Global Outbound Doorbell 5 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK6, 0x3ED18, REGDWORD, "NT Global Outbound Doorbell 6 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK7, 0x3ED1C, REGDWORD, "NT Global Outbound Doorbell 7 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK8, 0x3ED20, REGDWORD, "NT Global Outbound Doorbell 8 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK9, 0x3ED24, REGDWORD, "NT Global Outbound Doorbell 9 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK10, 0x3ED28, REGDWORD, "NT Global Outbound Doorbell 10 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK11, 0x3ED2C, REGDWORD, "NT Global Outbound Doorbell 11 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK12, 0x3ED30, REGDWORD, "NT Global Outbound Doorbell 12 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK13, 0x3ED34, REGDWORD, "NT Global Outbound Doorbell 13 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK14, 0x3ED38, REGDWORD, "NT Global Outbound Doorbell 14 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK15, 0x3ED3C, REGDWORD, "NT Global Outbound Doorbell 15 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK16, 0x3ED40, REGDWORD, "NT Global Outbound Doorbell 16 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK17, 0x3ED44, REGDWORD, "NT Global Outbound Doorbell 17 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK18, 0x3ED48, REGDWORD, "NT Global Outbound Doorbell 18 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK19, 0x3ED4C, REGDWORD, "NT Global Outbound Doorbell 19 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK20, 0x3ED50, REGDWORD, "NT Global Outbound Doorbell 20 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK21, 0x3ED54, REGDWORD, "NT Global Outbound Doorbell 21 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK22, 0x3ED58, REGDWORD, "NT Global Outbound Doorbell 22 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK23, 0x3ED5C, REGDWORD, "NT Global Outbound Doorbell 23 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK24, 0x3ED60, REGDWORD, "NT Global Outbound Doorbell 24 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK25, 0x3ED64, REGDWORD, "NT Global Outbound Doorbell 25 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK26, 0x3ED68, REGDWORD, "NT Global Outbound Doorbell 26 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK27, 0x3ED6C, REGDWORD, "NT Global Outbound Doorbell 27 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK28, 0x3ED70, REGDWORD, "NT Global Outbound Doorbell 28 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK29, 0x3ED74, REGDWORD, "NT Global Outbound Doorbell 29 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK30, 0x3ED78, REGDWORD, "NT Global Outbound Doorbell 30 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK31, 0x3ED7C, REGDWORD, "NT Global Outbound Doorbell 31 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK0, 0x3ED80, REGDWORD, "NT Global Inbound Doorbell 0 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK1, 0x3ED84, REGDWORD, "NT Global Inbound Doorbell 1 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK2, 0x3ED88, REGDWORD, "NT Global Inbound Doorbell 2 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK3, 0x3ED8C, REGDWORD, "NT Global Inbound Doorbell 3 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK4, 0x3ED90, REGDWORD, "NT Global Inbound Doorbell 4 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK5, 0x3ED94, REGDWORD, "NT Global Inbound Doorbell 5 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK6, 0x3ED98, REGDWORD, "NT Global Inbound Doorbell 6 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK7, 0x3ED9C, REGDWORD, "NT Global Inbound Doorbell 7 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK8, 0x3EDA0, REGDWORD, "NT Global Inbound Doorbell 8 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK9, 0x3EDA4, REGDWORD, "NT Global Inbound Doorbell 9 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK10, 0x3EDA8, REGDWORD, "NT Global Inbound Doorbell 10 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK11, 0x3EDAC, REGDWORD, "NT Global Inbound Doorbell 11 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK12, 0x3EDB0, REGDWORD, "NT Global Inbound Doorbell 12 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK13, 0x3EDB4, REGDWORD, "NT Global Inbound Doorbell 13 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK14, 0x3EDB8, REGDWORD, "NT Global Inbound Doorbell 14 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK15, 0x3EDBC, REGDWORD, "NT Global Inbound Doorbell 15 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK16, 0x3EDC0, REGDWORD, "NT Global Inbound Doorbell 16 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK17, 0x3EDC4, REGDWORD, "NT Global Inbound Doorbell 17 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK18, 0x3EDC8, REGDWORD, "NT Global Inbound Doorbell 18 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK19, 0x3EDCC, REGDWORD, "NT Global Inbound Doorbell 19 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK20, 0x3EDD0, REGDWORD, "NT Global Inbound Doorbell 20 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK21, 0x3EDD4, REGDWORD, "NT Global Inbound Doorbell 21 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK22, 0x3EDD8, REGDWORD, "NT Global Inbound Doorbell 22 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK23, 0x3EDDC, REGDWORD, "NT Global Inbound Doorbell 23 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK24, 0x3EDE0, REGDWORD, "NT Global Inbound Doorbell 24 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK25, 0x3EDE4, REGDWORD, "NT Global Inbound Doorbell 25 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK26, 0x3EDE8, REGDWORD, "NT Global Inbound Doorbell 26 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK27, 0x3EDEC, REGDWORD, "NT Global Inbound Doorbell 27 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK28, 0x3EDF0, REGDWORD, "NT Global Inbound Doorbell 28 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK29, 0x3EDF4, REGDWORD, "NT Global Inbound Doorbell 29 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK30, 0x3EDF8, REGDWORD, "NT Global Inbound Doorbell 30 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK31, 0x3EDFC, REGDWORD, "NT Global Inbound Doorbell 31 Mask", ## args) \
+ /* Switch partition messages control (msgs routing table) */ \
+ X(IDT_SW_PCI_SWP0MSGCTL0, 0x3EE00, REGDWORD, "Switch Partition 0 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL0, 0x3EE04, REGDWORD, "Switch Partition 1 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL0, 0x3EE08, REGDWORD, "Switch Partition 2 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL0, 0x3EE0C, REGDWORD, "Switch Partition 3 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL0, 0x3EE10, REGDWORD, "Switch Partition 4 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL0, 0x3EE14, REGDWORD, "Switch Partition 5 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL0, 0x3EE18, REGDWORD, "Switch Partition 6 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL0, 0x3EE1C, REGDWORD, "Switch Partition 7 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP0MSGCTL1, 0x3EE20, REGDWORD, "Switch Partition 0 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL1, 0x3EE24, REGDWORD, "Switch Partition 1 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL1, 0x3EE28, REGDWORD, "Switch Partition 2 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL1, 0x3EE2C, REGDWORD, "Switch Partition 3 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL1, 0x3EE30, REGDWORD, "Switch Partition 4 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL1, 0x3EE34, REGDWORD, "Switch Partition 5 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL1, 0x3EE38, REGDWORD, "Switch Partition 6 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL1, 0x3EE3C, REGDWORD, "Switch Partition 7 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP0MSGCTL2, 0x3EE40, REGDWORD, "Switch Partition 0 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL2, 0x3EE44, REGDWORD, "Switch Partition 1 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL2, 0x3EE48, REGDWORD, "Switch Partition 2 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL2, 0x3EE4C, REGDWORD, "Switch Partition 3 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL2, 0x3EE50, REGDWORD, "Switch Partition 4 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL2, 0x3EE54, REGDWORD, "Switch Partition 5 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL2, 0x3EE58, REGDWORD, "Switch Partition 6 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL2, 0x3EE5C, REGDWORD, "Switch Partition 7 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP0MSGCTL3, 0x3EE60, REGDWORD, "Switch Partition 0 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL3, 0x3EE64, REGDWORD, "Switch Partition 1 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL3, 0x3EE68, REGDWORD, "Switch Partition 2 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL3, 0x3EE6C, REGDWORD, "Switch Partition 3 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL3, 0x3EE70, REGDWORD, "Switch Partition 4 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL3, 0x3EE74, REGDWORD, "Switch Partition 5 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL3, 0x3EE78, REGDWORD, "Switch Partition 6 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL3, 0x3EE7C, REGDWORD, "Switch Partition 7 Message Control 3", ## args) \
+ /* SerDes's control registers */ \
+ X(IDT_SW_PCI_S0CTL, 0x3F000, REGDWORD, "SerDes 0 Control", ## args) \
+ X(IDT_SW_PCI_S0TXLCTL0, 0x3F004, REGDWORD, "SerDes 0 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S0TXLCTL1, 0x3F008, REGDWORD, "SerDes 0 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S0RXEQLCTL, 0x3F010, REGDWORD, "SerDes 0 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S1CTL, 0x3F020, REGDWORD, "SerDes 1 Control", ## args) \
+ X(IDT_SW_PCI_S1TXLCTL0, 0x3F024, REGDWORD, "SerDes 1 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S1TXLCTL1, 0x3F028, REGDWORD, "SerDes 1 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S1RXEQLCTL, 0x3F030, REGDWORD, "SerDes 1 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S2CTL, 0x3F040, REGDWORD, "SerDes 2 Control", ## args) \
+ X(IDT_SW_PCI_S2TXLCTL0, 0x3F044, REGDWORD, "SerDes 2 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S2TXLCTL1, 0x3F048, REGDWORD, "SerDes 2 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S2RXEQLCTL, 0x3F050, REGDWORD, "SerDes 2 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S3CTL, 0x3F060, REGDWORD, "SerDes 3 Control", ## args) \
+ X(IDT_SW_PCI_S3TXLCTL0, 0x3F064, REGDWORD, "SerDes 3 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S3TXLCTL1, 0x3F068, REGDWORD, "SerDes 3 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S3RXEQLCTL, 0x3F070, REGDWORD, "SerDes 3 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S4CTL, 0x3F080, REGDWORD, "SerDes 4 Control", ## args) \
+ X(IDT_SW_PCI_S4TXLCTL0, 0x3F084, REGDWORD, "SerDes 4 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S4TXLCTL1, 0x3F088, REGDWORD, "SerDes 4 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S4RXEQLCTL, 0x3F090, REGDWORD, "SerDes 4 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S5CTL, 0x3F0A0, REGDWORD, "SerDes 5 Control", ## args) \
+ X(IDT_SW_PCI_S5TXLCTL0, 0x3F0A4, REGDWORD, "SerDes 5 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S5TXLCTL1, 0x3F0A8, REGDWORD, "SerDes 5 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S5RXEQLCTL, 0x3F0B0, REGDWORD, "SerDes 5 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S6CTL, 0x3F0C0, REGDWORD, "SerDes 6 Control", ## args) \
+ X(IDT_SW_PCI_S6TXLCTL0, 0x3F0C4, REGDWORD, "SerDes 6 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S6TXLCTL1, 0x3F0C8, REGDWORD, "SerDes 6 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S6RXEQLCTL, 0x3F0D0, REGDWORD, "SerDes 6 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S7CTL, 0x3F0E0, REGDWORD, "SerDes 7 Control", ## args) \
+ X(IDT_SW_PCI_S7TXLCTL0, 0x3F0E4, REGDWORD, "SerDes 7 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S7TXLCTL1, 0x3F0E8, REGDWORD, "SerDes 7 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S7RXEQLCTL, 0x3F0F0, REGDWORD, "SerDes 7 Receiver Equalization Lane Control", ## args) \
+ /* GPIO/Hot-plug control registers */ \
+ X(IDT_SW_PCI_GPIOFUNC, 0x3F16C, REGDWORD, "General Purpose I/O Function", ## args) \
+ X(IDT_SW_PCI_GPIOAFSEL, 0x3F170, REGDWORD, "General Purpose I/O Alternate Function Select", ## args) \
+ X(IDT_SW_PCI_GPIOCFG, 0x3F174, REGDWORD, "General Purpose I/O Configuration", ## args) \
+ X(IDT_SW_PCI_GPIOD, 0x3F178, REGDWORD, "General Purpose I/O Data", ## args) \
+ X(IDT_SW_PCI_HPCFGCTL, 0x3F17C, REGDWORD, "Hot-Plug Configuration Control", ## args) \
+ /* SMBus related registers */ \
+ X(IDT_SW_PCI_SMBUSSTS, 0x3F188, REGDWORD, "SMBus Status", ## args) \
+ X(IDT_SW_PCI_SMBUSCTL, 0x3F18C, REGDWORD, "SMBus Control", ## args) \
+ X(IDT_SW_PCI_EEPROMINTF, 0x3F190, REGDWORD, "Serial EEPROM Interface", ## args) \
+ /* SMBus IO expanders */ \
+ X(IDT_SW_PCI_IOEXPADDR0, 0x3F198, REGDWORD, "SMBus I/O Expander Address 0", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR1, 0x3F19C, REGDWORD, "SMBus I/O Expander Address 1", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR2, 0x3F1A0, REGDWORD, "SMBus I/O Expander Address 2", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR3, 0x3F1A4, REGDWORD, "SMBus I/O Expander Address 3", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR4, 0x3F1A8, REGDWORD, "SMBus I/O Expander Address 4", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR5, 0x3F1AC, REGDWORD, "SMBus I/O Expander Address 5", ## args) \
+ /* General Purpose Events registers */ \
+ X(IDT_SW_PCI_GPECTL, 0x3F1B0, REGDWORD, "General Purpose Event Control", ## args) \
+ X(IDT_SW_PCI_GPESTS, 0x3F1B4, REGDWORD, "General Purpose Event Status", ## args) \
+ /* Temperature sensor */ \
+ X(IDT_SW_PCI_TMPCTL, 0x3F1D4, REGDWORD, "Temperature Sensor Control", ## args) \
+ X(IDT_SW_PCI_TMPSTS, 0x3F1D8, REGDWORD, "Temperature Sensor Status", ## args) \
+ X(IDT_SW_PCI_TMPALARM, 0x3F1DC, REGDWORD, "Temperature Sensor Alarm", ## args) \
+ X(IDT_SW_PCI_TMPADJ, 0x3F1E0, REGDWORD, "Temperature Sensor Adjustment", ## args) \
+ X(IDT_SW_PCI_TSSLOPE, 0x3F1E4, REGDWORD, "Temperature Sensor Slope", ## args) \
+ /* SMBus Configuration Block header log */ \
+ X(IDT_SW_PCI_SMBUSCBHL, 0x3F1E8, REGDWORD, "SMBus Configuration Block Header Log", ## args)
+
+/*
+ * Enumeration of the IDT PCIe-switch NT registers. It's not actual
+ * addresses or offsets, but the numerated names, which are used to find the
+ * necessary values from the tables above. Consenquently the switch-case
+ * shall help to retrieve all the information for the IO operations. Of course,
+ * we are sure the compiler will translate that statement into the jump table
+ * pattern.
+ *
+ * NOTE 1) The IDT PCIe-switch internal data is littel-endian
+ * so it must be taken into account in the driver
+ * internals.
+ * 2) Additionally the registers should be accessed either
+ * with byte-enables corresponding to their native size or
+ * a size of one DWORD
+ * 3) Global registers registers can be accessed by the
+ * GASAADDR and GASADATA registers of NT-functions only
+ */
+enum idt_ntb_cfgreg {
+ IDT_NT_CFGREGS(PAIR_ID_ENUM)
+ IDT_NTB_CFGREGS_SPLIT,
+ IDT_SW_CFGREGS(PAIR_ID_ENUM)
+ IDT_NTB_CFGREGS_END
+};
+
+/*
+ * IDT PCIe-switch register type. It's vital that the types are assigned
+ * with 0 and 1 since those values are used to determine the registers IO
+ * context
+ * @IDT_NT_REGTYPE: NT-function register accessed using the mmio
+ * @IDT_SW_REGTYPE: IDT PCIe-switch Gobal register accessed using GASA regs
+ */
+enum idt_ntb_regtype {
+ IDT_NT_REGTYPE = 0,
+ IDT_SW_REGTYPE = 1
+};
+
+/*
+ * R/W registers operation context structure
+ * @writereg: Register write function
+ * @readreg: Register read function
+ * @iolock: Spin lock of the registers access
+ */
+struct idt_ntb_regctx {
+ void (*writereg)(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val);
+ u32 (*readreg)(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize);
+ spinlock_t iolock;
+};
+
+#endif /* NTB_HW_IDT_REGMAP_H */
--
2.6.6
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 3/3] ntb: Test client drivers for asynchronous NTB devices
2016-07-26 19:50 [PATCH 0/3] ntb: Asynchronous NTB devices support Serge Semin
2016-07-26 19:50 ` [PATCH 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
2016-07-26 19:50 ` [PATCH 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
@ 2016-07-26 19:50 ` Serge Semin
2016-07-28 12:58 ` kbuild test robot
2016-07-28 10:01 ` [PATCH v2 0/3] ntb: Asynchronous NTB devices support Serge Semin
3 siblings, 1 reply; 11+ messages in thread
From: Serge Semin @ 2016-07-26 19:50 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
There are three drivers to independently test all interfaces implemented by
the IDT 89HPES*NT* NTB driver.
Doorbells are tested by new NTB Doorbell Pingpong client driver. It implements
the so-named algorithm. Driver starts working from setting the peer doorbell of
the last locally set doorbell bit. If there has not been locally set doorbell,
it sets the very first bit. After that the driver unmasks the events of the
just set doorbell bit and waits until the peer sets the same doorbell. When
peer does it, the local driver iterates to the next doorbell bit and starts
delayed work thread, which will set the corresponding bit and perform doorbell
bit umasking on waking up.
Messaging subsystem can be tested by the client driver implementing a simple
transmition/reception algorithm. A message can be send to a peer by writing
data to /sys/kernel/debug/ntb_msg_test/ntbA{N}/data file. The peer can read
it from the same file then.
Memory Windows test driver implements a simple write/read algorithm. The driver
allocates the predefined number of local buffers (inbound memory window -
inwndw{N}). In order to get a translated base address driver sends a
corresponding command to a peer. Then driver initialize the outbound memory
windows (outwndw{N}). The read/write operations can be performed using the
following debug nodes:
/sys/kernel/debug/ntb_mw_test/ntbA{N}/inwndw{N}
/sys/kernel/debug/ntb_mw_test/ntbA{N}/outwndw{N}
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
drivers/ntb/test/Kconfig | 32 +
drivers/ntb/test/Makefile | 9 +-
drivers/ntb/test/ntb_db_test.c | 677 +++++++++++++++++
drivers/ntb/test/ntb_msg_test.c | 736 +++++++++++++++++++
drivers/ntb/test/ntb_mw_test.c | 1531 +++++++++++++++++++++++++++++++++++++++
5 files changed, 2983 insertions(+), 2 deletions(-)
create mode 100644 drivers/ntb/test/ntb_db_test.c
create mode 100644 drivers/ntb/test/ntb_msg_test.c
create mode 100644 drivers/ntb/test/ntb_mw_test.c
diff --git a/drivers/ntb/test/Kconfig b/drivers/ntb/test/Kconfig
index a5d0eda..80f5058 100644
--- a/drivers/ntb/test/Kconfig
+++ b/drivers/ntb/test/Kconfig
@@ -25,3 +25,35 @@ config NTB_PERF
to and from the window without additional software interaction.
If unsure, say N.
+
+config NTB_DB_TEST
+ tristate "NTB Doorbell Test Client"
+ help
+ This is a driver to test doorbell subsystem of NTB bus devices.
+ The design is similar to the ping pong although it exchanges the
+ doorbell bits one-by-one, waiting for the peer response before getting
+ to a next doorbell.
+
+ If unsure, say N.
+
+config NTB_MSG_TEST
+ tristate "NTB Messaging Test Client"
+ help
+ This is a driver to test messaging subsystem of NTB. It just creates
+ one file in the DebugFS for each NTB device of asynchronous
+ architecture. In order to send a message one can just write a text to
+ the file. It will be immediately sent to the peer so user can get it
+ by reading from the corresponding file.
+
+ If unsure, say N.
+
+config NTB_MW_TEST
+ tristate "NTB Memory Windows Test Client"
+ help
+ This is a driver to test memory sharing amongst devices. It creates a
+ set of files in the DebugFS, one of which are used to write a text to
+ outbound memory windows and anothers can be used to read data written
+ by the peer to our inbound memory window.
+
+ If unsure, say N.
+
diff --git a/drivers/ntb/test/Makefile b/drivers/ntb/test/Makefile
index 9e77e0b..6ea6db4 100644
--- a/drivers/ntb/test/Makefile
+++ b/drivers/ntb/test/Makefile
@@ -1,3 +1,8 @@
+# Synchronous hardware clients (Intel/AMD)
obj-$(CONFIG_NTB_PINGPONG) += ntb_pingpong.o
-obj-$(CONFIG_NTB_TOOL) += ntb_tool.o
-obj-$(CONFIG_NTB_PERF) += ntb_perf.o
+obj-$(CONFIG_NTB_TOOL) += ntb_tool.o
+obj-$(CONFIG_NTB_PERF) += ntb_perf.o
+# Asynchronous hardware clients (IDT)
+obj-$(CONFIG_NTB_DB_TEST) += ntb_db_test.o
+obj-$(CONFIG_NTB_MSG_TEST) += ntb_msg_test.o
+obj-$(CONFIG_NTB_MW_TEST) += ntb_mw_test.o
diff --git a/drivers/ntb/test/ntb_db_test.c b/drivers/ntb/test/ntb_db_test.c
new file mode 100644
index 0000000..e93c0c6
--- /dev/null
+++ b/drivers/ntb/test/ntb_db_test.c
@@ -0,0 +1,677 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * PCIe NTB doorbells test Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the NTB doorbells pingpong driver design.
+ * The driver is designed to implement the pingpong algorithm. After a quick
+ * initailization the driver starts from setting the peer doorbell of the last
+ * locally set doorbell bit. If there is not any doorbell locally set, then it
+ * sets the very first bit. After that the driver unmasks the events of the
+ * just set bits and waits until the peer is set the same doorbell. When it's
+ * done, the driver iterates to the next doorbell and starts delayed work
+ * thread, which will set the corresponding bit and perform doorbell umasking
+ * on waking up.
+ */
+
+/* Note: You can load this module with either option 'dyndbg=+p' or define the
+ * next preprocessor constant */
+/*#define DEBUG*/
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+
+#include <linux/ntb.h>
+
+#define DRIVER_NAME "ntb_db_test"
+#define DRIVER_DESCRIPTION "PCIe NTB Doorbells Pingpong Client"
+#define DRIVER_VERSION "1.0"
+
+MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+static unsigned int delay_ms = 1000;
+module_param(delay_ms, uint, 0644);
+MODULE_PARM_DESC(delay_ms,
+ "Milliseconds to delay before setting a next doorbell bit");
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_dir;
+
+/*
+ * Enumeration of the driver states
+ * @PP_WAIT: Driver waits until the peer sets the corresponding doorbell bit
+ * @PP_SLEEP: Driver sleeps before to set the next doorbell bit
+ */
+enum db_pp_state {
+ PP_WAIT = 0,
+ PP_SLEEP = 1
+};
+
+/*
+ * Doorbells pingpong driver context
+ * @ntb: Pointer to the NTB device
+ * @cycle: Doorbells setting cycle made up until now
+ * @valid_ids: Valid Doorbel bits
+ * @delay: Delay between setting the next doorbell bit
+ * @state: Current cycle state
+ * @dwork: Kernel thread used to perform the delayed doorbell bit set
+ * @dbgfs_info: Handler of the DebugFS driver info-file
+ */
+struct pp_ctx {
+ struct ntb_dev *ntb;
+ unsigned long long cycle;
+ u64 valid_ids;
+ unsigned long delay;
+ enum db_pp_state state;
+ struct delayed_work dwork;
+ struct dentry *dbgfs_info;
+};
+#define to_ctx_dwork(work) \
+ container_of(to_delayed_work(work), struct pp_ctx, dwork)
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_pp(ctx, args...) \
+ dev_err(&ctx->ntb->dev, ## args)
+#define dev_warn_pp(ctx, args...) \
+ dev_warn(&ctx->ntb->dev, ## args)
+#define dev_info_pp(ctx, args...) \
+ dev_info(&ctx->ntb->dev, ## args)
+#define dev_dbg_pp(ctx, args...) \
+ dev_dbg(&ctx->ntb->dev, ## args)
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+
+/*===========================================================================
+ * Helper functions
+ *===========================================================================*/
+
+/*
+ * Create a contiguous bitmask starting at bit position @l and ending at
+ * position @h. For example
+ * GENMASK_ULL(39, 21) gives us the 64bit vector 0x000000ffffe00000.
+ */
+#ifndef GENMASK_ULL
+#define GENMASK_ULL(h, l) \
+ (((~0ULL) << (l)) & (~0ULL >> (BITS_PER_LONG_LONG - 1 - (h))))
+#endif /* !GENMASK_ULL */
+
+/*
+ * Set the corresponding bit in the 64-bits wide word
+ */
+#ifndef BIT_ULL
+#define BIT_ULL(nr) (1ULL << (nr))
+#endif /* !BIT_ULL */
+
+/*
+ * Method to find a first set bit in 64-bits wide word. The bits numbering is
+ * from 0 to 63. If there is no any set bit, then 64 is returned.
+ */
+static inline unsigned long find_first_bit64(u64 var)
+{
+ return (0x0ULL == var) ? BITS_PER_LONG_LONG : __ffs64(var);
+}
+
+/*
+ * Method to find a next set bit in 64-bits wide word starting from the
+ * specified position. The bits numbering is from 0 to 63. If there is no any
+ * set bit within the position and the last bit of the word, then 64 is
+ * returned.
+ */
+static inline unsigned long find_next_bit64(u64 var, unsigned long pos)
+{
+ /* Consider only the valuable positions */
+ var &= GENMASK_ULL(BITS_PER_LONG_LONG - 1, pos);
+
+ return find_first_bit64(var);
+}
+
+/*===========================================================================
+ * Pingpong algorithm functions definition
+ *===========================================================================*/
+
+/*
+ * Iterate Doorbell PingPong algorithm work thread
+ * This function clears the currently set doorbell bit, which has been
+ * unmasked before, and masks it back. Then method sets the next doorbell
+ * bit and locally unmasks it.
+ */
+static void pp_iterate_cycle(struct work_struct *work)
+{
+ struct pp_ctx *ctx = to_ctx_dwork(work);
+ struct ntb_dev *ntb = ctx->ntb;
+ u64 db_umsk, db_sts;
+ unsigned long db_id;
+ int ret;
+
+ /* Read the mask of the current disposition */
+ db_umsk = ~ntb_db_read_mask(ntb) & ctx->valid_ids;
+ if (1 != hweight64(db_umsk)) {
+ dev_err_pp(ctx,
+ "Got invalid doorbells mask %#018llx", db_umsk);
+ return;
+ }
+
+ /* Read the currently set doorbells */
+ db_sts = ntb_db_read(ntb);
+ if (0x0 == (db_sts & db_umsk)) {
+ dev_err_pp(ctx, "Got driver bug %#018llx & %#018llx == 0",
+ db_sts, db_umsk);
+ return;
+ }
+
+ /* Find the doorbell id (use db_umsk since db_sts can have several
+ * bits set) */
+ db_id = find_first_bit64(db_umsk);
+
+ dev_dbg_pp(ctx, "PingPong the doorbell bit %lu of cycle %llu",
+ db_id, ctx->cycle);
+
+ /* Mask the currently unmasked doorbell */
+ ret = ntb_db_set_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx, "Failed to mask db %lu by %#018llx",
+ db_id, db_umsk);
+ return;
+ }
+
+ /* Clear the currently set doorbell */
+ ret = ntb_db_clear(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to clear the db bit %lu", db_id);
+ return;
+ }
+
+ /* Iterate the doorbell id to set the next doorbell bit */
+ db_id = find_next_bit64(ctx->valid_ids, db_id + 1);
+ if (BITS_PER_LONG_LONG == db_id) {
+ db_id = find_first_bit64(ctx->valid_ids);
+ ctx->cycle++;
+ }
+
+ /* Calculate the new unmasking field */
+ db_umsk = BIT_ULL(db_id);
+
+ /* Set the new peer doorbell bit */
+ ret = ntb_peer_db_set(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to set the peer doorbell %lu by field %#018llx",
+ db_id, db_umsk);
+ return;
+ }
+
+ /* After this the driver is waiting for the peer response */
+ ctx->state = PP_WAIT;
+
+ /* Unmask the corresponding doorbell bit to receive the event */
+ ret = ntb_db_clear_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to unmask the doorbell %lu by field %#018llx",
+ db_id, db_umsk);
+ return;
+ }
+}
+
+/*
+ * Handle the event of Doorbell set
+ */
+static void pp_db_event(void *data, int vec)
+{
+ struct pp_ctx *ctx = data;
+
+ /* From now the driver is sleeping before sending the response */
+ ctx->state = PP_SLEEP;
+
+ /* Schedule the delayed work of the algorithm */
+ (void)schedule_delayed_work(&ctx->dwork, ctx->delay);
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t pp_dbgfs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+/*
+ * Driver DebugFS operations
+ */
+static const struct file_operations pp_dbgfs_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = pp_dbgfs_read
+};
+
+/*
+ * DebugFS read node info callback
+ */
+static ssize_t pp_dbgfs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ struct pp_ctx *ctx = filp->private_data;
+ struct ntb_dev *ntb = ctx->ntb;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, count, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Doorbells PingPong test driver:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "Link state\t- %s\n",
+ (ON == ntb_link_is_up(ntb, NULL, NULL)) ? "Up" : "Down");
+ off += scnprintf(strbuf + off, size - off,
+ "Cycle\t\t- %llu\n", ctx->cycle);
+ off += scnprintf(strbuf + off, size - off,
+ "Algo state\t- %s\n",
+ (PP_SLEEP == ctx->state) ? "sleep" : "wait");
+ off += scnprintf(strbuf + off, size - off,
+ "Delay\t\t- %u ms\n", delay_ms);
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, count, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * Driver DebugFS initialization function
+ */
+static int pp_init_dbgfs(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ const char *devname;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_dir)) {
+ dev_warn_pp(ctx,
+ "Top DebugFS directory has not been created for "
+ DRIVER_NAME);
+ return PTR_ERR(dbgfs_dir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(&ntb->dev);
+
+ /* Create the corresponding file node */
+ ctx->dbgfs_info = debugfs_create_file(devname, S_IRUSR,
+ dbgfs_dir, ctx, &pp_dbgfs_ops);
+ if (IS_ERR(ctx->dbgfs_info)) {
+ dev_err_pp(ctx, "Could not create the DebugFS node %s",
+ devname);
+ return PTR_ERR(ctx->dbgfs_info);
+ }
+
+ dev_dbg_pp(ctx, "Doorbell PingPong DebugFS node is created for %s",
+ devname);
+
+ return SUCCESS;
+}
+
+/*
+ * Driver DebugFS deinitialization function
+ */
+static void pp_deinit_dbgfs(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Remove the DebugFS file */
+ debugfs_remove(ctx->dbgfs_info);
+
+ dev_dbg_pp(ctx, "Doorbell PingPong DebugFS node %s is discarded",
+ dev_name(&ntb->dev));
+}
+
+/*===========================================================================
+ * NTB device/client driver initialization
+ *===========================================================================*/
+
+/*
+ * NTB device events handlers
+ */
+static const struct ntb_ctx_ops pp_ops = {
+ .db_event = pp_db_event
+};
+
+/*
+ * Create the driver context structure
+ */
+static struct pp_ctx *pp_create_ctx(struct ntb_dev *ntb)
+{
+ struct pp_ctx *ctx;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&ntb->dev);
+ ctx = kzalloc_node(sizeof(*ctx), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx)) {
+ dev_err(&ntb->dev,
+ "No memory for NTB PingPong driver context");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Initialize the NTB device descriptor and delayed work */
+ ctx->ntb = ntb;
+ ctx->cycle = 0;
+ ctx->valid_ids = ntb_db_valid_mask(ntb);
+ ctx->delay = msecs_to_jiffies(delay_ms);
+ ctx->state = PP_WAIT;
+ INIT_DELAYED_WORK(&ctx->dwork, pp_iterate_cycle);
+
+ dev_dbg_pp(ctx, "Context structure is created");
+
+ return ctx;
+}
+
+/*
+ * Free the driver context structure
+ */
+static void pp_free_ctx(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Just free the memory allocated for the context structure */
+ kfree(ctx);
+
+ dev_dbg(&ntb->dev, "Context structure is freed");
+}
+
+/*
+ * Correspondingly initialize the ntb device structure
+ */
+static int pp_init_ntb_dev(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ int ret;
+
+ /* Set the NTB device events context */
+ ret = ntb_set_ctx(ntb, ctx, &pp_ops);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx, "Failed to specify the NTB device context");
+ return ret;
+ }
+
+ /* Enable the link */
+ ntb_link_enable(ntb, NTB_SPEED_AUTO, NTB_WIDTH_AUTO);
+ /*ntb_link_event(ntb);*/
+
+ dev_dbg_pp(ctx, "NTB device is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the ntb device structure
+ */
+static void pp_stop_ntb_dev(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Disable the link */
+ ntb_link_disable(ntb);
+
+ /* Clear the context to make sure there won't be any doorbell event */
+ ntb_clear_ctx(ntb);
+
+ dev_dbg_pp(ctx, "NTB device is deinitialized");
+}
+
+/*
+ * Initialize the basic algorithm-related fields
+ */
+static int pp_init_algo(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ u64 db_sts, db_umsk;
+ int ret;
+
+ /* Read the current mask */
+ db_umsk = ~ntb_db_read_mask(ntb) & ctx->valid_ids;
+
+ /* If all doorbell have been unmasked then mask them all */
+ if (db_umsk == ctx->valid_ids) {
+ ret = ntb_db_set_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to mask all the doorbells "
+ "%#018llx", db_umsk);
+ return ret;
+ }
+ /* Set the unmasking variable to zero so the algorithm would
+ * initialize the corresponding DB bit */
+ db_umsk = 0;
+ }
+
+ /* If there is no any unmasked bit then set the very first peer doorbell
+ * bit and locally unmask it */
+ if (0x0 == db_umsk) {
+ db_umsk = BIT_ULL(0);
+ /* Set the new peer doorbell bit */
+ ret = ntb_peer_db_set(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to set the peer doorbell %u by field "
+ "%#018llx", 0, db_umsk);
+ return ret;
+ }
+ /* Clear the mask of the corresponding doorbell bit */
+ ret = ntb_db_clear_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to unmask the doorbell %u by field "
+ "%#018llx", 0, db_umsk);
+ return ret;
+ }
+ }
+ /* If there is one umasked bit then just read the doorbell status.
+ * If the bit is set then just start the work thread to handle the
+ * disposition otherwise just don't do anything waiting for the peer
+ * to set the doorbell bit */
+ else if (1 == hweight64(db_umsk)) {
+ db_sts = ntb_db_read(ntb);
+ if (0x0 != (db_sts & db_umsk)) {
+ /* Schedule the delayed work of the algorithm */
+ (void)schedule_delayed_work(&ctx->dwork, ctx->delay);
+ }
+ } else /* if (1 < hweight64(db_umsk)) */ {
+ dev_err_pp(ctx, "Invalid mask is found %#018llx", db_umsk);
+ return -EINVAL;
+ }
+
+ dev_dbg_pp(ctx, "Doorbell PingPong algorithm is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Stop the driver algorithm
+ */
+static void pp_stop_algo(struct pp_ctx *ctx)
+{
+ /* Make sure the delayed work is not started */
+ cancel_delayed_work_sync(&ctx->dwork);
+
+ dev_dbg_pp(ctx, "Doorbell PingPong algorithm is stopped");
+}
+
+/*
+ * NTB device probe() callback function
+ */
+static int pp_probe(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct pp_ctx *ctx;
+ int ret;
+
+ /* Both synchronous and asynchronous hardware is supported */
+ if (!ntb_valid_sync_dev_ops(ntb) && !ntb_valid_async_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
+ /* Create the current device context */
+ ctx = pp_create_ctx(ntb);
+ if (IS_ERR_OR_NULL(ctx)) {
+ return PTR_ERR(ctx);
+ }
+
+ /* Initialize the NTB device */
+ ret = pp_init_ntb_dev(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_ctx;
+ }
+
+ /* Initialize the pingpong algorithm */
+ ret = pp_init_algo(ctx);
+ if (SUCCESS != ret) {
+ goto err_stop_ntb_dev;
+ }
+
+ /* Create the DebugFS node */
+ (void)pp_init_dbgfs(ctx);
+
+ /* Start */
+
+ return SUCCESS;
+
+/*err_stop_algo:
+ pp_stop_algo(ctx);
+*/
+err_stop_ntb_dev:
+ pp_stop_ntb_dev(ctx);
+
+err_free_ctx:
+ pp_free_ctx(ctx);
+
+ return ret;
+}
+
+/*
+ * NTB device remove() callback function
+ */
+static void pp_remove(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct pp_ctx *ctx = ntb->ctx;
+
+ /* Remove the DebugFS node */
+ pp_deinit_dbgfs(ctx);
+
+ /* Disable the NTB device link and clear the context */
+ pp_stop_ntb_dev(ctx);
+
+ /* Stop the algorithm */
+ pp_stop_algo(ctx);
+
+ /* Free the allocated context */
+ pp_free_ctx(ctx);
+}
+
+/*
+ * NTB bus client driver structure definition
+ */
+static struct ntb_client pp_client = {
+ .ops = {
+ .probe = pp_probe,
+ .remove = pp_remove,
+ },
+};
+/* module_ntb_client(pp_client); */
+
+/*
+ * Driver initialize method
+ */
+static int __init ntb_pp_init(void)
+{
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_dir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Registers the client driver */
+ return ntb_register_client(&pp_client);
+}
+module_init(ntb_pp_init);
+
+/*
+ * Driver exit method
+ */
+static void __exit ntb_pp_exit(void)
+{
+ /* Unregister the client driver */
+ ntb_unregister_client(&pp_client);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_dir);
+}
+module_exit(ntb_pp_exit);
+
diff --git a/drivers/ntb/test/ntb_msg_test.c b/drivers/ntb/test/ntb_msg_test.c
new file mode 100644
index 0000000..a4aecf4
--- /dev/null
+++ b/drivers/ntb/test/ntb_msg_test.c
@@ -0,0 +1,736 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * PCIe NTB messaging test Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the NTB Messaging driver design.
+ * The driver is designed to implement the simple transmition/reception
+ * algorithm. User can send data to a peer by writing it to
+ * debugfs:ntb_msg_test/ntbA_/data file, and one can read it by reading the
+ * same file on the opposite side.
+ */
+
+/* Note: You can load this module with either option 'dyndbg=+p' or define the
+ * next preprocessor constant */
+/*#define DEBUG*/
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+
+#include <linux/ntb.h>
+
+#define DRIVER_NAME "ntb_msg_test"
+#define DRIVER_DESCRIPTION "PCIe NTB Simple Messaging Client"
+#define DRIVER_VERSION "1.0"
+#define CACHE_NAME "ntb_msg_cache"
+
+MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_topdir;
+
+/*
+ * Doorbells pingpong driver context
+ * @ntb: Pointer to the NTB device
+ * @msg_cache: Messages wrapper slab
+ * @msg_lock: Spin lock to synchrnize access to the messages list
+ * @msg_list: List of received messages
+ * @msgcnt: Number of received messages
+ * @failed: Number of failed transfers
+ * @succeeded: Number of succeeded transfers
+ * @datasize: Maximum size of message data (in bytes) excluding the size byte
+ * @dbgfs_dir: Handler of the driver DebugFS directory
+ */
+struct msg_ctx {
+ struct ntb_dev *ntb;
+ struct kmem_cache *msg_cache;
+ spinlock_t msg_lock;
+ struct list_head msg_list;
+ unsigned long msgcnt;
+ unsigned long failed;
+ unsigned long succeeded;
+ size_t datasize;
+ struct dentry *dbgfs_dir;
+};
+
+/*
+ * Received messages container
+ * @msg: Message
+ * @entry: List entry
+ */
+struct ntb_msg_wrap {
+ struct ntb_msg msg;
+ struct list_head entry;
+};
+
+/*
+ * Message converter is used to translate the struct ntb_msg to the
+ * char sized data structure with size.
+ * @size: Size of the data
+ * @data: Pointer to the data buffer
+ */
+struct ntb_msg_conv {
+ u8 size;
+ char data[];
+};
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_msg(ctx, args...) \
+ dev_err(&ctx->ntb->dev, ## args)
+#define dev_warn_msg(ctx, args...) \
+ dev_warn(&ctx->ntb->dev, ## args)
+#define dev_info_msg(ctx, args...) \
+ dev_info(&ctx->ntb->dev, ## args)
+#define dev_dbg_msg(ctx, args...) \
+ dev_dbg(&ctx->ntb->dev, ## args)
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+
+/*===========================================================================
+ * Incoming messages handlers
+ *===========================================================================*/
+
+/*
+ * Save the receive message
+ */
+static void msg_recv_handler(struct msg_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct ntb_msg_wrap *wrap;
+ struct ntb_msg_conv *conv;
+
+ /* Cast the message to the converted one */
+ conv = (struct ntb_msg_conv *)msg;
+
+ /* Allocate the memory from the slab */
+ wrap = kmem_cache_alloc(ctx->msg_cache, GFP_KERNEL);
+ if (NULL == wrap) {
+ dev_err_msg(ctx,
+ "Failed to allocate memory for incoming message %.*s",
+ conv->size, conv->data);
+ return;
+ }
+
+ /* Copy the message to the buffer */
+ memcpy(&wrap->msg, msg, conv->size + 1);
+
+ /* Add the wrapped message to the list of received messages */
+ spin_lock(&ctx->msg_lock);
+ list_add_tail(&wrap->entry, &ctx->msg_list);
+ /* Increment the number of received messages in the buffer */
+ ctx->msgcnt++;
+ spin_unlock(&ctx->msg_lock);
+
+ dev_dbg_msg(ctx, "Message '%.*s' was received",
+ conv->size, conv->data);
+}
+
+/*
+ * Handler of the transmit errors
+ */
+static void msg_fail_handler(struct msg_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct ntb_msg_conv *conv = (struct ntb_msg_conv *)msg;
+
+ /* Just print the error increment the errors counter */
+ dev_err_msg(ctx,
+ "Failed to send the submessage '%.*s'",
+ conv->size, conv->data);
+ ctx->failed++;
+}
+
+/*
+ * Handler of the succeeded transmits
+ */
+static void msg_sent_handler(struct msg_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct ntb_msg_conv *conv = (struct ntb_msg_conv *)msg;
+
+ /* Just print the debug text and increment the succeeded msgs counter */
+ dev_dbg_msg(ctx,
+ "Submessage '%.*s' has been successfully sent",
+ conv->size, conv->data);
+ ctx->succeeded++;
+}
+
+/*
+ * Message event handler
+ */
+static void msg_event_handler(void *data, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg)
+{
+ struct msg_ctx *ctx = data;
+
+ /* Call the corresponding event handler */
+ switch (ev) {
+ case NTB_MSG_NEW:
+ msg_recv_handler(ctx, msg);
+ break;
+ case NTB_MSG_SENT:
+ msg_sent_handler(ctx, msg);
+ break;
+ case NTB_MSG_FAIL:
+ msg_fail_handler(ctx, msg);
+ break;
+ default:
+ dev_err_msg(ctx, "Got invalid message event %d", ev);
+ break;
+ }
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t msg_dbgfs_data_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t msg_dbgfs_data_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t msg_dbgfs_stat_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+/*
+ * DebugFS data node operations
+ */
+static const struct file_operations msg_dbgfs_data_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = msg_dbgfs_data_read,
+ .write = msg_dbgfs_data_write
+};
+
+/*
+ * DebugFS statistics node operations
+ */
+static const struct file_operations msg_dbgfs_stat_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = msg_dbgfs_stat_read,
+};
+
+/*
+ * DebugFS callback of read messages node
+ */
+static ssize_t msg_dbgfs_data_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct msg_ctx *ctx = filep->private_data;
+ struct list_head *entry, *safe_entry;
+ struct ntb_msg_wrap *wrap;
+ struct ntb_msg_conv *conv;
+ size_t datasize, retsize;
+ char *databuf;
+ ssize_t ret;
+
+ /* Find the size of the retrieved data in messages */
+ datasize = 0;
+ spin_lock(&ctx->msg_lock);
+ list_for_each(entry, &ctx->msg_list) {
+ wrap = list_entry(entry, struct ntb_msg_wrap, entry);
+ conv = (struct ntb_msg_conv *)&wrap->msg;
+ datasize += conv->size;
+ }
+ spin_unlock(&ctx->msg_lock);
+
+ /* Calculate the size of the output buffer */
+ datasize = min(datasize, usize);
+
+ /* Allocate the buffer */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_msg(ctx, "No memory to allocate the output buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data from the messages to the output buffer */
+ retsize = 0;
+ spin_lock(&ctx->msg_lock);
+ list_for_each_safe(entry, safe_entry, &ctx->msg_list) {
+ /* Get the message and copy it to the buffer */
+ wrap = list_entry(entry, struct ntb_msg_wrap, entry);
+ conv = (struct ntb_msg_conv *)&wrap->msg;
+
+ /* If there is no enough space left in the buffer then stop the
+ * loop */
+ if ((datasize - retsize) < conv->size) {
+ break;
+ }
+
+ /* Copy the data to the output buffer */
+ memcpy(&databuf[retsize], conv->data, conv->size);
+
+ /* Increment the size of the retrieved data */
+ retsize += conv->size;
+
+ /* Delete the list entry and free the memory */
+ list_del(&wrap->entry);
+ kmem_cache_free(ctx->msg_cache, wrap);
+
+ /* Decrement the number of messages in the buffer */
+ ctx->msgcnt--;
+ }
+ spin_unlock(&ctx->msg_lock);
+
+ /* Copy the text to the output buffer */
+ ret = simple_read_from_buffer(ubuf, usize, offp, databuf, retsize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS callback of write messages node
+ */
+static ssize_t msg_dbgfs_data_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct msg_ctx *ctx = filep->private_data;
+ struct ntb_dev *ntb = ctx->ntb;
+ struct ntb_msg msg;
+ struct ntb_msg_conv *conv;
+ char *databuf;
+ int pos, copied, sts = SUCCESS;
+ ssize_t ret;
+
+ /* Allocate the memory for sending data */
+ databuf = kmalloc(usize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_msg(ctx, "No memory to allocate the sending data buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data to the output buffer */
+ ret = simple_write_to_buffer(databuf, usize, offp, ubuf, usize);
+ if (0 > ret) {
+ dev_err_msg(ctx, "Failed to copy the data from the User-space");
+ kfree(databuf);
+ return ret;
+ }
+
+ /* Start copying data to the message structure and send it straight away
+ * to the peer */
+ conv = (struct ntb_msg_conv *)&msg;
+ for (pos = 0, copied = 0; pos < usize; pos += copied) {
+ /* Calculate the size of data to copy to the message */
+ copied = min(ctx->datasize, (usize - pos));
+ /* Set the data size of the message */
+ conv->size = copied;
+ /* Copy the data */
+ memcpy(conv->data, &databuf[pos], copied);
+
+ /* Send the data stright away */
+ sts = ntb_msg_post(ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_msg(ctx, "Failed to post the submessage %.*s",
+ copied, conv->data);
+ }
+ }
+
+ return (SUCCESS == sts) ? usize : -EINVAL;
+}
+
+/*
+ * DebugFS callback to read statistics
+ */
+static ssize_t msg_dbgfs_stat_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct msg_ctx *ctx = filep->private_data;
+ struct ntb_dev *ntb = ctx->ntb;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, usize, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ dev_dbg_msg(ctx,
+ "Failed to allocate the memory for statistics "
+ "output buffer");
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Messaging Test driver:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "Link state\t\t- %s\n",
+ (ON == ntb_link_is_up(ntb, NULL, NULL)) ? "Up" : "Down");
+ off += scnprintf(strbuf + off, size - off,
+ "Message count\t\t- %lu\n", ctx->msgcnt);
+ off += scnprintf(strbuf + off, size - off,
+ "Message size\t\t- %u\n", ntb_msg_size(ntb));
+ off += scnprintf(strbuf + off, size - off,
+ "Data size\t\t- %lu\n", (unsigned long)ctx->datasize);
+ off += scnprintf(strbuf + off, size - off,
+ "Successfully sent\t- %lu\n", ctx->succeeded);
+ off += scnprintf(strbuf + off, size - off,
+ "Failed to send\t\t- %lu\n", ctx->failed);
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, usize, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS initialization function
+ */
+static int msg_init_dbgfs(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct dentry *dbgfs_data, *dbgfs_stat;
+ const char *devname;
+ int ret;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_topdir)) {
+ dev_warn_msg(ctx,
+ "Top DebugFS directory has not been created for "
+ DRIVER_NAME);
+ return PTR_ERR(dbgfs_topdir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(&ntb->dev);
+
+ /* Create the device related subdirectory */
+ ctx->dbgfs_dir = debugfs_create_dir(devname, dbgfs_topdir);
+ if (IS_ERR_OR_NULL(ctx->dbgfs_dir)) {
+ dev_warn_msg(ctx,
+ "Failed to create the DebugFS subdirectory %s",
+ devname);
+ return PTR_ERR(ctx->dbgfs_dir);
+ }
+
+ /* Create the file node for data io operations */
+ dbgfs_data = debugfs_create_file("data", S_IRWXU, ctx->dbgfs_dir, ctx,
+ &msg_dbgfs_data_ops);
+ if (IS_ERR(dbgfs_data)) {
+ dev_err_msg(ctx, "Could not create DebugFS data node");
+ ret = PTR_ERR(dbgfs_data);
+ goto err_rm_dir;
+ }
+
+ /* Create the file node for statistics io operations */
+ dbgfs_stat = debugfs_create_file("stat", S_IRWXU, ctx->dbgfs_dir, ctx,
+ &msg_dbgfs_stat_ops);
+ if (IS_ERR(dbgfs_stat)) {
+ dev_err_msg(ctx, "Could not create DebugFS statistics node");
+ ret = PTR_ERR(dbgfs_stat);
+ goto err_rm_dir;
+ }
+
+ dev_dbg_msg(ctx, "NTB Messaging DebugFS nodes are created for %s",
+ devname);
+
+ return SUCCESS;
+
+err_rm_dir:
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ return ret;
+}
+
+/*
+ * DebugFS deinitialization function
+ */
+static void msg_deinit_dbgfs(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Remove the DebugFS directory */
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ dev_dbg_msg(ctx, "NTB Messaging DebugFS nodes %s/ are discarded",
+ dev_name(&ntb->dev));
+}
+
+/*===========================================================================
+ * NTB device/client driver initialization
+ *===========================================================================*/
+
+/*
+ * NTB device events handlers
+ */
+static const struct ntb_ctx_ops msg_ops = {
+ .msg_event = msg_event_handler
+};
+
+/*
+ * Create the driver context structure
+ */
+static struct msg_ctx *msg_create_ctx(struct ntb_dev *ntb)
+{
+ struct msg_ctx *ctx;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&ntb->dev);
+ ctx = kzalloc_node(sizeof(*ctx), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx)) {
+ dev_err(&ntb->dev,
+ "No memory for NTB Messaging driver context");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Create the message cache */
+ ctx->msg_cache = kmem_cache_create(CACHE_NAME,
+ sizeof(struct ntb_msg_wrap), 0, 0, NULL);
+ if (NULL == ctx->msg_cache) {
+ dev_err(&ntb->dev,
+ "Failed to allocate the message wrap structures cache");
+ kfree(ctx);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Initialize the context NTB device pointer */
+ ctx->ntb = ntb;
+
+ /* Initialize the message list lock and the list head */
+ spin_lock_init(&ctx->msg_lock);
+ INIT_LIST_HEAD(&ctx->msg_list);
+
+ /* Initialize the counters */
+ ctx->msgcnt = 0;
+ ctx->failed = 0;
+ ctx->succeeded = 0;
+
+ /* Initialize the data size of one message excluding the size byte */
+ ctx->datasize = 4*ntb_msg_size(ntb) - 1;
+
+ dev_dbg_msg(ctx, "Context structure is created");
+
+ return ctx;
+}
+
+/*
+ * Free the driver context structure
+ */
+static void msg_free_ctx(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct ntb_msg_wrap *wrap;
+ struct list_head *entry, *safe_entry;
+
+ /* Walk through the list of messages and destroy all the allocated
+ * memory */
+ spin_lock(&ctx->msg_lock);
+ list_for_each_safe(entry, safe_entry, &ctx->msg_list) {
+ /* Get the message wrapper */
+ wrap = list_entry(entry, struct ntb_msg_wrap, entry);
+
+ /* Delete the list entry and free the memory */
+ list_del(entry);
+ kmem_cache_free(ctx->msg_cache, wrap);
+
+ /* Decrement the number of messages in the buffer */
+ ctx->msgcnt--;
+ }
+ spin_unlock(&ctx->msg_lock);
+
+ /* Destroy the IDT messages cache */
+ kmem_cache_destroy(ctx->msg_cache);
+
+ /* Free the memory allocated for the context structure */
+ kfree(ctx);
+
+ dev_dbg(&ntb->dev, "Context structure is freed");
+}
+
+/*
+ * Initialize the ntb device structure
+ */
+static int msg_init_ntb_dev(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ int ret;
+
+ /* Set the NTB device events context */
+ ret = ntb_set_ctx(ntb, ctx, &msg_ops);
+ if (SUCCESS != ret) {
+ dev_err_msg(ctx, "Failed to specify the NTB device context");
+ return ret;
+ }
+
+ /* Enable the link */
+ ntb_link_enable(ntb, NTB_SPEED_AUTO, NTB_WIDTH_AUTO);
+ /*ntb_link_event(ntb);*/
+
+ dev_dbg_msg(ctx, "NTB device is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the ntb device structure
+ */
+static void msg_stop_ntb_dev(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Disable the link */
+ ntb_link_disable(ntb);
+
+ /* Clear the context */
+ ntb_clear_ctx(ntb);
+
+ dev_dbg_msg(ctx, "NTB device is deinitialized");
+}
+
+/*
+ * NTB device probe() callback function
+ */
+static int msg_probe(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct msg_ctx *ctx;
+ int ret;
+
+ /* Only asynchronous hardware is supported */
+ if (!ntb_valid_async_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
+ /* Create the current device context */
+ ctx = msg_create_ctx(ntb);
+ if (IS_ERR_OR_NULL(ctx)) {
+ return PTR_ERR(ctx);
+ }
+
+ /* Initialize the NTB device */
+ ret = msg_init_ntb_dev(ctx);
+ if (SUCCESS != ret) {
+ msg_free_ctx(ctx);
+ return ret;
+ }
+
+ /* Create the DebugFS node */
+ (void)msg_init_dbgfs(ctx);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB device remove() callback function
+ */
+static void msg_remove(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct msg_ctx *ctx = ntb->ctx;
+
+ /* Remove the DebugFS node */
+ msg_deinit_dbgfs(ctx);
+
+ /* Disable the NTB device link and clear the context */
+ msg_stop_ntb_dev(ctx);
+
+ /* Free the allocated context */
+ msg_free_ctx(ctx);
+}
+
+/*
+ * NTB bus client driver structure definition
+ */
+static struct ntb_client msg_client = {
+ .ops = {
+ .probe = msg_probe,
+ .remove = msg_remove,
+ },
+};
+/* module_ntb_client(msg_client); */
+
+/*
+ * Driver initialize method
+ */
+static int __init ntb_msg_init(void)
+{
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_topdir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Registers the client driver */
+ return ntb_register_client(&msg_client);
+}
+module_init(ntb_msg_init);
+
+/*
+ * Driver exit method
+ */
+static void __exit ntb_msg_exit(void)
+{
+ /* Unregister the client driver */
+ ntb_unregister_client(&msg_client);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_topdir);
+}
+module_exit(ntb_msg_exit);
+
diff --git a/drivers/ntb/test/ntb_mw_test.c b/drivers/ntb/test/ntb_mw_test.c
new file mode 100644
index 0000000..b65a002
--- /dev/null
+++ b/drivers/ntb/test/ntb_mw_test.c
@@ -0,0 +1,1531 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * PCIe NTB memory windows test Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the NTB memory windows test driver design.
+ * The driver implements the simple read/write algorithm. It allocates the
+ * necessary inbound shared memory window by demand from the peer. Then it
+ * sends the physical address of the memory back to the peer. The corresponding
+ * inwndwN and outwndwN files are created at the DebugFS:ntb_mw_test/ntbA_/
+ * directory. The inwndwN file can be used to read the data written by a peer.
+ * The other outwndwN file is used to write data to the peer memory window.
+ */
+
+/* Note: You can load this module with either option 'dyndbg=+p' or define the
+ * next preprocessor constant */
+/*#define DEBUG*/
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/dma-mapping.h>
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+
+#include <linux/ntb.h>
+
+#define DRIVER_NAME "ntb_mw_test"
+#define DRIVER_DESCRIPTION "PCIe NTB Memory Window Test Client"
+#define DRIVER_VERSION "1.0"
+#define CACHE_NAME "ntb_mw_cache"
+
+MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_topdir;
+
+/*
+ * Inbound memory windows (locally allocated) structure
+ * @dma_addr: Address if the locally allocated memory and sent to the peer
+ * @virt_addr: Virtual address of that memory
+ * @size: Size of the allocated memory
+ * @addr_align: Address alignment
+ * @size_align: Size alignment
+ * @size_max: Maximum possible size of the window
+ * @dbgfs_node: DebugFS node to read data from peer
+ * @ctx: Pointer to the driver context
+ */
+struct mw_ctx;
+struct inmw_wrap {
+ dma_addr_t dma_addr;
+ void *virt_addr;
+ resource_size_t size;
+ resource_size_t addr_align;
+ resource_size_t size_align;
+ resource_size_t size_max;
+ struct dentry *dbgfs_node;
+ struct mw_ctx *ctx;
+};
+
+/*
+ * Outbound memory windows (remotely allocated) structure
+ * @enabled: Flag whether the window is enabled
+ * @dma_addr: DMA address of the remotely allocated memory window and
+ * retrieved from the peer
+ * @phys_addr: Physical address of the memory to locally map it (retrieved
+ * from the NTB subsystem, shortly it must be from BAR2 of IDT)
+ * @virt_addr: Virtual address of mapped IOMEM physical address
+ * @size: Size of the peer allocated memory
+ * @addr_align: Alignment of the DMA address allocated by the peer
+ * @size_align: Size alignment of the DMA address allocated by the peer
+ * @size_max: Maximum size of the peer allocated memory
+ * @dbgfs_node: DebugFS node to write data to peer
+ * @ctx: Pointer to the driver context
+ */
+struct outmw_wrap {
+ bool enabled;
+ dma_addr_t dma_addr;
+ phys_addr_t phys_addr;
+ void __iomem *virt_addr;
+ resource_size_t size;
+ resource_size_t addr_align;
+ resource_size_t size_align;
+ resource_size_t size_max;
+ struct dentry *dbgfs_node;
+ struct mw_ctx *ctx;
+};
+
+/*
+ * Doorbells pingpong driver context
+ * @ntb: Pointer to the NTB device
+ * @inmw_cnt: Number of possible inbound memory windows
+ * @outmw_cnt: Number of possible outbound memory windows
+ * @dbgfs_dir: Handler of the DebugFS driver info-file
+ */
+struct mw_ctx {
+ struct ntb_dev *ntb;
+ int inmws_cnt;
+ struct inmw_wrap *inmws;
+ int outmws_cnt;
+ struct outmw_wrap *outmws;
+ struct dentry *dbgfs_dir;
+};
+
+/*
+ * Enumeration of commands
+ * @MW_GETADDRS: Get the addresses of all memory windows peer allocated
+ * @MW_DMAADDR: DMA address of the memory window is sent within this msg
+ * @MW_FREEADDRS: Lock the memory windows shared from the local device
+ * @MW_TYPEMASK: Mask of the message type
+ */
+enum msg_type {
+ MW_GETADDRS,
+ MW_DMAADDR,
+ MW_FREEADDRS,
+ MW_TYPEMASK = 0xFFFFU
+};
+
+/*
+* Helper method to get the type string name
+*/
+static inline char *mw_get_typename(enum msg_type type)
+{
+ switch (type) {
+ case MW_GETADDRS:
+ return "GETADDRS";
+ case MW_DMAADDR:
+ return "DMAADDR";
+ case MW_FREEADDRS:
+ return "FREEADDRS";
+ default:
+ break;
+ }
+
+ return "INVALID";
+}
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_mw(ctx, args...) \
+ dev_err(&ctx->ntb->dev, ## args)
+#define dev_warn_mw(ctx, args...) \
+ dev_warn(&ctx->ntb->dev, ## args)
+#define dev_info_mw(ctx, args...) \
+ dev_info(&ctx->ntb->dev, ## args)
+#define dev_dbg_mw(ctx, args...) \
+ dev_dbg(&ctx->ntb->dev, ## args)
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ * @MIN_MW_CNT: Minimum memory windows count
+ * @MAX_MW_CNT: Maximum memory windows count
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+#define MIN_MW_CNT ((unsigned char)1)
+#define MAX_MW_CNT ((unsigned char)255)
+
+/*
+ * Shared data converter to support the different CPU architectures
+ */
+#define to_sh32(data) \
+ cpu_to_le32((data))
+#define from_sh32(data) \
+ le32_to_cpu((data))
+
+/*
+ * Module parameters:
+ * @inmw_cnt: Number of inbound memory windows [1; 255]
+ * @outmw_cnt: Number of outbound memory windows [1; 255]
+ * If the specified value exceeds the maximum possible valiue, then it is
+ * initialized with maximum one
+ */
+static unsigned char inmws_cnt = MAX_MW_CNT;
+module_param(inmws_cnt, byte, 0000);
+MODULE_PARM_DESC(inmws_cnt,
+ "Inbound memory windows count. Those are the memory windows, which are "
+ "locally allocated. Their address is sent to the remote host."
+ " - Parameter can be set within [1; 255], where 255 means maximum possible"
+ " number of windows");
+
+/*===========================================================================
+ * Helper methods
+ *===========================================================================*/
+
+/*
+ * Alter the passed driver paremeters
+ */
+static void mw_alter_params(struct mw_ctx *ctx)
+{
+ unsigned char inmws_cnt_bak = ctx->inmws_cnt;
+
+ /* Clamp the inbound memory windows parameter */
+ ctx->inmws_cnt = clamp(inmws_cnt,
+ MIN_MW_CNT, (unsigned char)ctx->inmws_cnt);
+ if (inmws_cnt_bak != ctx->inmws_cnt) {
+ dev_warn_mw(ctx,
+ "Inbound memory windows count is altered from "
+ "%hhu to %hhu", inmws_cnt_bak, ctx->inmws_cnt);
+ }
+
+ dev_dbg_mw(ctx, "Memory windows test driver parameter is verified");
+}
+
+/*
+ * Memory block IO write method
+ */
+static void iomem_write(void __iomem *dst, const void *src, size_t cnt)
+{
+ while (cnt--) {
+ iowrite8(*(u8 *)src, dst);
+ dst++;
+ src++;
+ }
+}
+
+/*
+ * Memory block IO read method
+ */
+static void iomem_read(void __iomem *src, void *dst, size_t cnt)
+{
+ while (cnt--) {
+ *(u8 *)dst = ioread8(src);
+ dst++;
+ src++;
+ }
+}
+
+/*===========================================================================
+ * Message command handlers
+ *===========================================================================*/
+
+/*
+ * Send MW_GETADDRS command method
+ */
+static void mw_send_getaddrs_cmd(struct mw_ctx *ctx)
+{
+ struct ntb_msg msg;
+ int sts;
+
+ /* Clear the message structure */
+ memset(&msg, 0, sizeof(msg));
+
+ /* Set the message type only */
+ msg.type = to_sh32(MW_GETADDRS);
+
+ /* Send the message */
+ sts = ntb_msg_post(ctx->ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx, "Failed to send message to get outbound window "
+ "addresses");
+ }
+}
+
+/*
+ * Send MW_FREEADDRS command method
+ */
+static void mw_send_freeaddrs_cmd(struct mw_ctx *ctx)
+{
+ struct ntb_msg msg;
+ int sts;
+
+ /* Clear the message structure */
+ memset(&msg, 0, sizeof(msg));
+
+ /* Set the message type only */
+ msg.type = to_sh32(MW_FREEADDRS);
+
+ /* Send the message */
+ sts = ntb_msg_post(ctx->ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx, "Failed to send a message to disable the peer "
+ "outbound windows");
+ }
+}
+
+/*
+ * Callback method for response on the command MW_GETADDRS
+ */
+static void mw_send_inmw_addrs(struct mw_ctx *ctx)
+{
+ struct ntb_msg msg;
+ struct inmw_wrap *inmw;
+ int mwindx, sts;
+
+ /* Clear the message structure */
+ memset(&msg, 0, sizeof(msg));
+
+ /* Walk through all the inbound memory windows and send the
+ * corresponding DMA address of the window */
+ for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
+ inmw = &ctx->inmws[mwindx];
+ /* Set the type and the memory window index */
+ msg.type = to_sh32(MW_DMAADDR | ((u32)mwindx << 16));
+
+ /* First set the size of the memory window */
+ msg.payload[0] = to_sh32(inmw->size);
+
+ /* Set the Upper part of the memory window address */
+#ifdef CONFIG_64BIT
+ msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
+#else
+ /* WARNING! NTB entpoints must either have the same architecture
+ * (x32 or x64) or use lower 4Gb for memory windows */
+ msg.payload[1] = 0;
+#endif /* !CONFIG_64BIT */
+ /* Set the Lower part of the memory window address */
+ msg.payload[2] = to_sh32((u32)(inmw->dma_addr));
+
+ /* Send the message */
+ sts = ntb_msg_post(ctx->ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx,
+ "Failed to send a message with window %d "
+ "address", mwindx);
+ }
+ }
+}
+
+/*
+ * Method to set the corresponding memory window and enable it
+ */
+static void mw_set_outmw_addr(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct outmw_wrap *outmw;
+ int mwindx, sts;
+
+ /* Read the memory windows index (it's the part of the message type) */
+ mwindx = from_sh32(msg->type) >> 16;
+ if (ctx->outmws_cnt <= mwindx) {
+ dev_err_mw(ctx,
+ "Retrieved invalid outbound memory window index %d",
+ mwindx);
+ return;
+ }
+ outmw = &ctx->outmws[mwindx];
+
+ /* Read the memory window size and check whether it has proper size and
+ * alignment */
+ outmw->size = from_sh32(msg->payload[0]);
+ if (!IS_ALIGNED(outmw->size, outmw->size_align) ||
+ outmw->size_max < outmw->size) {
+ dev_err_mw(ctx,
+ "Retrieved invalid memory window %d size %u "
+ "(max: %u, align: %u)", mwindx, (unsigned int)outmw->size,
+ (unsigned int)outmw->size_max,
+ (unsigned int)outmw->size_align);
+ return;
+ }
+
+ /* Read the DMA address, where the second DWORD is the upper part and
+ * the third DWORD - lower */
+ outmw->dma_addr = from_sh32(msg->payload[2]);
+#ifdef CONFIG_64BIT
+ outmw->dma_addr |= ((dma_addr_t)from_sh32(msg->payload[1]) << 32);
+#endif /* CONFIG_64BIT */
+ /* Check whether the retrieved address is properly aligned */
+ if (!IS_ALIGNED(outmw->dma_addr, outmw->addr_align)) {
+ dev_err_mw(ctx,
+ "Outbound memory window address %p is not aligned "
+ "within %u bytes", (void *)outmw->dma_addr,
+ (unsigned int)outmw->addr_align);
+ return;
+ }
+
+ /* Set the translation address of the outbound memory window */
+ sts = ntb_mw_set_trans(ctx->ntb, mwindx, outmw->dma_addr, outmw->size);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx, "Failed to set the translated address %p of "
+ "outbound memory window %d", (void *)outmw->dma_addr,
+ mwindx);
+ return;
+ }
+
+ /* Enable the memory window */
+ outmw->enabled = true;
+
+ dev_dbg_mw(ctx, "Outbound memory window %d is initialized with "
+ "address 0x%p", mwindx, (void *)outmw->dma_addr);
+}
+
+/*
+ * Lock all the outbound memory windows
+ */
+static void mw_lock_outmw_addrs(struct mw_ctx *ctx)
+{
+ int mwindx;
+
+ /* Walk through all the memory windows and lock whem by falsing
+ * the flag */
+ for (mwindx = 0; mwindx < ctx->outmws_cnt; mwindx++) {
+ ctx->outmws[mwindx].enabled = false;
+ }
+
+ dev_dbg_mw(ctx, "Outbound memory windows are locked");
+}
+
+/*===========================================================================
+ * Messages and link events handlers
+ *===========================================================================*/
+
+/*
+ * Handle the retrieved message
+ */
+static void msg_recv_handler(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ enum msg_type type = from_sh32(msg->type) & MW_TYPEMASK;
+
+ /* Check the message types */
+ switch (type) {
+ case MW_GETADDRS:
+ mw_send_inmw_addrs(ctx);
+ break;
+ case MW_DMAADDR:
+ mw_set_outmw_addr(ctx, msg);
+ break;
+ case MW_FREEADDRS:
+ mw_lock_outmw_addrs(ctx);
+ break;
+ default:
+ dev_err_mw(ctx, "Invalid message type retrieved %d", type);
+ return;
+ }
+
+ dev_dbg_mw(ctx, "Message of type %s was received",
+ mw_get_typename(type));
+}
+
+/*
+ * Handler of the transmit errors
+ */
+static void msg_fail_handler(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ enum msg_type type = from_sh32(msg->type) & MW_TYPEMASK;
+
+ /* Just print the error */
+ dev_err_mw(ctx, "Failed to send the message of type %s",
+ mw_get_typename(type));
+}
+
+/*
+ * Handler of the succeeded transmits
+ */
+static void msg_sent_handler(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ enum msg_type type = from_sh32(msg->type) & MW_TYPEMASK;
+
+ /* Just print the debug text and increment the succeeded msgs counter */
+ dev_dbg_mw(ctx, "Message of type %s has been successfully sent",
+ mw_get_typename(type));
+}
+
+/*
+ * Message event handler
+ */
+static void msg_event_handler(void *data, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg)
+{
+ struct mw_ctx *ctx = data;
+
+ /* Call the corresponding event handler */
+ switch (ev) {
+ case NTB_MSG_NEW:
+ msg_recv_handler(ctx, msg);
+ break;
+ case NTB_MSG_SENT:
+ msg_sent_handler(ctx, msg);
+ break;
+ case NTB_MSG_FAIL:
+ msg_fail_handler(ctx, msg);
+ break;
+ default:
+ dev_err_mw(ctx, "Got invalid message event %d", ev);
+ break;
+ }
+}
+
+/*
+ * Link Up/Down event handler
+ */
+static void link_event_handler(void *data)
+{
+ struct mw_ctx *ctx = data;
+ int sts;
+
+ /* If link is up then send the message with GETADDRS command, otherwise
+ * the outbound memory windows must be disabled */
+ sts = ntb_link_is_up(ctx->ntb, NULL, NULL);
+ if (ON == sts) {
+ mw_send_getaddrs_cmd(ctx);
+ } else /* if (OFF == sts) */ {
+ mw_lock_outmw_addrs(ctx);
+ }
+
+ dev_dbg_mw(ctx, "Link %s event was retrieved",
+ ON == sts ? "Up" : "Down");
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t mw_dbgfs_outmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_outmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_outmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_inmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_inmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_inmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+/*
+ * DebugFS outbound memory window node operations
+ */
+static const struct file_operations mw_dbgfs_outmw_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_outmw_read,
+ .write = mw_dbgfs_outmw_write
+};
+
+/*
+ * DebugFS outbound memory window configuration node operations
+ */
+static const struct file_operations mw_dbgfs_outmw_cfg_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_outmw_cfg_read,
+};
+
+/*
+ * DebugFS inbound memory window node operations
+ */
+static const struct file_operations mw_dbgfs_inmw_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_inmw_read,
+ .write = mw_dbgfs_inmw_write
+};
+
+/*
+ * DebugFS inbound memory window configuration node operations
+ */
+static const struct file_operations mw_dbgfs_inmw_cfg_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_inmw_cfg_read,
+};
+
+/*
+ * DebugFS read callback of outbound memory window node
+ */
+static ssize_t mw_dbgfs_outmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct outmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+ int sts;
+
+ /* Check whether the link is up and the outbound window is enabled */
+ sts = ntb_link_is_up(ctx->ntb, NULL, NULL);
+ if (OFF == sts || !wrap->enabled) {
+ dev_err_mw(ctx, "NTB link is %s, memory window status is %s",
+ OFF == sts ? "Down" : "Up",
+ wrap->enabled ? "enabled" : "disabled");
+ return -ENODEV;
+ }
+
+ /* Read the first DWORD with the size of the message */
+ datasize = readl(wrap->virt_addr);
+
+ /* Check the read data size */
+ if (wrap->size < datasize) {
+ dev_err_mw(ctx, "Data size %u exceeds the memory window size %u",
+ (unsigned int)datasize, (unsigned int)wrap->size);
+ return -EINVAL;
+ }
+
+ /* Calculate the size of the output buffer */
+ datasize = min(datasize, usize);
+
+ /* If there is nothing to copy then just return from the function */
+ if (0 == datasize) {
+ return 0;
+ }
+
+ /* Allocate the buffer */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the output buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data from the shared memory to the temporary buffer */
+ /* NOTE memcpy_toio could be used instead, but it weirdly works so
+ * the traditional looping is used */
+ iomem_read((wrap->virt_addr + 4), databuf, datasize);
+ /*memcpy_fromio(databuf, wrap->virt_addr + 4, datasize);*/
+
+ /* Copy the data to the output buffer */
+ ret = simple_read_from_buffer(ubuf, usize, offp, databuf, datasize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS write callback of outbound memory window node
+ */
+static ssize_t mw_dbgfs_outmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct outmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+ int sts;
+
+ /* Check whether the link is up and the outbound window is enabled */
+ sts = ntb_link_is_up(ctx->ntb, NULL, NULL);
+ if (OFF == sts || !wrap->enabled) {
+ dev_err_mw(ctx, "NTB link is %s, memory window status is %s",
+ OFF == sts ? "Down" : "Up",
+ wrap->enabled ? "enabled" : "disabled");
+ return -ENODEV;
+ }
+
+ /* Calculate the data size */
+ datasize = min(((size_t)wrap->size - 4), usize);
+
+ /* Allocate the memory for sending data */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the input data buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data to the output buffer */
+ ret = simple_write_to_buffer(databuf, datasize, offp, ubuf, usize);
+ if (0 > ret) {
+ dev_err_mw(ctx, "Failed to copy the data from the User-space");
+ kfree(databuf);
+ return ret;
+ }
+
+ /* First DWORD is the data size */
+ writel((u32)datasize, wrap->virt_addr);
+
+ /* Copy the data to the memory window */
+ /* NOTE memcpy_toio could be used instead, but it weirdly works so
+ * the traditional looping is used */
+ iomem_write((wrap->virt_addr + 4), databuf, datasize);
+ /*memcpy_toio((wrap->virt_addr + 4), databuf, datasize);*/
+
+ /* Ensure that the data is fully copied out by setting the memory
+ * barrier */
+ wmb();
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS read callback of outbound memory window configurations
+ */
+static ssize_t mw_dbgfs_outmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct outmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+ int id;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, usize, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ dev_dbg_mw(ctx,
+ "Failed to allocated the memory for outbound memory "
+ "window configuration");
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Outbound Memory Window configuration:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "Status\t\t\t- %s\n",
+ wrap->enabled ? "enabled" : "disabled");
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address\t\t- 0x%p\n", (void *)wrap->dma_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address alignment\t- %lu\n",
+ (unsigned long int)wrap->addr_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Physycal map address\t- 0x%p\n", (void *)wrap->phys_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "Virtual map address\t- 0x%p\n", (void *)wrap->virt_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "Size of the window\t- %lu\n",
+ (unsigned long int)wrap->size);
+ off += scnprintf(strbuf + off, size - off,
+ "Size alignment\t\t- %lu\n",
+ (unsigned long int)wrap->size_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Maximum size\t\t- %lu\n",
+ (unsigned long int)wrap->size_max);
+ /* Print raw data from the inbound window */
+ off += scnprintf(strbuf + off, size - off,
+ "Raw data (16 bytes)\t- ");
+ for (id = 0; id < 16; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "%02hhx ", ioread8(wrap->virt_addr + id));
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, usize, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS read callback of inbound memory window node
+ */
+static ssize_t mw_dbgfs_inmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct inmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+
+ /* Read the first DWORD with the size of the data */
+ datasize = le32_to_cpu(*(u32 *)wrap->virt_addr);
+
+ /* Calculate the size of the output buffer */
+ datasize = min(datasize, usize);
+
+ /* If there is nothing to copy then just return from the function */
+ if (0 == datasize) {
+ return 0;
+ }
+
+ /* Allocate the buffer */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the output buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data from the shared memory to the temporary buffer */
+ memcpy(databuf, wrap->virt_addr + 4, datasize);
+
+ /* Copy the data to the output buffer */
+ ret = simple_read_from_buffer(ubuf, usize, offp, databuf, datasize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS write callback of inbound memory window node
+ */
+static ssize_t mw_dbgfs_inmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct inmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+
+ /* Calculate the data size */
+ datasize = min((size_t)wrap->size - 4, usize);
+
+ /* Allocate the memory for sending data */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the input data buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data to the output buffer */
+ ret = simple_write_to_buffer(databuf, usize, offp, ubuf, usize);
+ if (0 > ret) {
+ dev_err_mw(ctx, "Failed to copy the data from the User-space");
+ kfree(databuf);
+ return ret;
+ }
+
+ /* First DWORD is the data size */
+ *(u32 *)wrap->virt_addr = cpu_to_le32(datasize);
+
+ /* Copy the data to the memory window */
+ memcpy(wrap->virt_addr + 4, databuf, datasize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return datasize;
+}
+
+/*
+ * DebugFS read callback of outbound memory window configurations
+ */
+static ssize_t mw_dbgfs_inmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct inmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+ int id;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, usize, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ dev_dbg_mw(ctx,
+ "Failed to allocated the memory for inbound memory "
+ "window configuration");
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Inbound Memory Window configuration:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address\t\t- 0x%p\n", (void *)wrap->dma_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address alignment\t- %lu\n",
+ (unsigned long int)wrap->addr_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Virtual address\t\t- 0x%p\n", (void *)wrap->virt_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "Size of the window\t- %lu\n",
+ (unsigned long int)wrap->size);
+ off += scnprintf(strbuf + off, size - off,
+ "Size alignment\t\t- %lu\n",
+ (unsigned long int)wrap->size_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Maximum size\t\t- %lu\n",
+ (unsigned long int)wrap->size_max);
+
+ /* Print raw data from the inbound window */
+ off += scnprintf(strbuf + off, size - off,
+ "Raw data (16 bytes)\t- ");
+ for (id = 0; id < 16; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "%02hhx ", *(char *)(wrap->virt_addr + id));
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, usize, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS initialization function
+ */
+#define NAMESIZE 16
+static int mw_init_dbgfs(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct dentry *dbgfs_node;
+ char nodename[NAMESIZE];
+ const char *devname;
+ int outmwindx, inmwindx, ret;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_topdir)) {
+ dev_warn_mw(ctx,
+ "Top DebugFS directory has not been created for "
+ DRIVER_NAME);
+ return PTR_ERR(dbgfs_topdir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(&ntb->dev);
+
+ /* Create the device related subdirectory */
+ ctx->dbgfs_dir = debugfs_create_dir(devname, dbgfs_topdir);
+ if (IS_ERR_OR_NULL(ctx->dbgfs_dir)) {
+ dev_warn_mw(ctx,
+ "Failed to create the DebugFS subdirectory %s",
+ devname);
+ return PTR_ERR(ctx->dbgfs_dir);
+ }
+
+ /* Walk through all the outbound memory windows creating the
+ * corresponding nodes */
+ for (outmwindx = 0; outmwindx < ctx->outmws_cnt; outmwindx++) {
+ /* Create the name of the read/write node */
+ snprintf(nodename, NAMESIZE, "outmw%d", outmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->outmws[outmwindx],
+ &mw_dbgfs_outmw_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+
+ /* Create the name of the configuration node */
+ snprintf(nodename, NAMESIZE, "outmwcfg%d", outmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->outmws[outmwindx],
+ &mw_dbgfs_outmw_cfg_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+ }
+
+ /* Walk through all the inbound memory windows creating the
+ * corresponding nodes */
+ for (inmwindx = 0; inmwindx < ctx->inmws_cnt; inmwindx++) {
+ /* Create the name of the read/write node */
+ snprintf(nodename, NAMESIZE, "inmw%d", inmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->inmws[inmwindx],
+ &mw_dbgfs_inmw_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+
+ /* Create the name of the configuration node */
+ snprintf(nodename, NAMESIZE, "inmwcfg%d", inmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->inmws[inmwindx],
+ &mw_dbgfs_inmw_cfg_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+ }
+
+ dev_dbg_mw(ctx, "NTB Memory Windows DebugFS top diretory is created "
+ "for %s", devname);
+
+ return SUCCESS;
+
+err_rm_dir:
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ return ret;
+}
+
+/*
+ * DebugFS deinitialization function
+ */
+static void mw_deinit_dbgfs(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Remove the DebugFS directory */
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ dev_dbg_mw(ctx, "Memory Windows DebugFS nodes %s/ are discarded",
+ dev_name(&ntb->dev));
+}
+
+/*===========================================================================
+ * NTB device/client driver initialization
+ *===========================================================================*/
+
+/*
+ * NTB device events handlers
+ */
+static const struct ntb_ctx_ops mw_ops = {
+ .link_event = link_event_handler,
+ .msg_event = msg_event_handler
+};
+
+/*
+ * Create the outbound memory windows
+ */
+static int mw_create_outmws(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct outmw_wrap *outmw;
+ int ret, mwindx;
+
+ /* Loop over all the outbound memory window descriptors initializing the
+ * corresponding fields */
+ for (mwindx = 0; mwindx < ctx->outmws_cnt; mwindx++) {
+ outmw = &ctx->outmws[mwindx];
+ /* Outbound memory windows are disabled by default */
+ outmw->enabled = false;
+
+ /* Set the context */
+ outmw->ctx = ctx;
+
+ /* Retrieve the physical address of the memory to map */
+ ret = ntb_mw_get_maprsc(ntb, mwindx, &outmw->phys_addr,
+ &outmw->size);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to get map resources of "
+ "outbound window %d", mwindx);
+ mwindx--;
+ goto err_unmap_rsc;
+ }
+
+ /* Map the memory window resources */
+ outmw->virt_addr = ioremap_nocache(outmw->phys_addr, outmw->size);
+
+ /* Retrieve the memory windows maximum size and alignments */
+ ret = ntb_mw_get_align(ntb, mwindx, &outmw->addr_align,
+ &outmw->size_align, &outmw->size_max);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to get alignment options of "
+ "outbound window %d", mwindx);
+ goto err_unmap_rsc;
+ }
+ }
+
+ dev_dbg_mw(ctx, "Outbound memory windows are created");
+
+ return SUCCESS;
+
+err_unmap_rsc:
+ for (; 0 <= mwindx; mwindx--) {
+ outmw = &ctx->outmws[mwindx];
+ iounmap(outmw->virt_addr);
+ }
+
+ return ret;
+}
+
+/*
+ * Free the outbound memory windows
+ */
+static void mw_free_outmws(struct mw_ctx *ctx)
+{
+ struct outmw_wrap *outmw;
+ int mwindx;
+
+ /* Loop over all the outbound memory window descriptors and unmap the
+ * resources */
+ for (mwindx = 0; mwindx < ctx->outmws_cnt; mwindx++) {
+ outmw = &ctx->outmws[mwindx];
+
+ /* Disable the memory window */
+ outmw->enabled = false;
+
+ /* Unmap the resource */
+ iounmap(outmw->virt_addr);
+ }
+
+ dev_dbg_mw(ctx, "Outbound memory windows are freed");
+}
+
+/*
+ * Create the inbound memory windows
+ */
+static int mw_create_inmws(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct inmw_wrap *inmw;
+ int mwindx, ret = SUCCESS;
+
+ /* Loop over all the inbound memory window descriptors initializing the
+ * corresponding fields */
+ for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
+ inmw = &ctx->inmws[mwindx];
+ /* Set the context */
+ inmw->ctx = ctx;
+ /* Retrieve the memory windows maximum size and alignments */
+ ret = ntb_peer_mw_get_align(ntb, mwindx, &inmw->addr_align,
+ &inmw->size_align, &inmw->size_max);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to get alignment options of "
+ "inbound window %d", mwindx);
+ mwindx--;
+ ret = -ENOMEM;
+ goto err_free_dma_bufs;
+ }
+ /* Allocate all the maximum possible size */
+ inmw->size = inmw->size_max;
+
+ /* Allocate the cache coherent DMA memory windows */
+ inmw->virt_addr = dma_zalloc_coherent(&ntb->dev, inmw->size,
+ &inmw->dma_addr, GFP_KERNEL);
+ if (IS_ERR_OR_NULL(inmw->virt_addr)) {
+ dev_err_mw(ctx,
+ "Failed to allocate the inbound buffer for %d",
+ mwindx);
+ mwindx--;
+ ret = -ENOMEM;
+ goto err_free_dma_bufs;
+ }
+ /* Make sure the allocated address is properly aligned */
+ if (!IS_ALIGNED(inmw->dma_addr, inmw->addr_align)) {
+ dev_err_mw(ctx, "DMA address %p of inbound mw %d isn't "
+ "aligned with %lu", (void *)inmw->dma_addr, mwindx,
+ (unsigned long int)inmw->addr_align);
+ ret = -EINVAL;
+ goto err_free_dma_bufs;
+ }
+ }
+
+ dev_dbg_mw(ctx, "Inbound memory windows are created");
+
+ return SUCCESS;
+
+err_free_dma_bufs:
+ for (; 0 <= mwindx; mwindx--) {
+ inmw = &ctx->inmws[mwindx];
+ dma_free_coherent(&ntb->dev, inmw->size, inmw->virt_addr,
+ inmw->dma_addr);
+ }
+
+ return ret;
+}
+
+/*
+ * Free the inbound memory windows
+ */
+static void mw_free_inmws(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct inmw_wrap *inmw;
+ int mwindx;
+
+ /* Loop over all the inbound memory window descriptors and free
+ * the allocated memory */
+ for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
+ inmw = &ctx->inmws[mwindx];
+
+ /* Free the cache coherent DMA memory window */
+ dma_free_coherent(&ntb->dev, inmw->size, inmw->virt_addr,
+ inmw->dma_addr);
+ }
+
+ dev_dbg_mw(ctx, "Inbound memory windows are freed");
+}
+
+/*
+ * Create the driver context structure
+ */
+static struct mw_ctx *mw_create_ctx(struct ntb_dev *ntb)
+{
+ struct mw_ctx *ctx, *ret;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&ntb->dev);
+ ctx = kzalloc_node(sizeof(*ctx), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx)) {
+ dev_err(&ntb->dev,
+ "No memory for NTB Memory windows driver context");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Initialize the context NTB device pointer */
+ ctx->ntb = ntb;
+
+ /* Read the number of memory windows */
+ /* Number of memory windows local NTB device can set to the translated
+ * address register */
+ ctx->outmws_cnt = ntb_mw_count(ntb);
+ /* Number of memory windows peer can set to his translated address
+ * register */
+ ctx->inmws_cnt = ntb_peer_mw_count(ntb);
+
+ /* Alter the inbound memory windows count with respect to the driver
+ * parameter */
+ mw_alter_params(ctx);
+
+ /* Allocate the memory for memory window descriptors */
+ ctx->outmws = kzalloc_node(ctx->outmws_cnt * sizeof(*ctx->outmws),
+ GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx->outmws)) {
+ dev_err_mw(ctx,
+ "Failed to allocate memory for outbound MW descriptors");
+ ret = ERR_PTR(-ENOMEM);
+ goto err_free_ctx;
+ }
+ ctx->inmws = kzalloc_node(ctx->inmws_cnt * sizeof(*ctx->inmws),
+ GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx->inmws)) {
+ dev_err_mw(ctx,
+ "Failed to allocate memory for inbound MW descriptors");
+ ret = ERR_PTR(-ENOMEM);
+ goto err_free_outmws;
+ }
+
+ dev_dbg_mw(ctx, "Context structure is created");
+
+ return ctx;
+
+/*err_free_inmws:
+ kfree(ctx->inmws);
+*/
+err_free_outmws:
+ kfree(ctx->outmws);
+err_free_ctx:
+ kfree(ctx);
+
+ return ret;
+}
+
+/*
+ * Free the driver context structure
+ */
+static void mw_free_ctx(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Free the outbound memory windows descriptors */
+ kfree(ctx->outmws);
+
+ /* Free the inbound memory windows descriptors */
+ kfree(ctx->inmws);
+
+ /* Free the memory allocated for the context structure */
+ kfree(ctx);
+
+ dev_dbg(&ntb->dev, "Context structure is freed");
+}
+
+/*
+ * Initialize the ntb device structure
+ */
+static int mw_init_ntb_dev(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ int ret;
+
+ /* Set the NTB device events context */
+ ret = ntb_set_ctx(ntb, ctx, &mw_ops);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to specify the NTB device context");
+ return ret;
+ }
+
+ /* Enable the link and rise the event to check the link state */
+ ntb_link_enable(ntb, NTB_SPEED_AUTO, NTB_WIDTH_AUTO);
+ /*ntb_link_event(ntb);*/
+
+ dev_dbg_mw(ctx, "NTB device is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the ntb device structure
+ */
+static void mw_stop_ntb_dev(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Clear the context */
+ ntb_clear_ctx(ntb);
+
+ /* Disable the link */
+ ntb_link_disable(ntb);
+
+ dev_dbg_mw(ctx, "NTB device is deinitialized");
+}
+
+/*
+ * Initialize the DMA masks
+ */
+static int __maybe_unused mw_ntb_set_dma_mask(struct ntb_dev *ntb)
+{
+ struct device *dev;
+ int ret = SUCCESS;
+
+ /* Get the NTB device structure */
+ dev = &ntb->dev;
+ /* Try to set the Highmem DMA address */
+ ret = dma_set_mask(dev, DMA_BIT_MASK(64));
+ if (SUCCESS == ret) {
+ /* Next call won't fail of the upper one returned OK */
+ dma_set_coherent_mask(dev, DMA_BIT_MASK(64));
+ return SUCCESS;
+ }
+
+ /* Warn if the HIGHMEM can be used for DMA */
+ dev_warn(dev, "Cannot set the NTB device DMA highmem mask");
+
+ /* Try the Low 32-bits DMA addresses */
+ ret = dma_set_mask(dev, DMA_BIT_MASK(32));
+ if (SUCCESS == ret) {
+ /* The same is here */
+ dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
+ return SUCCESS;
+ }
+
+ dev_err(dev, "Failed to set the NTB device DMA lowmem mask");
+
+ return ret;
+}
+
+/*
+ * NTB device probe() callback function
+ */
+static int mw_probe(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct mw_ctx *ctx;
+ int ret;
+
+ /* Only asynchronous hardware is supported */
+ if (!ntb_valid_async_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
+ /* Check whether the messaging supports at least 4 DWORDS */
+ ret = ntb_msg_size(ntb);
+ if (4 > ret) {
+ dev_err(&ntb->dev, "NTB Messaging supports just %d < 4 dwords",
+ ret);
+ return -EINVAL;
+ }
+
+ /* Set the NTB device DMA mask */
+ /*ret = mw_ntb_set_dma_mask(ntb);
+ if (SUCCESS != ret) {
+ return ret;
+ }*/
+
+ /* Create the current device context */
+ ctx = mw_create_ctx(ntb);
+ if (IS_ERR_OR_NULL(ctx)) {
+ return PTR_ERR(ctx);
+ }
+
+ /* Allocate the inbound memory windows */
+ ret = mw_create_inmws(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_ctx;
+ }
+
+ /* Map the outbound memory windows */
+ ret = mw_create_outmws(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_inmws;
+ }
+
+ /* Initialize the NTB device */
+ ret = mw_init_ntb_dev(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_outmws;
+ }
+
+ /* Create the DebugFS nodes */
+ (void)mw_init_dbgfs(ctx);
+
+ return SUCCESS;
+
+/*err_stop_ntb_dev:
+ mw_stop_ntb_dev(ctx);
+*/
+err_free_outmws:
+ mw_free_outmws(ctx);
+err_free_inmws:
+ mw_free_inmws(ctx);
+err_free_ctx:
+ mw_free_ctx(ctx);
+
+ return ret;
+
+}
+
+/*
+ * NTB device remove() callback function
+ */
+static void mw_remove(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct mw_ctx *ctx = ntb->ctx;
+
+ /* Send the message so the peer would lock outbound memory windows */
+ mw_send_freeaddrs_cmd(ctx);
+
+ /* Remove the DebugFS node */
+ mw_deinit_dbgfs(ctx);
+
+ /* Disable the NTB device link and clear the context */
+ mw_stop_ntb_dev(ctx);
+
+ /* Clear the outbound memory windows */
+ mw_free_outmws(ctx);
+
+ /* Clear the inbound memory windows */
+ mw_free_inmws(ctx);
+
+ /* Free the allocated context */
+ mw_free_ctx(ctx);
+}
+
+/*
+ * NTB bus client driver structure definition
+ */
+static struct ntb_client mw_client = {
+ .ops = {
+ .probe = mw_probe,
+ .remove = mw_remove,
+ },
+};
+/* module_ntb_client(mw_client); */
+
+/*
+ * Driver initialize method
+ */
+static int __init ntb_mw_init(void)
+{
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_topdir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Registers the client driver */
+ return ntb_register_client(&mw_client);
+}
+module_init(ntb_mw_init);
+
+/*
+ * Driver exit method
+ */
+static void __exit ntb_mw_exit(void)
+{
+ /* Unregister the client driver */
+ ntb_unregister_client(&mw_client);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_topdir);
+}
+module_exit(ntb_mw_exit);
+
--
2.6.6
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver
2016-07-26 19:50 ` [PATCH 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
@ 2016-07-27 16:29 ` kbuild test robot
0 siblings, 0 replies; 11+ messages in thread
From: kbuild test robot @ 2016-07-27 16:29 UTC (permalink / raw)
To: Serge Semin
Cc: kbuild-all, jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu,
Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
[-- Attachment #1: Type: text/plain, Size: 2460 bytes --]
Hi,
[auto build test WARNING on ntb/ntb-next]
[cannot apply to v4.7 next-20160727]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Serge-Semin/ntb-Asynchronous-NTB-devices-support/20160727-082033
base: https://github.com/jonmason/ntb ntb-next
config: sparc64-allmodconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 5.4.0-6) 5.4.0 20160609
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=sparc64
All warnings (new ones prefixed by >>):
drivers/ntb/hw/idt/ntb_hw_idt.c: In function 'idt_ntb_setmw':
>> drivers/ntb/hw/idt/ntb_hw_idt.c:1170:56: warning: right shift count >= width of type [-Wshift-count-overflow]
idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)(addr >> 32));
^
vim +1170 drivers/ntb/hw/idt/ntb_hw_idt.c
1154
1155 /* Collect the Lookup table offset */
1156 idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_INDEX,
1157 ndev->mw_self_offset + mwindx);
1158 idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_BAR, BAR2);
1159
1160 /* Collect the Lookup table entry partition and valid bits */
1161 idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_PART, ndev->part);
1162 idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_VALID, ON);
1163
1164 /* Start critical section writing to the local port Lookup table */
1165 spin_lock_irqsave(&pdata->lut_lock, irqflags);
1166 /* Write the data to the Lookup table registers of the peer */
1167 idt_ntb_writereg(cfg, IDT_NT_PCI_LUTOFFSET, lut_indxbar);
1168 idt_ntb_writereg(cfg, IDT_NT_PCI_LUTLDATA, (u32)addr);
1169 #ifdef CONFIG_64BIT
> 1170 idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)(addr >> 32));
1171 #else
1172 idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)0);
1173 #endif /* !CONFIG_64BIT */
1174 idt_ntb_writereg(cfg, IDT_NT_PCI_LUTUDATA, lut_partval);
1175 /* Finally unlock the Lookup table */
1176 spin_unlock_irqrestore(&pdata->lut_lock, irqflags);
1177
1178 return SUCCESS;
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 46369 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 0/3] ntb: Asynchronous NTB devices support
2016-07-26 19:50 [PATCH 0/3] ntb: Asynchronous NTB devices support Serge Semin
` (2 preceding siblings ...)
2016-07-26 19:50 ` [PATCH 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
@ 2016-07-28 10:01 ` Serge Semin
2016-07-28 10:01 ` [PATCH v2 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
` (2 more replies)
3 siblings, 3 replies; 11+ messages in thread
From: Serge Semin @ 2016-07-28 10:01 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
Please, find the general patchset description in the cover letter of the first
patchset (see the very first message in thread).
Changes in v2:
- Fix sparc64 compilation warning in drivers/ntb/hw/idt/ntb_hw_idt.c :
warning: right shift count >= width of type
- Fix sparc64 compilation warnings in drivers/ntb/test/ntb_mw_test.c :
warning: right shift count >= width of type
warning: cast to pointer from integer of different size
Thanks,
=============================
Serge V. Semin
Leading Programmer
Embedded SW development group
T-platforms
=============================
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
fancer (3):
ntb: Add asynchronous devices support to NTB-bus interface
ntb: IDT 89HPES*NT* PCIe-switches NTB device driver
ntb: Test client drivers for asynchronous NTB devices
drivers/ntb/Kconfig | 4 +-
drivers/ntb/hw/Kconfig | 1 +
drivers/ntb/hw/Makefile | 6 +-
drivers/ntb/hw/amd/ntb_hw_amd.c | 49 +-
drivers/ntb/hw/idt/Kconfig | 21 +
drivers/ntb/hw/idt/Makefile | 5 +
drivers/ntb/hw/idt/ntb_hw_idt.c | 4050 ++++++++++++++++++++++++++++++++
drivers/ntb/hw/idt/ntb_hw_idt.h | 390 +++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.c | 163 ++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.h | 114 +
drivers/ntb/hw/idt/ntb_hw_idt_regmap.h | 877 +++++++
drivers/ntb/hw/intel/ntb_hw_intel.c | 59 +-
drivers/ntb/ntb.c | 86 +-
drivers/ntb/ntb_transport.c | 19 +-
drivers/ntb/test/Kconfig | 32 +
drivers/ntb/test/Makefile | 9 +-
drivers/ntb/test/ntb_db_test.c | 677 ++++++
drivers/ntb/test/ntb_msg_test.c | 736 ++++++
drivers/ntb/test/ntb_mw_test.c | 1539 ++++++++++++
drivers/ntb/test/ntb_perf.c | 16 +-
drivers/ntb/test/ntb_pingpong.c | 5 +
drivers/ntb/test/ntb_tool.c | 25 +-
include/linux/ntb.h | 600 ++++-
23 files changed, 9317 insertions(+), 166 deletions(-)
create mode 100644 drivers/ntb/hw/idt/Kconfig
create mode 100644 drivers/ntb/hw/idt/Makefile
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
create mode 100644 drivers/ntb/test/ntb_db_test.c
create mode 100644 drivers/ntb/test/ntb_msg_test.c
create mode 100644 drivers/ntb/test/ntb_mw_test.c
--
2.6.6
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 1/3] ntb: Add asynchronous devices support to NTB-bus interface
2016-07-28 10:01 ` [PATCH v2 0/3] ntb: Asynchronous NTB devices support Serge Semin
@ 2016-07-28 10:01 ` Serge Semin
2016-07-28 10:01 ` [PATCH v2 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
2016-07-28 10:01 ` [PATCH v2 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
2 siblings, 0 replies; 11+ messages in thread
From: Serge Semin @ 2016-07-28 10:01 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
Currently supported AMD and Intel Non-transparent PCIe-bridges are synchronous
devices, so translated base address of memory windows can be direcly written
to peer registers. But there are some IDT PCIe-switches which implement
complex interfaces using Lookup Tables of translation addresses. Due to
the way the table is accessed, it can not be done synchronously from different
RCs, that's why the asynchronous interface should be developed.
For these purpose the Memory Window related interface is correspondingly split
as it is for Doorbell and Scratchpad registers. The definition of Memory Window
is following: "It is a virtual memory region, which locally reflects a physical
memory of peer device." So to speak the "ntb_peer_mw_"-prefixed methods control
the peers memory windows, "ntb_mw_"-prefixed functions work with the local
memory windows.
Here is the description of the Memory Window related NTB-bus callback
functions:
- ntb_mw_count() - number of local memory windows.
- ntb_mw_get_maprsc() - get the physical address and size of the local memory
window to map.
- ntb_mw_set_trans() - set translation address of local memory window (this
address should be somehow retrieved from a peer).
- ntb_mw_get_trans() - get translation address of local memory window.
- ntb_mw_get_align() - get alignment of translated base address and size of
local memory window. Additionally one can get the
upper size limit of the memory window.
- ntb_peer_mw_count() - number of peer memory windows (it can differ from the
local number).
- ntb_peer_mw_set_trans() - set translation address of peer memory window
- ntb_peer_mw_get_trans() - get translation address of peer memory window
- ntb_peer_mw_get_align() - get alignment of translated base address and size
of peer memory window.Additionally one can get the
upper size limit of the memory window.
As one can see current AMD and Intel NTB drivers mostly implement the
"ntb_peer_mw_"-prefixed methods. So this patch correspondingly renames the
driver functions. IDT NTB driver mostly expose "ntb_nw_"-prefixed methods,
since it doesn't have convenient access to the peer Lookup Table.
In order to pass information from one RC to another NTB functions of IDT
PCIe-switch implement Messaging subsystem. They currently support four message
registers to transfer DWORD sized data to a specified peer. So there are two
new callback methods are introduced:
- ntb_msg_size() - get the number of DWORDs supported by NTB function to send
and receive messages
- ntb_msg_post() - send message of size retrieved from ntb_msg_size()
to a peer
Additionally there is a new event function:
- ntb_msg_event() - it is invoked when either a new message was retrieved
(NTB_MSG_NEW), or last message was successfully sent
(NTB_MSG_SENT), or the last message failed to be sent
(NTB_MSG_FAIL).
The last change concerns the IDs (practically names) of NTB-devices on the
NTB-bus. It is not good to have the devices with same names in the system
and it brakes my IDT NTB driver from being loaded =) So I developed a simple
algorithm of NTB devices naming. Particulary it generates names "ntbS{N}" for
synchronous devices, "ntbA{N}" for asynchronous devices, and "ntbAS{N}" for
devices supporting both interfaces.
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
drivers/ntb/Kconfig | 4 +-
drivers/ntb/hw/amd/ntb_hw_amd.c | 49 ++-
drivers/ntb/hw/intel/ntb_hw_intel.c | 59 +++-
drivers/ntb/ntb.c | 86 +++++-
drivers/ntb/ntb_transport.c | 19 +-
drivers/ntb/test/ntb_perf.c | 16 +-
drivers/ntb/test/ntb_pingpong.c | 5 +
drivers/ntb/test/ntb_tool.c | 25 +-
include/linux/ntb.h | 600 +++++++++++++++++++++++++++++-------
9 files changed, 701 insertions(+), 162 deletions(-)
diff --git a/drivers/ntb/Kconfig b/drivers/ntb/Kconfig
index 95944e5..67d80c4 100644
--- a/drivers/ntb/Kconfig
+++ b/drivers/ntb/Kconfig
@@ -14,8 +14,6 @@ if NTB
source "drivers/ntb/hw/Kconfig"
-source "drivers/ntb/test/Kconfig"
-
config NTB_TRANSPORT
tristate "NTB Transport Client"
help
@@ -25,4 +23,6 @@ config NTB_TRANSPORT
If unsure, say N.
+source "drivers/ntb/test/Kconfig"
+
endif # NTB
diff --git a/drivers/ntb/hw/amd/ntb_hw_amd.c b/drivers/ntb/hw/amd/ntb_hw_amd.c
index 6ccba0d..ab6f353 100644
--- a/drivers/ntb/hw/amd/ntb_hw_amd.c
+++ b/drivers/ntb/hw/amd/ntb_hw_amd.c
@@ -55,6 +55,7 @@
#include <linux/pci.h>
#include <linux/random.h>
#include <linux/slab.h>
+#include <linux/sizes.h>
#include <linux/ntb.h>
#include "ntb_hw_amd.h"
@@ -84,11 +85,8 @@ static int amd_ntb_mw_count(struct ntb_dev *ntb)
return ntb_ndev(ntb)->mw_count;
}
-static int amd_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base,
- resource_size_t *size,
- resource_size_t *align,
- resource_size_t *align_size)
+static int amd_ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size)
{
struct amd_ntb_dev *ndev = ntb_ndev(ntb);
int bar;
@@ -103,17 +101,40 @@ static int amd_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
if (size)
*size = pci_resource_len(ndev->ntb.pdev, bar);
- if (align)
- *align = SZ_4K;
+ return 0;
+}
+
+static int amd_ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ return ntb_ndev(ntb)->mw_count;
+}
+
+static int amd_ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct amd_ntb_dev *ndev = ntb_ndev(ntb);
+ int bar;
+
+ bar = ndev_mw_to_bar(ndev, idx);
+ if (bar < 0)
+ return bar;
+
+ if (addr_align)
+ *addr_align = SZ_4K;
+
+ if (size_align)
+ *size_align = 1;
- if (align_size)
- *align_size = 1;
+ if (size_max)
+ *size_max = pci_resource_len(ndev->ntb.pdev, bar);
return 0;
}
-static int amd_ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
- dma_addr_t addr, resource_size_t size)
+static int amd_ntb_peer_mw_set_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size)
{
struct amd_ntb_dev *ndev = ntb_ndev(ntb);
unsigned long xlat_reg, limit_reg = 0;
@@ -432,8 +453,10 @@ static int amd_ntb_peer_spad_write(struct ntb_dev *ntb,
static const struct ntb_dev_ops amd_ntb_ops = {
.mw_count = amd_ntb_mw_count,
- .mw_get_range = amd_ntb_mw_get_range,
- .mw_set_trans = amd_ntb_mw_set_trans,
+ .mw_get_maprsc = amd_ntb_mw_get_maprsc,
+ .peer_mw_count = amd_ntb_peer_mw_count,
+ .peer_mw_get_align = amd_ntb_peer_mw_get_align,
+ .peer_mw_set_trans = amd_ntb_peer_mw_set_trans,
.link_is_up = amd_ntb_link_is_up,
.link_enable = amd_ntb_link_enable,
.link_disable = amd_ntb_link_disable,
diff --git a/drivers/ntb/hw/intel/ntb_hw_intel.c b/drivers/ntb/hw/intel/ntb_hw_intel.c
index 40d04ef..fdb2838 100644
--- a/drivers/ntb/hw/intel/ntb_hw_intel.c
+++ b/drivers/ntb/hw/intel/ntb_hw_intel.c
@@ -804,11 +804,8 @@ static int intel_ntb_mw_count(struct ntb_dev *ntb)
return ntb_ndev(ntb)->mw_count;
}
-static int intel_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base,
- resource_size_t *size,
- resource_size_t *align,
- resource_size_t *align_size)
+static int intel_ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size)
{
struct intel_ntb_dev *ndev = ntb_ndev(ntb);
int bar;
@@ -828,17 +825,51 @@ static int intel_ntb_mw_get_range(struct ntb_dev *ntb, int idx,
*size = pci_resource_len(ndev->ntb.pdev, bar) -
(idx == ndev->b2b_idx ? ndev->b2b_off : 0);
- if (align)
- *align = pci_resource_len(ndev->ntb.pdev, bar);
+ return 0;
+}
+
+static int intel_ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ return ntb_ndev(ntb)->mw_count;
+}
+
+static int intel_ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct intel_ntb_dev *ndev = ntb_ndev(ntb);
+ resource_size_t bar_size, mw_size;
+ int bar;
+
+ if (idx >= ndev->b2b_idx && !ndev->b2b_off)
+ idx += 1;
+
+ bar = ndev_mw_to_bar(ndev, idx);
+ if (bar < 0)
+ return bar;
+
+ bar_size = pci_resource_len(ndev->ntb.pdev, bar);
+
+ if (idx == ndev->b2b_idx)
+ mw_size = bar_size - ndev->b2b_off;
+ else
+ mw_size = bar_size;
+
+ if (addr_align)
+ *addr_align = bar_size;
+
+ if (size_align)
+ *size_align = 1;
- if (align_size)
- *align_size = 1;
+ if (size_max)
+ *size_max = mw_size;
return 0;
}
-static int intel_ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
- dma_addr_t addr, resource_size_t size)
+static int intel_ntb_peer_mw_set_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size)
{
struct intel_ntb_dev *ndev = ntb_ndev(ntb);
unsigned long base_reg, xlat_reg, limit_reg;
@@ -2220,8 +2251,10 @@ static struct intel_b2b_addr xeon_b2b_dsd_addr = {
/* operations for primary side of local ntb */
static const struct ntb_dev_ops intel_ntb_ops = {
.mw_count = intel_ntb_mw_count,
- .mw_get_range = intel_ntb_mw_get_range,
- .mw_set_trans = intel_ntb_mw_set_trans,
+ .mw_get_maprsc = intel_ntb_mw_get_maprsc,
+ .peer_mw_count = intel_ntb_peer_mw_count,
+ .peer_mw_get_align = intel_ntb_peer_mw_get_align,
+ .peer_mw_set_trans = intel_ntb_peer_mw_set_trans,
.link_is_up = intel_ntb_link_is_up,
.link_enable = intel_ntb_link_enable,
.link_disable = intel_ntb_link_disable,
diff --git a/drivers/ntb/ntb.c b/drivers/ntb/ntb.c
index 2e25307..37c3b36 100644
--- a/drivers/ntb/ntb.c
+++ b/drivers/ntb/ntb.c
@@ -54,6 +54,7 @@
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/atomic.h>
#include <linux/ntb.h>
#include <linux/pci.h>
@@ -72,8 +73,62 @@ MODULE_AUTHOR(DRIVER_AUTHOR);
MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
static struct bus_type ntb_bus;
+static struct ntb_bus_data ntb_data;
static void ntb_dev_release(struct device *dev);
+static int ntb_gen_devid(struct ntb_dev *ntb)
+{
+ const char *name;
+ unsigned long *mask;
+ int id;
+
+ if (ntb_valid_sync_dev_ops(ntb) && ntb_valid_async_dev_ops(ntb)) {
+ name = "ntbAS%d";
+ mask = ntb_data.both_msk;
+ } else if (ntb_valid_sync_dev_ops(ntb)) {
+ name = "ntbS%d";
+ mask = ntb_data.sync_msk;
+ } else if (ntb_valid_async_dev_ops(ntb)) {
+ name = "ntbA%d";
+ mask = ntb_data.async_msk;
+ } else {
+ return -EINVAL;
+ }
+
+ for (id = 0; NTB_MAX_DEVID > id; id++) {
+ if (0 == test_and_set_bit(id, mask)) {
+ ntb->id = id;
+ break;
+ }
+ }
+
+ if (NTB_MAX_DEVID > id) {
+ dev_set_name(&ntb->dev, name, ntb->id);
+ } else {
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void ntb_free_devid(struct ntb_dev *ntb)
+{
+ unsigned long *mask;
+
+ if (ntb_valid_sync_dev_ops(ntb) && ntb_valid_async_dev_ops(ntb)) {
+ mask = ntb_data.both_msk;
+ } else if (ntb_valid_sync_dev_ops(ntb)) {
+ mask = ntb_data.sync_msk;
+ } else if (ntb_valid_async_dev_ops(ntb)) {
+ mask = ntb_data.async_msk;
+ } else {
+ /* It's impossible */
+ BUG();
+ }
+
+ clear_bit(ntb->id, mask);
+}
+
int __ntb_register_client(struct ntb_client *client, struct module *mod,
const char *mod_name)
{
@@ -99,13 +154,15 @@ EXPORT_SYMBOL(ntb_unregister_client);
int ntb_register_device(struct ntb_dev *ntb)
{
+ int ret;
+
if (!ntb)
return -EINVAL;
if (!ntb->pdev)
return -EINVAL;
if (!ntb->ops)
return -EINVAL;
- if (!ntb_dev_ops_is_valid(ntb->ops))
+ if (!ntb_valid_sync_dev_ops(ntb) && !ntb_valid_async_dev_ops(ntb))
return -EINVAL;
init_completion(&ntb->released);
@@ -114,13 +171,21 @@ int ntb_register_device(struct ntb_dev *ntb)
ntb->dev.bus = &ntb_bus;
ntb->dev.parent = &ntb->pdev->dev;
ntb->dev.release = ntb_dev_release;
- dev_set_name(&ntb->dev, "%s", pci_name(ntb->pdev));
ntb->ctx = NULL;
ntb->ctx_ops = NULL;
spin_lock_init(&ntb->ctx_lock);
- return device_register(&ntb->dev);
+ /* No need to wait for completion if failed */
+ ret = ntb_gen_devid(ntb);
+ if (ret)
+ return ret;
+
+ ret = device_register(&ntb->dev);
+ if (ret)
+ ntb_free_devid(ntb);
+
+ return ret;
}
EXPORT_SYMBOL(ntb_register_device);
@@ -128,6 +193,7 @@ void ntb_unregister_device(struct ntb_dev *ntb)
{
device_unregister(&ntb->dev);
wait_for_completion(&ntb->released);
+ ntb_free_devid(ntb);
}
EXPORT_SYMBOL(ntb_unregister_device);
@@ -191,6 +257,20 @@ void ntb_db_event(struct ntb_dev *ntb, int vector)
}
EXPORT_SYMBOL(ntb_db_event);
+void ntb_msg_event(struct ntb_dev *ntb, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg)
+{
+ unsigned long irqflags;
+
+ spin_lock_irqsave(&ntb->ctx_lock, irqflags);
+ {
+ if (ntb->ctx_ops && ntb->ctx_ops->msg_event)
+ ntb->ctx_ops->msg_event(ntb->ctx, ev, msg);
+ }
+ spin_unlock_irqrestore(&ntb->ctx_lock, irqflags);
+}
+EXPORT_SYMBOL(ntb_msg_event);
+
static int ntb_probe(struct device *dev)
{
struct ntb_dev *ntb;
diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
index d5c5894..2626ba0 100644
--- a/drivers/ntb/ntb_transport.c
+++ b/drivers/ntb/ntb_transport.c
@@ -673,7 +673,7 @@ static void ntb_free_mw(struct ntb_transport_ctx *nt, int num_mw)
if (!mw->virt_addr)
return;
- ntb_mw_clear_trans(nt->ndev, num_mw);
+ ntb_peer_mw_set_trans(nt->ndev, num_mw, 0, 0);
dma_free_coherent(&pdev->dev, mw->buff_size,
mw->virt_addr, mw->dma_addr);
mw->xlat_size = 0;
@@ -730,7 +730,8 @@ static int ntb_set_mw(struct ntb_transport_ctx *nt, int num_mw,
}
/* Notify HW the memory location of the receive buffer */
- rc = ntb_mw_set_trans(nt->ndev, num_mw, mw->dma_addr, mw->xlat_size);
+ rc = ntb_peer_mw_set_trans(nt->ndev, num_mw, mw->dma_addr,
+ mw->xlat_size);
if (rc) {
dev_err(&pdev->dev, "Unable to set mw%d translation", num_mw);
ntb_free_mw(nt, num_mw);
@@ -1060,7 +1061,11 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
int node;
int rc, i;
- mw_count = ntb_mw_count(ndev);
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ndev))
+ return -EINVAL;
+
+ mw_count = ntb_peer_mw_count(ndev);
if (ntb_spad_count(ndev) < (NUM_MWS + 1 + mw_count * 2)) {
dev_err(&ndev->dev, "Not enough scratch pad registers for %s",
NTB_TRANSPORT_NAME);
@@ -1094,8 +1099,12 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
for (i = 0; i < mw_count; i++) {
mw = &nt->mw_vec[i];
- rc = ntb_mw_get_range(ndev, i, &mw->phys_addr, &mw->phys_size,
- &mw->xlat_align, &mw->xlat_align_size);
+ rc = ntb_mw_get_maprsc(ndev, i, &mw->phys_addr, &mw->phys_size);
+ if (rc)
+ goto err1;
+
+ rc = ntb_peer_mw_get_align(ndev, i, &mw->xlat_align,
+ &mw->xlat_align_size, NULL);
if (rc)
goto err1;
diff --git a/drivers/ntb/test/ntb_perf.c b/drivers/ntb/test/ntb_perf.c
index 6a50f20..f2952f7 100644
--- a/drivers/ntb/test/ntb_perf.c
+++ b/drivers/ntb/test/ntb_perf.c
@@ -452,7 +452,7 @@ static void perf_free_mw(struct perf_ctx *perf)
if (!mw->virt_addr)
return;
- ntb_mw_clear_trans(perf->ntb, 0);
+ ntb_peer_mw_set_trans(perf->ntb, 0, 0, 0);
dma_free_coherent(&pdev->dev, mw->buf_size,
mw->virt_addr, mw->dma_addr);
mw->xlat_size = 0;
@@ -488,7 +488,7 @@ static int perf_set_mw(struct perf_ctx *perf, resource_size_t size)
mw->buf_size = 0;
}
- rc = ntb_mw_set_trans(perf->ntb, 0, mw->dma_addr, mw->xlat_size);
+ rc = ntb_peer_mw_set_trans(perf->ntb, 0, mw->dma_addr, mw->xlat_size);
if (rc) {
dev_err(&perf->ntb->dev, "Unable to set mw0 translation\n");
perf_free_mw(perf);
@@ -559,8 +559,12 @@ static int perf_setup_mw(struct ntb_dev *ntb, struct perf_ctx *perf)
mw = &perf->mw;
- rc = ntb_mw_get_range(ntb, 0, &mw->phys_addr, &mw->phys_size,
- &mw->xlat_align, &mw->xlat_align_size);
+ rc = ntb_mw_get_maprsc(ntb, 0, &mw->phys_addr, &mw->phys_size);
+ if (rc)
+ return rc;
+
+ rc = ntb_peer_mw_get_align(ntb, 0, &mw->xlat_align,
+ &mw->xlat_align_size, NULL);
if (rc)
return rc;
@@ -758,6 +762,10 @@ static int perf_probe(struct ntb_client *client, struct ntb_dev *ntb)
int node;
int rc = 0;
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ntb))
+ return -EINVAL;
+
if (ntb_spad_count(ntb) < MAX_SPAD) {
dev_err(&ntb->dev, "Not enough scratch pad registers for %s",
DRIVER_NAME);
diff --git a/drivers/ntb/test/ntb_pingpong.c b/drivers/ntb/test/ntb_pingpong.c
index 7d31179..e833649 100644
--- a/drivers/ntb/test/ntb_pingpong.c
+++ b/drivers/ntb/test/ntb_pingpong.c
@@ -214,6 +214,11 @@ static int pp_probe(struct ntb_client *client,
struct pp_ctx *pp;
int rc;
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
if (ntb_db_is_unsafe(ntb)) {
dev_dbg(&ntb->dev, "doorbell is unsafe\n");
if (!unsafe) {
diff --git a/drivers/ntb/test/ntb_tool.c b/drivers/ntb/test/ntb_tool.c
index 61bf2ef..5dfe12f 100644
--- a/drivers/ntb/test/ntb_tool.c
+++ b/drivers/ntb/test/ntb_tool.c
@@ -675,8 +675,11 @@ static int tool_setup_mw(struct tool_ctx *tc, int idx, size_t req_size)
if (mw->peer)
return 0;
- rc = ntb_mw_get_range(tc->ntb, idx, &base, &size, &align,
- &align_size);
+ rc = ntb_mw_get_maprsc(tc->ntb, idx, &base, &size);
+ if (rc)
+ return rc;
+
+ rc = ntb_peer_mw_get_align(tc->ntb, idx, &align, &align_size, NULL);
if (rc)
return rc;
@@ -689,7 +692,7 @@ static int tool_setup_mw(struct tool_ctx *tc, int idx, size_t req_size)
if (!mw->peer)
return -ENOMEM;
- rc = ntb_mw_set_trans(tc->ntb, idx, mw->peer_dma, mw->size);
+ rc = ntb_peer_mw_set_trans(tc->ntb, idx, mw->peer_dma, mw->size);
if (rc)
goto err_free_dma;
@@ -716,7 +719,7 @@ static void tool_free_mw(struct tool_ctx *tc, int idx)
struct tool_mw *mw = &tc->mws[idx];
if (mw->peer) {
- ntb_mw_clear_trans(tc->ntb, idx);
+ ntb_peer_mw_set_trans(tc->ntb, idx, 0, 0);
dma_free_coherent(&tc->ntb->pdev->dev, mw->size,
mw->peer,
mw->peer_dma);
@@ -751,8 +754,8 @@ static ssize_t tool_peer_mw_trans_read(struct file *filep,
if (!buf)
return -ENOMEM;
- ntb_mw_get_range(mw->tc->ntb, mw->idx,
- &base, &mw_size, &align, &align_size);
+ ntb_mw_get_maprsc(mw->tc->ntb, mw->idx, &base, &mw_size);
+ ntb_peer_mw_get_align(mw->tc->ntb, mw->idx, &align, &align_size, NULL);
off += scnprintf(buf + off, buf_size - off,
"Peer MW %d Information:\n", mw->idx);
@@ -827,8 +830,7 @@ static int tool_init_mw(struct tool_ctx *tc, int idx)
phys_addr_t base;
int rc;
- rc = ntb_mw_get_range(tc->ntb, idx, &base, &mw->win_size,
- NULL, NULL);
+ rc = ntb_mw_get_maprsc(tc->ntb, idx, &base, &mw->win_size);
if (rc)
return rc;
@@ -913,6 +915,11 @@ static int tool_probe(struct ntb_client *self, struct ntb_dev *ntb)
int rc;
int i;
+ /* Synchronous hardware is only supported */
+ if (!ntb_valid_sync_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
if (ntb_db_is_unsafe(ntb))
dev_dbg(&ntb->dev, "doorbell is unsafe\n");
@@ -928,7 +935,7 @@ static int tool_probe(struct ntb_client *self, struct ntb_dev *ntb)
tc->ntb = ntb;
init_waitqueue_head(&tc->link_wq);
- tc->mw_count = min(ntb_mw_count(tc->ntb), MAX_MWS);
+ tc->mw_count = min(ntb_peer_mw_count(tc->ntb), MAX_MWS);
for (i = 0; i < tc->mw_count; i++) {
rc = tool_init_mw(tc, i);
if (rc)
diff --git a/include/linux/ntb.h b/include/linux/ntb.h
index 6f47562..d1937d3 100644
--- a/include/linux/ntb.h
+++ b/include/linux/ntb.h
@@ -159,13 +159,44 @@ static inline int ntb_client_ops_is_valid(const struct ntb_client_ops *ops)
}
/**
+ * struct ntb_msg - ntb driver message structure
+ * @type: Message type.
+ * @payload: Payload data to send to a peer
+ * @data: Array of u32 data to send (size might be hw dependent)
+ */
+#define NTB_MAX_MSGSIZE 4
+struct ntb_msg {
+ union {
+ struct {
+ u32 type;
+ u32 payload[NTB_MAX_MSGSIZE - 1];
+ };
+ u32 data[NTB_MAX_MSGSIZE];
+ };
+};
+
+/**
+ * enum NTB_MSG_EVENT - message event types
+ * @NTB_MSG_NEW: New message just arrived and passed to the handler
+ * @NTB_MSG_SENT: Posted message has just been successfully sent
+ * @NTB_MSG_FAIL: Posted message failed to be sent
+ */
+enum NTB_MSG_EVENT {
+ NTB_MSG_NEW,
+ NTB_MSG_SENT,
+ NTB_MSG_FAIL
+};
+
+/**
* struct ntb_ctx_ops - ntb driver context operations
* @link_event: See ntb_link_event().
* @db_event: See ntb_db_event().
+ * @msg_event: See ntb_msg_event().
*/
struct ntb_ctx_ops {
void (*link_event)(void *ctx);
void (*db_event)(void *ctx, int db_vector);
+ void (*msg_event)(void *ctx, enum NTB_MSG_EVENT ev, struct ntb_msg *msg);
};
static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
@@ -174,18 +205,24 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
return
/* ops->link_event && */
/* ops->db_event && */
+ /* ops->msg_event && */
1;
}
/**
* struct ntb_ctx_ops - ntb device operations
- * @mw_count: See ntb_mw_count().
- * @mw_get_range: See ntb_mw_get_range().
- * @mw_set_trans: See ntb_mw_set_trans().
- * @mw_clear_trans: See ntb_mw_clear_trans().
* @link_is_up: See ntb_link_is_up().
* @link_enable: See ntb_link_enable().
* @link_disable: See ntb_link_disable().
+ * @mw_count: See ntb_mw_count().
+ * @mw_get_maprsc: See ntb_mw_get_maprsc().
+ * @mw_set_trans: See ntb_mw_set_trans().
+ * @mw_get_trans: See ntb_mw_get_trans().
+ * @mw_get_align: See ntb_mw_get_align().
+ * @peer_mw_count: See ntb_peer_mw_count().
+ * @peer_mw_set_trans: See ntb_peer_mw_set_trans().
+ * @peer_mw_get_trans: See ntb_peer_mw_get_trans().
+ * @peer_mw_get_align: See ntb_peer_mw_get_align().
* @db_is_unsafe: See ntb_db_is_unsafe().
* @db_valid_mask: See ntb_db_valid_mask().
* @db_vector_count: See ntb_db_vector_count().
@@ -210,22 +247,38 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
* @peer_spad_addr: See ntb_peer_spad_addr().
* @peer_spad_read: See ntb_peer_spad_read().
* @peer_spad_write: See ntb_peer_spad_write().
+ * @msg_post: See ntb_msg_post().
+ * @msg_size: See ntb_msg_size().
*/
struct ntb_dev_ops {
- int (*mw_count)(struct ntb_dev *ntb);
- int (*mw_get_range)(struct ntb_dev *ntb, int idx,
- phys_addr_t *base, resource_size_t *size,
- resource_size_t *align, resource_size_t *align_size);
- int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
- dma_addr_t addr, resource_size_t size);
- int (*mw_clear_trans)(struct ntb_dev *ntb, int idx);
-
int (*link_is_up)(struct ntb_dev *ntb,
enum ntb_speed *speed, enum ntb_width *width);
int (*link_enable)(struct ntb_dev *ntb,
enum ntb_speed max_speed, enum ntb_width max_width);
int (*link_disable)(struct ntb_dev *ntb);
+ int (*mw_count)(struct ntb_dev *ntb);
+ int (*mw_get_maprsc)(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size);
+ int (*mw_get_align)(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max);
+ int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size);
+ int (*mw_get_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size);
+
+ int (*peer_mw_count)(struct ntb_dev *ntb);
+ int (*peer_mw_get_align)(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max);
+ int (*peer_mw_set_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size);
+ int (*peer_mw_get_trans)(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size);
+
int (*db_is_unsafe)(struct ntb_dev *ntb);
u64 (*db_valid_mask)(struct ntb_dev *ntb);
int (*db_vector_count)(struct ntb_dev *ntb);
@@ -259,47 +312,10 @@ struct ntb_dev_ops {
phys_addr_t *spad_addr);
u32 (*peer_spad_read)(struct ntb_dev *ntb, int idx);
int (*peer_spad_write)(struct ntb_dev *ntb, int idx, u32 val);
-};
-
-static inline int ntb_dev_ops_is_valid(const struct ntb_dev_ops *ops)
-{
- /* commented callbacks are not required: */
- return
- ops->mw_count &&
- ops->mw_get_range &&
- ops->mw_set_trans &&
- /* ops->mw_clear_trans && */
- ops->link_is_up &&
- ops->link_enable &&
- ops->link_disable &&
- /* ops->db_is_unsafe && */
- ops->db_valid_mask &&
- /* both set, or both unset */
- (!ops->db_vector_count == !ops->db_vector_mask) &&
-
- ops->db_read &&
- /* ops->db_set && */
- ops->db_clear &&
- /* ops->db_read_mask && */
- ops->db_set_mask &&
- ops->db_clear_mask &&
- /* ops->peer_db_addr && */
- /* ops->peer_db_read && */
- ops->peer_db_set &&
- /* ops->peer_db_clear && */
- /* ops->peer_db_read_mask && */
- /* ops->peer_db_set_mask && */
- /* ops->peer_db_clear_mask && */
- /* ops->spad_is_unsafe && */
- ops->spad_count &&
- ops->spad_read &&
- ops->spad_write &&
- /* ops->peer_spad_addr && */
- /* ops->peer_spad_read && */
- ops->peer_spad_write &&
- 1;
-}
+ int (*msg_post)(struct ntb_dev *ntb, struct ntb_msg *msg);
+ int (*msg_size)(struct ntb_dev *ntb);
+};
/**
* struct ntb_client - client interested in ntb devices
@@ -310,10 +326,22 @@ struct ntb_client {
struct device_driver drv;
const struct ntb_client_ops ops;
};
-
#define drv_ntb_client(__drv) container_of((__drv), struct ntb_client, drv)
/**
+ * struct ntb_bus_data - NTB bus data
+ * @sync_msk: Synchroous devices mask
+ * @async_msk: Asynchronous devices mask
+ * @both_msk: Both sync and async devices mask
+ */
+#define NTB_MAX_DEVID (8*BITS_PER_LONG)
+struct ntb_bus_data {
+ unsigned long sync_msk[8];
+ unsigned long async_msk[8];
+ unsigned long both_msk[8];
+};
+
+/**
* struct ntb_device - ntb device
* @dev: Linux device object.
* @pdev: Pci device entry of the ntb.
@@ -332,15 +360,151 @@ struct ntb_dev {
/* private: */
+ /* device id */
+ int id;
/* synchronize setting, clearing, and calling ctx_ops */
spinlock_t ctx_lock;
/* block unregister until device is fully released */
struct completion released;
};
-
#define dev_ntb(__dev) container_of((__dev), struct ntb_dev, dev)
/**
+ * ntb_valid_sync_dev_ops() - valid operations for synchronous hardware setup
+ * @ntb: NTB device
+ *
+ * There might be two types of NTB hardware differed by the way of the settings
+ * configuration. The synchronous chips allows to set the memory windows by
+ * directly writing to the peer registers. Additionally there can be shared
+ * Scratchpad registers for synchronous information exchange. Client drivers
+ * should call this function to make sure the hardware supports the proper
+ * functionality.
+ */
+static inline int ntb_valid_sync_dev_ops(const struct ntb_dev *ntb)
+{
+ const struct ntb_dev_ops *ops = ntb->ops;
+
+ /* Commented callbacks are not required, but might be developed */
+ return /* NTB link status ops */
+ ops->link_is_up &&
+ ops->link_enable &&
+ ops->link_disable &&
+
+ /* Synchronous memory windows ops */
+ ops->mw_count &&
+ ops->mw_get_maprsc &&
+ /* ops->mw_get_align && */
+ /* ops->mw_set_trans && */
+ /* ops->mw_get_trans && */
+ ops->peer_mw_count &&
+ ops->peer_mw_get_align &&
+ ops->peer_mw_set_trans &&
+ /* ops->peer_mw_get_trans && */
+
+ /* Doorbell ops */
+ /* ops->db_is_unsafe && */
+ ops->db_valid_mask &&
+ /* both set, or both unset */
+ (!ops->db_vector_count == !ops->db_vector_mask) &&
+ ops->db_read &&
+ /* ops->db_set && */
+ ops->db_clear &&
+ /* ops->db_read_mask && */
+ ops->db_set_mask &&
+ ops->db_clear_mask &&
+ /* ops->peer_db_addr && */
+ /* ops->peer_db_read && */
+ ops->peer_db_set &&
+ /* ops->peer_db_clear && */
+ /* ops->peer_db_read_mask && */
+ /* ops->peer_db_set_mask && */
+ /* ops->peer_db_clear_mask && */
+
+ /* Scratchpad ops */
+ /* ops->spad_is_unsafe && */
+ ops->spad_count &&
+ ops->spad_read &&
+ ops->spad_write &&
+ /* ops->peer_spad_addr && */
+ /* ops->peer_spad_read && */
+ ops->peer_spad_write &&
+
+ /* Messages IO ops */
+ /* ops->msg_post && */
+ /* ops->msg_size && */
+ 1;
+}
+
+/**
+ * ntb_valid_async_dev_ops() - valid operations for asynchronous hardware setup
+ * @ntb: NTB device
+ *
+ * There might be two types of NTB hardware differed by the way of the settings
+ * configuration. The asynchronous chips does not allow to set the memory
+ * windows by directly writing to the peer registers. Instead it implements
+ * the additional method to communinicate between NTB nodes like messages.
+ * Scratchpad registers aren't likely supported by such hardware. Client
+ * drivers should call this function to make sure the hardware supports
+ * the proper functionality.
+ */
+static inline int ntb_valid_async_dev_ops(const struct ntb_dev *ntb)
+{
+ const struct ntb_dev_ops *ops = ntb->ops;
+
+ /* Commented callbacks are not required, but might be developed */
+ return /* NTB link status ops */
+ ops->link_is_up &&
+ ops->link_enable &&
+ ops->link_disable &&
+
+ /* Asynchronous memory windows ops */
+ ops->mw_count &&
+ ops->mw_get_maprsc &&
+ ops->mw_get_align &&
+ ops->mw_set_trans &&
+ /* ops->mw_get_trans && */
+ ops->peer_mw_count &&
+ ops->peer_mw_get_align &&
+ /* ops->peer_mw_set_trans && */
+ /* ops->peer_mw_get_trans && */
+
+ /* Doorbell ops */
+ /* ops->db_is_unsafe && */
+ ops->db_valid_mask &&
+ /* both set, or both unset */
+ (!ops->db_vector_count == !ops->db_vector_mask) &&
+ ops->db_read &&
+ /* ops->db_set && */
+ ops->db_clear &&
+ /* ops->db_read_mask && */
+ ops->db_set_mask &&
+ ops->db_clear_mask &&
+ /* ops->peer_db_addr && */
+ /* ops->peer_db_read && */
+ ops->peer_db_set &&
+ /* ops->peer_db_clear && */
+ /* ops->peer_db_read_mask && */
+ /* ops->peer_db_set_mask && */
+ /* ops->peer_db_clear_mask && */
+
+ /* Scratchpad ops */
+ /* ops->spad_is_unsafe && */
+ /* ops->spad_count && */
+ /* ops->spad_read && */
+ /* ops->spad_write && */
+ /* ops->peer_spad_addr && */
+ /* ops->peer_spad_read && */
+ /* ops->peer_spad_write && */
+
+ /* Messages IO ops */
+ ops->msg_post &&
+ ops->msg_size &&
+ 1;
+}
+
+
+
+/**
* ntb_register_client() - register a client for interest in ntb devices
* @client: Client context.
*
@@ -441,10 +605,84 @@ void ntb_link_event(struct ntb_dev *ntb);
void ntb_db_event(struct ntb_dev *ntb, int vector);
/**
- * ntb_mw_count() - get the number of memory windows
+ * ntb_msg_event() - notify driver context of event in messaging subsystem
* @ntb: NTB device context.
+ * @ev: Event type caused the handler invocation
+ * @msg: Message related to the event
+ *
+ * Notify the driver context that there is some event happaned in the event
+ * subsystem. If NTB_MSG_NEW is emitted then the new message has just arrived.
+ * NTB_MSG_SENT is rised if some message has just been successfully sent to a
+ * peer. If a message failed to be sent then NTB_MSG_FAIL is emitted. The very
+ * last argument is used to pass the event related message. It discarded right
+ * after the handler returns.
+ */
+void ntb_msg_event(struct ntb_dev *ntb, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg);
+
+/**
+ * ntb_link_is_up() - get the current ntb link state
+ * @ntb: NTB device context.
+ * @speed: OUT - The link speed expressed as PCIe generation number.
+ * @width: OUT - The link width expressed as the number of PCIe lanes.
+ *
+ * Get the current state of the ntb link. It is recommended to query the link
+ * state once after every link event. It is safe to query the link state in
+ * the context of the link event callback.
+ *
+ * Return: One if the link is up, zero if the link is down, otherwise a
+ * negative value indicating the error number.
+ */
+static inline int ntb_link_is_up(struct ntb_dev *ntb,
+ enum ntb_speed *speed, enum ntb_width *width)
+{
+ return ntb->ops->link_is_up(ntb, speed, width);
+}
+
+/**
+ * ntb_link_enable() - enable the link on the secondary side of the ntb
+ * @ntb: NTB device context.
+ * @max_speed: The maximum link speed expressed as PCIe generation number.
+ * @max_width: The maximum link width expressed as the number of PCIe lanes.
*
- * Hardware and topology may support a different number of memory windows.
+ * Enable the link on the secondary side of the ntb. This can only be done
+ * from only one (primary or secondary) side of the ntb in primary or b2b
+ * topology. The ntb device should train the link to its maximum speed and
+ * width, or the requested speed and width, whichever is smaller, if supported.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_link_enable(struct ntb_dev *ntb,
+ enum ntb_speed max_speed,
+ enum ntb_width max_width)
+{
+ return ntb->ops->link_enable(ntb, max_speed, max_width);
+}
+
+/**
+ * ntb_link_disable() - disable the link on the secondary side of the ntb
+ * @ntb: NTB device context.
+ *
+ * Disable the link on the secondary side of the ntb. This can only be
+ * done from only one (primary or secondary) side of the ntb in primary or b2b
+ * topology. The ntb device should disable the link. Returning from this call
+ * must indicate that a barrier has passed, though with no more writes may pass
+ * in either direction across the link, except if this call returns an error
+ * number.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_link_disable(struct ntb_dev *ntb)
+{
+ return ntb->ops->link_disable(ntb);
+}
+
+/**
+ * ntb_mw_count() - get the number of local memory windows
+ * @ntb: NTB device context.
+ *
+ * Hardware and topology may support a different number of memory windows at
+ * local and remote devices
*
* Return: the number of memory windows.
*/
@@ -454,122 +692,186 @@ static inline int ntb_mw_count(struct ntb_dev *ntb)
}
/**
- * ntb_mw_get_range() - get the range of a memory window
+ * ntb_mw_get_maprsc() - get the range of a memory window to map
* @ntb: NTB device context.
* @idx: Memory window number.
* @base: OUT - the base address for mapping the memory window
* @size: OUT - the size for mapping the memory window
- * @align: OUT - the base alignment for translating the memory window
- * @align_size: OUT - the size alignment for translating the memory window
*
- * Get the range of a memory window. NULL may be given for any output
- * parameter if the value is not needed. The base and size may be used for
- * mapping the memory window, to access the peer memory. The alignment and
- * size may be used for translating the memory window, for the peer to access
- * memory on the local system.
+ * Get the map range of a memory window. The base and size may be used for
+ * mapping the memory window to access the peer memory.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base, resource_size_t *size,
- resource_size_t *align, resource_size_t *align_size)
+static inline int ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx,
+ phys_addr_t *base, resource_size_t *size)
{
- return ntb->ops->mw_get_range(ntb, idx, base, size,
- align, align_size);
+ return ntb->ops->mw_get_maprsc(ntb, idx, base, size);
+}
+
+/**
+ * ntb_mw_get_align() - get memory window alignment of the local node
+ * @ntb: NTB device context.
+ * @idx: Memory window number.
+ * @addr_align: OUT - the translated base address alignment of the memory window
+ * @size_align: OUT - the translated memory size alignment of the memory window
+ * @size_max: OUT - the translated memory maximum size
+ *
+ * Get the alignment parameters to allocate the proper memory window. NULL may
+ * be given for any output parameter if the value is not needed.
+ *
+ * Drivers of synchronous hardware don't have to support it.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ if (!ntb->ops->mw_get_align)
+ return -EINVAL;
+
+ return ntb->ops->mw_get_align(ntb, idx, addr_align, size_align, size_max);
}
/**
- * ntb_mw_set_trans() - set the translation of a memory window
+ * ntb_mw_set_trans() - set the translated base address of a peer memory window
* @ntb: NTB device context.
* @idx: Memory window number.
- * @addr: The dma address local memory to expose to the peer.
- * @size: The size of the local memory to expose to the peer.
+ * @addr: DMA memory address exposed by the peer.
+ * @size: Size of the memory exposed by the peer.
+ *
+ * Set the translated base address of a memory window. The peer preliminary
+ * allocates a memory, then someway passes the address to the remote node, that
+ * finally sets up the memory window at the address, up to the size. The address
+ * and size must be aligned to the parameters specified by ntb_mw_get_align() of
+ * the local node and ntb_peer_mw_get_align() of the peer, which must return the
+ * same values. Zero size effectively disables the memory window.
*
- * Set the translation of a memory window. The peer may access local memory
- * through the window starting at the address, up to the size. The address
- * must be aligned to the alignment specified by ntb_mw_get_range(). The size
- * must be aligned to the size alignment specified by ntb_mw_get_range().
+ * Drivers of synchronous hardware don't have to support it.
*
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
dma_addr_t addr, resource_size_t size)
{
+ if (!ntb->ops->mw_set_trans)
+ return -EINVAL;
+
return ntb->ops->mw_set_trans(ntb, idx, addr, size);
}
/**
- * ntb_mw_clear_trans() - clear the translation of a memory window
+ * ntb_mw_get_trans() - get the translated base address of a memory window
* @ntb: NTB device context.
* @idx: Memory window number.
+ * @addr: The dma memory address exposed by the peer.
+ * @size: The size of the memory exposed by the peer.
*
- * Clear the translation of a memory window. The peer may no longer access
- * local memory through the window.
+ * Get the translated base address of a memory window spicified for the local
+ * hardware and allocated by the peer. If the addr and size are zero, the
+ * memory window is effectively disabled.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int idx)
+static inline int ntb_mw_get_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size)
{
- if (!ntb->ops->mw_clear_trans)
- return ntb->ops->mw_set_trans(ntb, idx, 0, 0);
+ if (!ntb->ops->mw_get_trans)
+ return -EINVAL;
- return ntb->ops->mw_clear_trans(ntb, idx);
+ return ntb->ops->mw_get_trans(ntb, idx, addr, size);
}
/**
- * ntb_link_is_up() - get the current ntb link state
+ * ntb_peer_mw_count() - get the number of peer memory windows
* @ntb: NTB device context.
- * @speed: OUT - The link speed expressed as PCIe generation number.
- * @width: OUT - The link width expressed as the number of PCIe lanes.
*
- * Get the current state of the ntb link. It is recommended to query the link
- * state once after every link event. It is safe to query the link state in
- * the context of the link event callback.
+ * Hardware and topology may support a different number of memory windows at
+ * local and remote nodes.
*
- * Return: One if the link is up, zero if the link is down, otherwise a
- * negative value indicating the error number.
+ * Return: the number of memory windows.
*/
-static inline int ntb_link_is_up(struct ntb_dev *ntb,
- enum ntb_speed *speed, enum ntb_width *width)
+static inline int ntb_peer_mw_count(struct ntb_dev *ntb)
{
- return ntb->ops->link_is_up(ntb, speed, width);
+ return ntb->ops->peer_mw_count(ntb);
}
/**
- * ntb_link_enable() - enable the link on the secondary side of the ntb
+ * ntb_peer_mw_get_align() - get memory window alignment of the peer
* @ntb: NTB device context.
- * @max_speed: The maximum link speed expressed as PCIe generation number.
- * @max_width: The maximum link width expressed as the number of PCIe lanes.
+ * @idx: Memory window number.
+ * @addr_align: OUT - the translated base address alignment of the memory window
+ * @size_align: OUT - the translated memory size alignment of the memory window
+ * @size_max: OUT - the translated memory maximum size
*
- * Enable the link on the secondary side of the ntb. This can only be done
- * from the primary side of the ntb in primary or b2b topology. The ntb device
- * should train the link to its maximum speed and width, or the requested speed
- * and width, whichever is smaller, if supported.
+ * Get the alignment parameters to allocate the proper memory window for the
+ * peer. NULL may be given for any output parameter if the value is not needed.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_link_enable(struct ntb_dev *ntb,
- enum ntb_speed max_speed,
- enum ntb_width max_width)
+static inline int ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
{
- return ntb->ops->link_enable(ntb, max_speed, max_width);
+ if (!ntb->ops->peer_mw_get_align)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_get_align(ntb, idx, addr_align, size_align,
+ size_max);
}
/**
- * ntb_link_disable() - disable the link on the secondary side of the ntb
+ * ntb_peer_mw_set_trans() - set the translated base address of a peer
+ * memory window
* @ntb: NTB device context.
+ * @idx: Memory window number.
+ * @addr: Local DMA memory address exposed to the peer.
+ * @size: Size of the memory exposed to the peer.
*
- * Disable the link on the secondary side of the ntb. This can only be
- * done from the primary side of the ntb in primary or b2b topology. The ntb
- * device should disable the link. Returning from this call must indicate that
- * a barrier has passed, though with no more writes may pass in either
- * direction across the link, except if this call returns an error number.
+ * Set the translated base address of a memory window exposed to the peer.
+ * The local node preliminary allocates the window, then directly writes the
+ * address and size to the peer control registers. The address and size must
+ * be aligned to the parameters specified by ntb_peer_mw_get_align() of
+ * the local node and ntb_mw_get_align() of the peer, which must return the
+ * same values. Zero size effectively disables the memory window.
+ *
+ * Drivers of synchronous hardware must support it.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_link_disable(struct ntb_dev *ntb)
+static inline int ntb_peer_mw_set_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t addr, resource_size_t size)
{
- return ntb->ops->link_disable(ntb);
+ if (!ntb->ops->peer_mw_set_trans)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_set_trans(ntb, idx, addr, size);
+}
+
+/**
+ * ntb_peer_mw_get_trans() - get the translated base address of a peer
+ * memory window
+ * @ntb: NTB device context.
+ * @idx: Memory window number.
+ * @addr: Local dma memory address exposed to the peer.
+ * @size: Size of the memory exposed to the peer.
+ *
+ * Get the translated base address of a memory window spicified for the peer
+ * hardware. If the addr and size are zero then the memory window is effectively
+ * disabled.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_peer_mw_get_trans(struct ntb_dev *ntb, int idx,
+ dma_addr_t *addr, resource_size_t *size)
+{
+ if (!ntb->ops->peer_mw_get_trans)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_get_trans(ntb, idx, addr, size);
}
/**
@@ -751,6 +1053,8 @@ static inline int ntb_db_clear_mask(struct ntb_dev *ntb, u64 db_bits)
* append one additional dma memory copy with the doorbell register as the
* destination, after the memory copy operations.
*
+ * This is unusual, and hardware may not be suitable to implement it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_peer_db_addr(struct ntb_dev *ntb,
@@ -901,10 +1205,15 @@ static inline int ntb_spad_is_unsafe(struct ntb_dev *ntb)
*
* Hardware and topology may support a different number of scratchpads.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: the number of scratchpads.
*/
static inline int ntb_spad_count(struct ntb_dev *ntb)
{
+ if (!ntb->ops->spad_count)
+ return -EINVAL;
+
return ntb->ops->spad_count(ntb);
}
@@ -915,10 +1224,15 @@ static inline int ntb_spad_count(struct ntb_dev *ntb)
*
* Read the local scratchpad register, and return the value.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: The value of the local scratchpad register.
*/
static inline u32 ntb_spad_read(struct ntb_dev *ntb, int idx)
{
+ if (!ntb->ops->spad_read)
+ return 0;
+
return ntb->ops->spad_read(ntb, idx);
}
@@ -930,10 +1244,15 @@ static inline u32 ntb_spad_read(struct ntb_dev *ntb, int idx)
*
* Write the value to the local scratchpad register.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_spad_write(struct ntb_dev *ntb, int idx, u32 val)
{
+ if (!ntb->ops->spad_write)
+ return -EINVAL;
+
return ntb->ops->spad_write(ntb, idx, val);
}
@@ -946,6 +1265,8 @@ static inline int ntb_spad_write(struct ntb_dev *ntb, int idx, u32 val)
* Return the address of the peer doorbell register. This may be used, for
* example, by drivers that offload memory copy operations to a dma engine.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_peer_spad_addr(struct ntb_dev *ntb, int idx,
@@ -964,10 +1285,15 @@ static inline int ntb_peer_spad_addr(struct ntb_dev *ntb, int idx,
*
* Read the peer scratchpad register, and return the value.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: The value of the local scratchpad register.
*/
static inline u32 ntb_peer_spad_read(struct ntb_dev *ntb, int idx)
{
+ if (!ntb->ops->peer_spad_read)
+ return 0;
+
return ntb->ops->peer_spad_read(ntb, idx);
}
@@ -979,11 +1305,59 @@ static inline u32 ntb_peer_spad_read(struct ntb_dev *ntb, int idx)
*
* Write the value to the peer scratchpad register.
*
+ * Asynchronous hardware may not support it.
+ *
* Return: Zero on success, otherwise an error number.
*/
static inline int ntb_peer_spad_write(struct ntb_dev *ntb, int idx, u32 val)
{
+ if (!ntb->ops->peer_spad_write)
+ return -EINVAL;
+
return ntb->ops->peer_spad_write(ntb, idx, val);
}
+/**
+ * ntb_msg_post() - post the message to the peer
+ * @ntb: NTB device context.
+ * @msg: Message
+ *
+ * Post the message to a peer. It shall be delivered to the peer by the
+ * corresponding hardware method. The peer should be notified about the new
+ * message by calling the ntb_msg_event() handler of NTB_MSG_NEW event type.
+ * If delivery is fails for some reasong the local node will get NTB_MSG_FAIL
+ * event. Otherwise the NTB_MSG_SENT is emitted.
+ *
+ * Synchronous hardware may not support it.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_msg_post(struct ntb_dev *ntb, struct ntb_msg *msg)
+{
+ if (!ntb->ops->msg_post)
+ return -EINVAL;
+
+ return ntb->ops->msg_post(ntb, msg);
+}
+
+/**
+ * ntb_msg_size() - size of the message data
+ * @ntb: NTB device context.
+ *
+ * Different hardware may support different number of message registers. This
+ * callback shall return the number of those used for data sending and
+ * receiving including the type field.
+ *
+ * Synchronous hardware may not support it.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_msg_size(struct ntb_dev *ntb)
+{
+ if (!ntb->ops->msg_size)
+ return 0;
+
+ return ntb->ops->msg_size(ntb);
+}
+
#endif
--
2.6.6
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver
2016-07-28 10:01 ` [PATCH v2 0/3] ntb: Asynchronous NTB devices support Serge Semin
2016-07-28 10:01 ` [PATCH v2 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
@ 2016-07-28 10:01 ` Serge Semin
2016-07-28 10:01 ` [PATCH v2 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
2 siblings, 0 replies; 11+ messages in thread
From: Serge Semin @ 2016-07-28 10:01 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
IDT 89HPES*NT* PCIe-switches are the multi-port switches, which can be
configured to have more than two NT-functions, connected to each other.
Using these facility NT-function of local RC can access any NT-functions of
others. This driver implements Port-to-Ports architecture, so only Primary
port can have access to prededefined number of Secondary ports. Secondary
ports on the other hand can access the primary port memory only. The ports
hierarchy is configured using NTSDATA register of each NT-function, which
should be preinitialized with corresponding Primary port ID. Then driver will
evenly split the IDT PCIe-switch Doorbell and Memory Window resources between
all of the NT-function pairs, and add the corresponding devices to the NTB bus.
The NTSDATA register preinitialization can be done either by EEPROM or by linux
kernel PCI fixup interface. The default value is zero, so the port 0 is
considered being primary, the others are secondary.
The supported IDT PCIe-switches do not implement synchronous interface to
access Memory Windows configuration registers. There is no way to exclusively
access a peer translated base addresses Lookup Table. Instead the messaging
subsystem can be used to post the translated base address to a peer, so one
could properly initialize the table. In order to resolve the race conditions of
possible concurrent messages transfer by parallel tasks and different root
complexes, the ntb_msg_post() method places messages into the outbound messages
queue. An extra kernel work thread is used to perform an actual message
transfer with a constant frequency of attempts. If kernel thread fails to send
a message after a constant number of tries, then the corresponding client
driver is notified by rising the ntb_msg_event() method with NTB_MSG_FAIL
status. Additionaly there is a special tasklet, which reads messages from
incoming message registers and puts them into the inbound messages queue.
An another kernel thread is then started to deliver a message to a
corresponding client driver.
There are only 32 Global doorbell bits, which can be routed between any two
NT-functions. So the driver evenly distributes doorbell bits between all the
NT-function pairs. Of course it is not always possible being equally done, so
depending on the NT Primary/Secondary topology, one pair can have greater
number of Doorbell bits than another. The difference cannot exceed for more
than one bit.
IDT 89HPES*NT* PCIe-switches don't have Scrathpad registers, so the
corresponding callback methods are unsupported.
There are three DebugFS nodes, which can be used to debug the driver:
1. /sys/kernel/debug/ntb_hw_idt/ntbA{N}/info - contain a general information
about the local IDT NTB driver and NT-function, like port, partition, role
(primary or secondary) and number of connected peers, a link status, of all
the locally available pairs, global and local doorbell registers status,
doorbells mapping, NTB mapping table, chip teperature and many others.
2. /sys/kernel/debug/ntb_hw_idt/ntbA{N}/ntregs - debug file to print the state
of local NT-function registers. Thanks to the "enum-switch-case-macro" pattern
it can be easily done just by one loop.
3. /sys/kernel/debug/ntb_hw_idt/ntbA{N}/swregs - debug file to print the state
of PCIe-swtich global registers.
The detailed description of the driver design and source code navigation
can be found in the header of drivers/ntb/hw/idt/ntb_hw_idt.c file.
NOTE IDT 89HPES12NT3 PCIe-switch is not supported by this driver, since it has
synchronous interface.
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
drivers/ntb/hw/Kconfig | 1 +
drivers/ntb/hw/Makefile | 6 +-
drivers/ntb/hw/idt/Kconfig | 21 +
drivers/ntb/hw/idt/Makefile | 5 +
drivers/ntb/hw/idt/ntb_hw_idt.c | 4050 ++++++++++++++++++++++++++++++++
drivers/ntb/hw/idt/ntb_hw_idt.h | 390 +++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.c | 163 ++
drivers/ntb/hw/idt/ntb_hw_idt_quirks.h | 114 +
drivers/ntb/hw/idt/ntb_hw_idt_regmap.h | 877 +++++++
9 files changed, 5625 insertions(+), 2 deletions(-)
create mode 100644 drivers/ntb/hw/idt/Kconfig
create mode 100644 drivers/ntb/hw/idt/Makefile
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
create mode 100644 drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
diff --git a/drivers/ntb/hw/Kconfig b/drivers/ntb/hw/Kconfig
index 7116472..a89243c 100644
--- a/drivers/ntb/hw/Kconfig
+++ b/drivers/ntb/hw/Kconfig
@@ -1,2 +1,3 @@
source "drivers/ntb/hw/amd/Kconfig"
+source "drivers/ntb/hw/idt/Kconfig"
source "drivers/ntb/hw/intel/Kconfig"
diff --git a/drivers/ntb/hw/Makefile b/drivers/ntb/hw/Makefile
index 532e085..5d438ea 100644
--- a/drivers/ntb/hw/Makefile
+++ b/drivers/ntb/hw/Makefile
@@ -1,2 +1,4 @@
-obj-$(CONFIG_NTB_AMD) += amd/
-obj-$(CONFIG_NTB_INTEL) += intel/
+obj-$(CONFIG_NTB_AMD) += amd/
+obj-$(CONFIG_NTB_IDT) += idt/
+obj-$(CONFIG_NTB_IDT_QUIRKS) += idt/
+obj-$(CONFIG_NTB_INTEL) += intel/
diff --git a/drivers/ntb/hw/idt/Kconfig b/drivers/ntb/hw/idt/Kconfig
new file mode 100644
index 0000000..45b2b01
--- /dev/null
+++ b/drivers/ntb/hw/idt/Kconfig
@@ -0,0 +1,21 @@
+config NTB_IDT
+ tristate "IDT PCIe-switch Non-Transparent Bridge support"
+ depends on PCI
+ help
+ This driver supports NTB of cappable IDT PCIe-switches.
+
+ If unsure, say N.
+
+# BAR's early enable quirks
+config NTB_IDT_QUIRKS
+ bool "Enable PCI fixups built-in kernel for IDT PCIe-switch"
+ depends on NTB_IDT
+ default y if NTB_IDT=y
+ select PCI_QUIRKS # NTB_IDT depends on PCI so it's ok to force quirks
+ help
+ There are some configurations of IDT PCIe-switch which must be made
+ before the kernel PCI subsystem starts bus-devices scan and
+ initialization. For instance, the corresponding NT-function BARs must
+ be enabled. This option builds the fixups into the kernel to bypass
+ the basic IDT quirks.
+
diff --git a/drivers/ntb/hw/idt/Makefile b/drivers/ntb/hw/idt/Makefile
new file mode 100644
index 0000000..3adcd9b
--- /dev/null
+++ b/drivers/ntb/hw/idt/Makefile
@@ -0,0 +1,5 @@
+# Driver core
+obj-$(CONFIG_NTB_IDT) += ntb_hw_idt.o
+
+# BAR's early activation quirk
+obj-$(CONFIG_NTB_IDT_QUIRKS) += ntb_hw_idt_quirks.o
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt.c b/drivers/ntb/hw/idt/ntb_hw_idt.c
new file mode 100644
index 0000000..7562d55
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt.c
@@ -0,0 +1,4050 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the IDT PCIe-switch NT-function driver design.
+ * Here is presented some lirics about the NT-functions of the IDT PCIe-switch
+ * and the driver concept.
+ *
+ * There are a lot of different architectures or configurations the IDT
+ * PCIe-switch can be placed into, like NT Bridge-to-Bridge, Port-to-Port,
+ * Ports-to-Ports, Port-to-Ports, etc. But there is always BUT! Here it is.
+ * But the problem is that the PCIe-switch resources are not balanced enough
+ * to create efficient, the most comprehensive driver for Ports-to-Ports
+ * configuration. Here is what each IDT PCIe-switch have (IDT family of
+ * PCIe-switch solutions):
+ * - up to 24 Memory Windows per each port (incredibly a lot comparing to the
+ * Intel and AMD controllers)
+ * - one 32 bits Doorbell register shared amongst all the ports (Why IDT, why
+ * would you do that? Why so few?!)
+ * - 4 Message registers per each port (IDT, thanks at least for that...)
+ * - No Scratchpad registers (Surprise, huh?!)
+ *
+ * Since there are no scratchpad registers integrated in the IDT family PCI
+ * ExpressR switches, the tradition synchronous Linux NTB device can't be
+ * implemented (unlike Intel and AMD controllers, that are strictly synchronous).
+ * Instead the Messaging mechanism should be used to exchange the necessary
+ * informatin among the NT-functions. It leads to the asynchronous
+ * interface since there is no easy way to pass the address of the locally
+ * allocated shared memory window to the opposite NT-function. It can only be
+ * done by sending a message, which must be correcly handled by a peer. If one
+ * is looking for strictly synchronous solutions, then it's better to use Intel
+ * and AMD controllers. Regarding the IDT PreciseTM family of PCI ExpressR
+ * switches, they actually support both synchronous (scratchpads) and
+ * asynchronous (message registers) interfaces, but there is no suitable driver
+ * to use them in Linux.
+ *
+ * Lets get back to the actual driver architecture. Since there are no enough
+ * doorbell registers and after a lot of thoughts of the possible sidewalks to
+ * bypass the PCIe-switch limitations we came to the conclusion, that the best
+ * architecture of the driver using as much resources as possible would be the
+ * Port-to-Port/Port-to-Ports one. Shortly speaking it implies the only one
+ * NT-function being able to communicate with all the other NT-functions
+ * simultaniously. Suppose there are eight ports working as NT-bridge, then the
+ * Primary port would have 7 devices on the NTB bus, but the Secondary ports
+ * will expose just one device. As one can see it also perfectly fits the
+ * Primary-Secondary topology of the Linux NTB bus. The NTSDATA registers must
+ * be preinitialized with the corresponding Primary side port numbers. It is the
+ * way how the NTB topology can be configurated. For instance, suppose there are
+ * only two NT-functions enabled on the IDT PCIe-switch ports 0 and 2, where
+ * port 2 is chosen to be the primary one. Then all NTSDATA of the both
+ * NT-functions must be preinitialized with value 2. Similarly the topology with
+ * several Primary ports can be created.
+ *
+ * Howto study the code below.
+ * Here is the content of the driver:
+ * 1. IDT PCIe-switch registers IO-functions
+ * 2. Synchronization methods: atomic queue ops
+ * 3. Link status operations
+ * 4. Memory Window subsystem
+ * 5. Doorbells subsystem
+ * 6. Messaging subsystem
+ * 7. IRQ-related functions
+ * 8. NTB bus initialization
+ * 9. IDT NT-functions topology
+ * 10. Basic initialization functions
+ * 11. DebugFS callback functions
+ * 12. PCI bus callback functions
+ *
+ * I would recommend to start from the chapter "1. IDT PCIe-switch registers
+ * IO-functions". Since there are a lot of registers must be initialized before
+ * the switch starts working, it's better to have the register addresses and
+ * the corresponding values being collected at some structured table.
+ * Particulary one can find these tables in ntb_hw_idt_regmap.h file as the set
+ * of preprocessor macro-functions. Regarding the chapter 1 in this file, it
+ * resides the basic functions used to create the NT-functions and Switch Global
+ * registers table and the registers fields table. There are also r/w functions
+ * determined in there.
+ *
+ * Since there are list structures used to handle in and out messages, then
+ * there has to be managed synchronous access to those lists. Therefore the
+ * operations with message queues are made atomic in chapter "2. Synchronization
+ * methods: atomic queue ops".
+ *
+ * Then I would get stright to the chapter "12. PCI bus callback functions",
+ * which perform the algorithm of the PCI-bus device basic initialzation.
+ * Particulary it checks whether the kernel supports IDT PCIe-switch NTB
+ * devices, allocates the necessary structures, initialize the PCI-related
+ * fields, scans the IDT PCIe-switch NT-functions topology, adds all the
+ * available peers, initalizes doorbells, memory windows and messaging
+ * subsystem, starts link polling work-thread, initialize the interrupt
+ * handlers and finally registers the NTB devices on the NTB linux kernel bus.
+ *
+ * The basic PCI-bus device initialization and data structures allocation are
+ * performed by means of methods defined in the chapter "10. Basic
+ * initialization functions". NTB topology scanning is made by function from
+ * the chapter "9. IDT NT-functions topology".
+ *
+ * The NTB basic interfaces like Link event handlers, memory windows, doorbells
+ * and messages subsystems are described in the chapters 3 - 6 with corresponding
+ * titles. They traditionally consist of helpers, initializing/deinitializing
+ * functions and particular NTB devices kernel driver methods. These kernel
+ * driver methods - are callback functions used to register the new devices on
+ * the linux kernel NTB bus defined in the chapter "8. NTB bus initialization".
+ */
+
+/*#define DEBUG*/
+
+#include <linux/stddef.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/bitops.h>
+#include <linux/sizes.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/init.h>
+#include <linux/ntb.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/pci.h>
+#include <linux/aer.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+#include <linux/debugfs.h>
+
+#include "ntb_hw_idt.h"
+#include "ntb_hw_idt_regmap.h"
+#include "ntb_hw_idt_quirks.h"
+
+#define NTB_NAME "ntb_hw_idt"
+#define NTB_DESC "IDT PCI-E Non-Transparent Bridge Driver"
+#define NTB_VER "1.0"
+#define NTB_IRQNAME "idt_ntb_irq"
+#define NTB_WQNAME "idt_ntb_wq"
+#define NTB_CACHENAME "idt_ntb_cache"
+
+MODULE_DESCRIPTION(NTB_DESC);
+MODULE_VERSION(NTB_VER);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_ndev(ndev, args...) \
+ dev_err(to_dev_ndev(ndev), ## args)
+#define dev_err_data(data, args...) \
+ dev_err(to_dev_data(data), ## args)
+#define dev_warn_ndev(ndev, args...) \
+ dev_warn(to_dev_ndev(ndev), ## args)
+#define dev_warn_data(data, args...) \
+ dev_warn(to_dev_data(data), ## args)
+#define dev_info_ndev(ndev, args...) \
+ dev_info(to_dev_ndev(ndev), ## args)
+#define dev_info_data(data, args...) \
+ dev_info(to_dev_data(data), ## args)
+#define dev_dbg_ndev(ndev, args...) \
+ dev_dbg(to_dev_ndev(ndev), ## args)
+#define dev_dbg_data(data, args...) \
+ dev_dbg(to_dev_data(data), ## args)
+
+/*
+ * NT Endpoint ports table with the corresponding pcie link status, signal data,
+ * control and status registers
+ */
+static struct idt_ntb_port portdata_tbl[IDT_NTB_MAXPORTS_CNT] = {
+/*0*/ {IDT_SW_PCI_NTP0_CMD, IDT_SW_PCI_NTP0_PCIELSTS,
+ IDT_SW_PCI_NTP0_NTSDATA, IDT_SW_PCI_NTP0_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT0CTL, IDT_SW_PCI_SWPORT0STS},
+/*1*/ {0},
+/*2*/ {IDT_SW_PCI_NTP2_CMD, IDT_SW_PCI_NTP2_PCIELSTS,
+ IDT_SW_PCI_NTP2_NTSDATA, IDT_SW_PCI_NTP2_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT2CTL, IDT_SW_PCI_SWPORT2STS},
+/*3*/ {0},
+/*4*/ {IDT_SW_PCI_NTP4_CMD, IDT_SW_PCI_NTP4_PCIELSTS,
+ IDT_SW_PCI_NTP4_NTSDATA, IDT_SW_PCI_NTP4_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT4CTL, IDT_SW_PCI_SWPORT4STS},
+/*5*/ {0},
+/*6*/ {IDT_SW_PCI_NTP6_CMD, IDT_SW_PCI_NTP6_PCIELSTS,
+ IDT_SW_PCI_NTP6_NTSDATA, IDT_SW_PCI_NTP6_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT6CTL, IDT_SW_PCI_SWPORT6STS},
+/*7*/ {0},
+/*8*/ {IDT_SW_PCI_NTP8_CMD, IDT_SW_PCI_NTP8_PCIELSTS,
+ IDT_SW_PCI_NTP8_NTSDATA, IDT_SW_PCI_NTP8_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT8CTL, IDT_SW_PCI_SWPORT8STS},
+/*9*/ {0},
+/*10*/ {0},
+/*11*/ {0},
+/*12*/ {IDT_SW_PCI_NTP12_CMD, IDT_SW_PCI_NTP12_PCIELSTS,
+ IDT_SW_PCI_NTP12_NTSDATA, IDT_SW_PCI_NTP12_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT12CTL, IDT_SW_PCI_SWPORT12STS},
+/*13*/ {0},
+/*14*/ {0},
+/*15*/ {0},
+/*16*/ {IDT_SW_PCI_NTP16_CMD, IDT_SW_PCI_NTP16_PCIELSTS,
+ IDT_SW_PCI_NTP16_NTSDATA, IDT_SW_PCI_NTP16_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT16CTL, IDT_SW_PCI_SWPORT16STS},
+/*17*/ {0},
+/*18*/ {0},
+/*19*/ {0},
+/*20*/ {IDT_SW_PCI_NTP20_CMD, IDT_SW_PCI_NTP20_PCIELSTS,
+ IDT_SW_PCI_NTP20_NTSDATA, IDT_SW_PCI_NTP20_NTGSIGNAL,
+ IDT_SW_PCI_SWPORT20CTL, IDT_SW_PCI_SWPORT20STS},
+/*21*/ {0},
+/*22*/ {0},
+/*23*/ {0}
+};
+
+/*
+ * IDT PCIe-switch partitions table with the corresponding control, status
+ * and messages control registers
+ */
+static struct idt_ntb_part partdata_tbl[IDT_NTB_MAXPARTS_CNT] = {
+/*0*/ { IDT_SW_PCI_SWPART0CTL, IDT_SW_PCI_SWPART0STS,
+ {IDT_SW_PCI_SWP0MSGCTL0, IDT_SW_PCI_SWP0MSGCTL1,
+ IDT_SW_PCI_SWP0MSGCTL2, IDT_SW_PCI_SWP0MSGCTL3} },
+/*1*/ { IDT_SW_PCI_SWPART1CTL, IDT_SW_PCI_SWPART1STS,
+ {IDT_SW_PCI_SWP1MSGCTL0, IDT_SW_PCI_SWP1MSGCTL1,
+ IDT_SW_PCI_SWP1MSGCTL2, IDT_SW_PCI_SWP1MSGCTL3} },
+/*2*/ { IDT_SW_PCI_SWPART2CTL, IDT_SW_PCI_SWPART2STS,
+ {IDT_SW_PCI_SWP2MSGCTL0, IDT_SW_PCI_SWP2MSGCTL1,
+ IDT_SW_PCI_SWP2MSGCTL2, IDT_SW_PCI_SWP2MSGCTL3} },
+/*3*/ { IDT_SW_PCI_SWPART3CTL, IDT_SW_PCI_SWPART3STS,
+ {IDT_SW_PCI_SWP3MSGCTL0, IDT_SW_PCI_SWP3MSGCTL1,
+ IDT_SW_PCI_SWP3MSGCTL2, IDT_SW_PCI_SWP3MSGCTL3} },
+/*4*/ { IDT_SW_PCI_SWPART4CTL, IDT_SW_PCI_SWPART4STS,
+ {IDT_SW_PCI_SWP4MSGCTL0, IDT_SW_PCI_SWP4MSGCTL1,
+ IDT_SW_PCI_SWP4MSGCTL2, IDT_SW_PCI_SWP4MSGCTL3} },
+/*5*/ { IDT_SW_PCI_SWPART5CTL, IDT_SW_PCI_SWPART5STS,
+ {IDT_SW_PCI_SWP5MSGCTL0, IDT_SW_PCI_SWP5MSGCTL1,
+ IDT_SW_PCI_SWP5MSGCTL2, IDT_SW_PCI_SWP5MSGCTL3} },
+/*6*/ { IDT_SW_PCI_SWPART6CTL, IDT_SW_PCI_SWPART6STS,
+ {IDT_SW_PCI_SWP6MSGCTL0, IDT_SW_PCI_SWP6MSGCTL1,
+ IDT_SW_PCI_SWP6MSGCTL2, IDT_SW_PCI_SWP6MSGCTL3} },
+/*7*/ { IDT_SW_PCI_SWPART7CTL, IDT_SW_PCI_SWPART7STS,
+ {IDT_SW_PCI_SWP7MSGCTL0, IDT_SW_PCI_SWP7MSGCTL1,
+ IDT_SW_PCI_SWP7MSGCTL2, IDT_SW_PCI_SWP7MSGCTL3} }
+};
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_topdir;
+
+/*===========================================================================
+ * 1. IDT PCIe-switch registers IO-functions
+ *===========================================================================*/
+
+static void __idt_nt_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val);
+static u32 __idt_nt_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize);
+static void __idt_sw_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val);
+static u32 __idt_sw_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize);
+
+/*
+ * Registers IO contexts to perform the r/w operations either with NT-function
+ * registers or with the PCIe-switch Global registers. The context is chosen
+ * by the register type "enum idt_ntb_regtype"
+ */
+static struct idt_ntb_regctx regctx[2] = {
+ {.writereg = __idt_nt_writereg, .readreg = __idt_nt_readreg,
+ .iolock = __SPIN_LOCK_UNLOCKED(iolock)},
+ {.writereg = __idt_sw_writereg, .readreg = __idt_sw_readreg,
+ .iolock = __SPIN_LOCK_UNLOCKED(iolock)}
+};
+
+/*
+ * Internal function to set the value bits of a variable
+ */
+static inline u32 idt_ntb_setbits(u32 var, u32 mask, unsigned char offset, u32 val)
+{
+ return (var & ~(mask << offset)) | ((val & mask) << offset);
+}
+
+/*
+ * Internal function to retrieve the value bits of a variable
+ */
+static inline u32 idt_ntb_getbits(u32 var, u32 mask, unsigned char offset)
+{
+ return (var >> offset) & mask;
+}
+
+/*
+ * Retrieve the register type, address and size by the passed enumerated ID
+ * NOTE Compiler should produce the jump table for the subsequent switch-case
+ * statement which gives just simple o(1) complexity
+ */
+static int idt_ntb_regparams(const enum idt_ntb_cfgreg reg,
+ enum idt_ntb_regtype *type, ptrdiff_t *offset,
+ enum idt_ntb_regsize *size, const char **desc)
+{
+ const char *tmpdesc;
+
+ /* Determine the register type */
+ *type = (IDT_NTB_CFGREGS_SPLIT > reg) ? IDT_NT_REGTYPE : IDT_SW_REGTYPE;
+
+ /* Retrieve the register parameters by the enumerated ID */
+ switch (reg) {
+ IDT_NT_CFGREGS(PAIR_REGID_ACCESS, *offset, *size, tmpdesc)
+ IDT_SW_CFGREGS(PAIR_REGID_ACCESS, *offset, *size, tmpdesc)
+ default :
+ /* Got invalid register ID */
+ BUG();
+ return -EINVAL;
+ }
+
+ /* Return the pointer to the string with the register description
+ * only if the passed pointer isn't NULL*/
+ if (NULL != desc) {
+ *desc = tmpdesc;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Retrieve the registers fields parameters like the register id, mask
+ * and offset
+ * NOTE Compiler should produce the jump table for the subsequent switch-case
+ * statement which gives just simple o(1) complexity
+ */
+static int idt_ntb_fldparams(const enum idt_ntb_regfld fld,
+ enum idt_ntb_cfgreg *reg,
+ u32 *mask, unsigned char *offset)
+{
+ /* Retrieve the field parameters by the enumerated ID */
+ switch (fld) {
+ IDT_NT_REGFLDS(PAIR_FLDID_ACCESS, *reg, *mask, *offset)
+ IDT_SW_REGFLDS(PAIR_FLDID_ACCESS, *reg, *mask, *offset)
+ default :
+ /* Got invalid register field ID */
+ BUG();
+ return -EINVAL;
+ }
+ return SUCCESS;
+}
+
+/*
+ * Set the corresponding field of the passed variable
+ */
+static void idt_ntb_writefld_var(u32 *var, const enum idt_ntb_regfld fld,
+ const u32 val)
+{
+ enum idt_ntb_cfgreg reg;
+ unsigned char bitoffset;
+ u32 mask;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Init the corresponding bits of the passed variable */
+ *var = idt_ntb_setbits(*var, mask, bitoffset, val);
+}
+
+/*
+ * Get the corresponding field of the passed variable
+ */
+static u32 idt_ntb_readfld_var(u32 var, const enum idt_ntb_regfld fld)
+{
+ enum idt_ntb_cfgreg reg;
+ unsigned char bitoffset;
+ u32 mask;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Retrieve the corresponding field of the variable */
+ return idt_ntb_getbits(var, mask, bitoffset);
+}
+
+/*
+ * NT-function registers basic write function
+ *
+ * WARNING! Our target platform is Big Endian, but PCI registers are always
+ * Little endian. So corresponding write{w,l} operations must have
+ * embedded endiannes conversion. If your platform doesn't have it,
+ * the driver won't properly work.
+ */
+static void __idt_nt_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val)
+{
+ /* Perform fast IO operation */
+ switch (regsize) {
+ case REGBYTE:
+ writeb((u8)val, cfg_mmio + regoffset);
+ break;
+ case REGWORD:
+ writew((u16)val, cfg_mmio + regoffset);
+ break;
+ case REGDWORD:
+ writel((u32)val, cfg_mmio + regoffset);
+ break;
+ default:
+ /* Invalid register size was retrieved */
+ BUG();
+ break;
+ }
+}
+
+/*
+ * NT-function registers basic read function
+ *
+ * WARNING! Our target platform is Big Endian, but PCI registers are always
+ * Little endian. So corresponding read{w,l} operations must have
+ * embedded endiannes conversion. If your platform doesn't have it,
+ * the driver won't properly work.
+ */
+static u32 __idt_nt_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize)
+{
+ u32 retval;
+
+ /* Perform fast IO operation */
+ switch (regsize) {
+ case REGBYTE:
+ retval = readb(cfg_mmio + regoffset);
+ break;
+ case REGWORD:
+ retval = readw(cfg_mmio + regoffset);
+ break;
+ case REGDWORD:
+ retval = readl(cfg_mmio + regoffset);
+ break;
+ default:
+ /* Invalid register size was retrieved */
+ BUG();
+ break;
+ }
+
+ return retval;
+}
+
+/*
+ * IDT PCIe-switch Global registers basic write function
+ *
+ * WARNING! Our target platform is Big Endian, but PCI registers are always
+ * Little endian. So corresponding write{w,l} operations must have
+ * embedded endiannes conversion. If your platform doesn't have it,
+ * the driver won't properly work.
+ * In addition the GASA* registers support the 4 bytes R/W operations
+ * so the data must be correspondingly shifted
+ */
+static void __idt_sw_writereg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val)
+{
+ u32 data, fldmask;
+ unsigned char fldoffset;
+
+ /* Post the IDT PCIe-switch register offset first */
+ writel((u32)regoffset, cfg_mmio + GASAADDR_OFFSET);
+
+ /* Read the data of the passed register, which offset is aligned with
+ * two last bits by the GASAADDR register */
+ data = readl(cfg_mmio + GASADATA_OFFSET);
+
+ /* Alter the corresponding field of the data with the passed value */
+ fldmask = GENMASK(BITS_PER_BYTE * regsize - 1, 0);
+ fldoffset = BITS_PER_BYTE * (regoffset & 0x3);
+ data = idt_ntb_setbits(data, fldmask, fldoffset, val);
+
+ /* Whatever the size of the register is, just write the value to the
+ * data register */
+ writel(data, cfg_mmio + GASADATA_OFFSET);
+}
+
+/*
+ * IDT PCIe-switch Global registers basic read function
+ */
+static u32 __idt_sw_readreg(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize)
+{
+ u32 data, fldmask;
+ unsigned char fldoffset;
+
+ /* Post the IDT PCIe-switch register offset first */
+ writel((u32)regoffset, cfg_mmio + GASAADDR_OFFSET);
+
+ /* Read the data of the passed register, which offset is aligned with
+ * two last bits by the GASAADDR register */
+ data = readl(cfg_mmio + GASADATA_OFFSET);
+
+ /* Alter the corresponding field of the data with the passed value */
+ fldmask = GENMASK(BITS_PER_BYTE * regsize - 1, 0);
+ fldoffset = BITS_PER_BYTE * (regoffset & 0x3);
+ data = idt_ntb_getbits(data, fldmask, fldoffset);
+
+ /* Return the corresponding field of the register */
+ return data;
+}
+
+/*
+ * General function to perform the write operation to the register
+ */
+static void idt_ntb_writereg(void __iomem *cfg_mmio,
+ const enum idt_ntb_cfgreg reg, const u32 val)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_regtype regtype;
+ ptrdiff_t regoffset;
+ enum idt_ntb_regsize regsize;
+ unsigned long irqflags;
+
+ /* Retrieve the register type, offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast write operation */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ curctx->writereg(cfg_mmio, regoffset, regsize, val);
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+}
+
+/*
+ * General function to perform the read operation from the register
+ */
+static u32 idt_ntb_readreg(void __iomem *cfg_mmio, const enum idt_ntb_cfgreg reg)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_regtype regtype;
+ ptrdiff_t regoffset;
+ enum idt_ntb_regsize regsize;
+ unsigned long irqflags;
+ u32 val;
+
+ /* Retrieve the register type, offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast read operation */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ val = curctx->readreg(cfg_mmio, regoffset, regsize);
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+
+ return val;
+}
+
+/*
+ * General function to perform the write operation to the field of the register
+ */
+static void idt_ntb_writefld_mem(void __iomem *cfg_mmio,
+ const enum idt_ntb_regfld fld, const u32 val)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_cfgreg reg;
+ enum idt_ntb_regsize regsize;
+ ptrdiff_t regoffset;
+ unsigned char bitoffset;
+ u32 mask, curval;
+ enum idt_ntb_regtype regtype;
+ unsigned long irqflags;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Retrieve the register offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register set context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast IO operations */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ /* Retrieve the current value of the register */
+ curval = curctx->readreg(cfg_mmio, regoffset, regsize);
+ /* Set the corresponding bits in there */
+ curval = idt_ntb_setbits(curval, mask, bitoffset, val);
+ /* Write the register value back */
+ curctx->writereg(cfg_mmio, regoffset, regsize, val);
+ /* The critical section is over */
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+}
+
+/*
+ * General function to perform the read operation from the field of the register
+ */
+static u32 idt_ntb_readfld_mem(void __iomem *cfg_mmio,
+ const enum idt_ntb_regfld fld)
+{
+ struct idt_ntb_regctx *curctx;
+ enum idt_ntb_cfgreg reg;
+ enum idt_ntb_regsize regsize;
+ ptrdiff_t regoffset;
+ unsigned char bitoffset;
+ u32 mask, curval;
+ enum idt_ntb_regtype regtype;
+ unsigned long irqflags;
+
+ /* Retrieve the field parameters */
+ idt_ntb_fldparams(fld, ®, &mask, &bitoffset);
+
+ /* Retrieve the register offset and size */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, NULL);
+
+ /* Get the current register set context */
+ curctx = ®ctx[regtype];
+
+ /* Perform fast IO operations */
+ spin_lock_irqsave(&curctx->iolock, irqflags);
+ /* Retrieve the current value of the register */
+ curval = curctx->readreg(cfg_mmio, regoffset, regsize);
+ /* The critical section is over */
+ spin_unlock_irqrestore(&curctx->iolock, irqflags);
+
+ return idt_ntb_getbits(curval, mask, bitoffset);
+}
+
+/*===========================================================================
+ * 2. Synchronization methods: atomic queue ops
+ *===========================================================================*/
+
+/*
+ * Initialize the atomic queue structure
+ */
+static inline void atomic_queue_init(queue_atomic_t *queue)
+{
+ /* Init the queue head */
+ INIT_LIST_HEAD(&queue->head);
+
+ /* Initialize the spin lock protecting the queue head */
+ spin_lock_init(&queue->lock);
+}
+
+/*
+ * Add item to the atomic queue at the first position
+ */
+static inline void atomic_queue_add(queue_atomic_t *queue,
+ struct list_head *new)
+{
+ unsigned long irqflags;
+
+ /* Lock the list add operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ list_add(new, &queue->head);
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+}
+
+/*
+ * Add item to the atomic queue tail
+ */
+static inline void atomic_queue_add_tail(queue_atomic_t *queue,
+ struct list_head *new)
+{
+ unsigned long irqflags;
+
+ /* Lock the list add tail operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ list_add_tail(new, &queue->head);
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+}
+
+/*
+ * Get the very first entry from the atomic queue
+ */
+static inline struct list_head *atomic_queue_get(queue_atomic_t *queue)
+{
+ struct list_head *entry;
+ unsigned long irqflags;
+
+ /* Lock the list entry delete operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ if (!list_empty(&queue->head)) {
+ entry = queue->head.next;
+ list_del(entry);
+ } else /* if (entry != &queue->head) */ {
+ entry = NULL;
+ }
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+
+ return entry;
+}
+
+/*
+ * Check whether the atomic queue is empty
+ */
+static inline bool atomic_queue_empty(queue_atomic_t *queue)
+{
+ unsigned long irqflags;
+ bool ret;
+
+ /* Lock the list empty operation */
+ spin_lock_irqsave(&queue->lock, irqflags);
+ ret = list_empty(&queue->head);
+ spin_unlock_irqrestore(&queue->lock, irqflags);
+
+ return ret;
+}
+
+/*===========================================================================
+ * 3. Link status operations
+ *===========================================================================*/
+
+/*
+ * Effectively enable the NTB link.
+ *
+ * From the moment of return from this function the inter-partition
+ * communications are enabled as well as translating Request and Complition TLPs.
+ * This function is called by the Primary side on the initialization phase. The
+ * Secondary ports can invoke it by calling the ntb_link_enable() callback.
+ */
+static void idt_ntb_link_effective_enable(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntctl = 0, reqid, ntmtbldata = 0;
+
+ /* Retrieve the current complex Requester ID (Bus:Device:Function) */
+ reqid = idt_ntb_readfld_mem(cfg, IDT_NT_MTBL_REQID);
+
+ /* Set the corresponding NT Mapping table entry of port partition index
+ * with the data to perform the Request ID translation */
+ idt_ntb_writefld_var(&ntmtbldata, IDT_NT_MTBL_BDF, reqid);
+ idt_ntb_writefld_var(&ntmtbldata, IDT_NT_MTBL_PART, (u32)pdata->part);
+ idt_ntb_writefld_var(&ntmtbldata, IDT_NT_MTBL_VALID, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLADDR, (u32)pdata->part);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLDATA, ntmtbldata);
+
+ /* Enable the ID protection and Completion TLPs translation */
+ idt_ntb_writefld_var(&ntctl, IDT_NT_IDPROTDIS, OFF);
+ idt_ntb_writefld_var(&ntctl, IDT_NT_CPEN, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTCTL, ntctl);
+
+ /* Enable the bus mastering, which effectively enables the Request TLPs
+ * translation and MSI IRQs generation */
+ pci_set_master(pdata->pdev);
+
+ /* The ndevs->lnk_sts variable is going to change in the work thread */
+}
+
+/*
+ * Effectively disable the NTB link.
+ *
+ * From the moment of return from this function the inter-partition
+ * communications are disabled.
+ */
+static void idt_ntb_link_effective_disable(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+
+ /* Disable the bus mastering, which effectively stops translating the
+ * Request TLPs across the boundary of local partition */
+ pci_clear_master(pdata->pdev);
+
+ /* Disable Completion TLPs */
+ idt_ntb_writefld_mem(cfg, IDT_NT_CPEN, OFF);
+
+ /* Disable the corresponding NT Mapping table entry */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLADDR, (u32)pdata->part);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLDATA, (u32)OFF);
+
+ /* The ndevs->lnk_sts variable is going to change in the work thread */
+}
+
+/*
+ * Notify the peer device that the local side is ready.
+ *
+ * Since the Primary side can't enable/disable link by demand of the client
+ * driver, there should be some way to notify the opposite side, what the local
+ * client driver is installed and started working (by calling the
+ * ntb_enable_link method). So Global Signal register is used for that purpose.
+ */
+static void idt_ntb_link_notify(struct idt_ntb_dev *ndev)
+{
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Just write ON to the first bit of device NTGSIGNAL register
+ * It is available only using GASA* registers */
+ idt_ntb_writereg(cfg, portdata_tbl[ndev->port].ntgsignal, ON);
+}
+
+/*
+ * Clear the notification set before in the Global Signal Status register.
+ */
+static void idt_ntb_link_clear_notification(struct idt_ntb_dev *ndev)
+{
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Clear the Global Signal status bit of the device partition */
+ idt_ntb_writereg(cfg, IDT_SW_PCI_SEGSIGSTS, ((u32)1 << ndev->part));
+}
+
+/*
+ * Retrieve the current link status
+ */
+static int idt_ntb_link_status(struct idt_ntb_dev *ndev)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 localbme, peerbme, pciests, gsigsts;
+ unsigned int part;
+
+ /* Read the local Bus Master Enable status */
+ localbme = idt_ntb_readfld_mem(cfg, IDT_NT_BME);
+
+ /* Read the Global Signal Status bit related to the device partition */
+ gsigsts = idt_ntb_readreg(cfg, IDT_SW_PCI_SEGSIGSTS);
+ /* Retrieve the partition of the corresponding device */
+ part = (NTB_TOPO_PRI == pdata->role) ? ndev->part : pdata->part;
+ gsigsts = (gsigsts & ((u32)1 << part)) ? ON : OFF;
+
+ /* Read the peer Bus Master Enable status */
+ peerbme = idt_ntb_readreg(cfg, portdata_tbl[ndev->port].pcicmd);
+ peerbme = idt_ntb_readfld_var(peerbme, IDT_NT_BME);
+
+ /* Retrieve the peer port link status */
+ pciests = idt_ntb_readreg(cfg, portdata_tbl[ndev->port].sts);
+ pciests = idt_ntb_readfld_var(pciests, IDT_SW_PORT_LNKUP);
+
+ /* If Both BME fields are ON and PCIe data link is up then the NTB
+ * link is effectively up */
+ if (ON == pciests && ON == peerbme && ON == localbme && ON == gsigsts) {
+ return ON;
+ } /* else if (OFF == pciests || OFF == peerbme || Off == localbme ||
+ * OFF == gsigsts) {return OFF} */
+
+ return OFF;
+}
+
+/*
+ * Kernel thread polling the peer side link status by reading the corresponding
+ * PCIe link status register and NT Mapping table entry
+ */
+static void idt_ntb_poll_link_work(struct work_struct *work)
+{
+ struct idt_ntb_data *pdata = to_data_lnkwork(work);
+ struct idt_ntb_dev *ndev;
+ unsigned char id;
+ int curlnksts;
+
+ /* Walk through all available peers reading their status */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Get the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Retrieve the current link status */
+ curlnksts = idt_ntb_link_status(ndev);
+
+ /* If the link status has changed then call the event handler */
+ if (curlnksts != ndev->lnk_sts) {
+ ndev->lnk_sts = curlnksts;
+ ntb_link_event(&ndev->ntb);
+ }
+ }
+
+ /* Reschedule the work */
+ (void)queue_delayed_work(pdata->idt_wq, &pdata->lnk_work,
+ IDT_NTB_LNKPOLL_TOUT);
+}
+
+/*
+ * Initialize NTB link subsystem
+ *
+ * NOTE This function is not used by the client driver but just for
+ * initialization
+ */
+static void idt_ntb_init_link(struct idt_ntb_data *pdata)
+{
+ unsigned char id;
+
+ /* Initialize all the peers link status with OFF */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ pdata->ndevs[id].lnk_sts = OFF;
+ }
+
+ /* Enable the link if it's primary port */
+ if (NTB_TOPO_PRI == pdata->role) {
+ /* Clear all the Global Signal Status bits related to the
+ * locally available NTB device */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ idt_ntb_link_clear_notification(&pdata->ndevs[id]);
+ }
+ /* Next function enables the whole link no matter which NTB
+ * device it's */
+ idt_ntb_link_effective_enable(pdata);
+ }
+
+ /* Initialize the delayed kernel thread polling the link status */
+ INIT_DELAYED_WORK(&pdata->lnk_work, idt_ntb_poll_link_work);
+ (void)queue_delayed_work(pdata->idt_wq, &pdata->lnk_work,
+ IDT_NTB_LNKPOLL_TOUT);
+
+ dev_dbg_data(pdata, "IDT NTB peer device link polling started");
+}
+
+/*
+ * Clear the link polling subsystem
+ *
+ * NOTE This function is not used by the client driver but just for
+ * final deinitialization
+ */
+static void idt_ntb_clear_link(struct idt_ntb_data *pdata)
+{
+ unsigned char id;
+
+ /* Stop the link status polling thread */
+ cancel_delayed_work_sync(&pdata->lnk_work);
+
+ /* Disable the link */
+ idt_ntb_link_effective_disable(pdata);
+
+ /* Clear all the Global Signal Status bits related to the
+ * Primary port available NTB device */
+ if (NTB_TOPO_PRI == pdata->role) {
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ idt_ntb_link_clear_notification(&pdata->ndevs[id]);
+ }
+ }
+
+ dev_dbg_data(pdata, "IDT NTB peer device link polling stopped");
+}
+
+/*
+ * NTB bus callback - get the current ntb link state
+ */
+static int idt_ntb_link_is_up(struct ntb_dev *ntb, enum ntb_speed *speed,
+ enum ntb_width *width)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 pcielsts;
+ int lnksts;
+
+ /* Get the curret link status */
+ lnksts = idt_ntb_link_status(ndev);
+
+ /* Retrieve the PCIe data link parameters */
+ if (ON == lnksts) {
+ /* Read the PCIe link status */
+ pcielsts = idt_ntb_readreg(cfg,
+ portdata_tbl[ndev->port].pcielsts);
+ /* The register values numerically match the enum values */
+ if (speed) {
+ *speed = idt_ntb_readfld_var(pcielsts, IDT_NT_CURLNKSPD);
+ }
+ if (width) {
+ *width = idt_ntb_readfld_var(pcielsts, IDT_NT_CURLNKWDTH);
+ }
+ } else /* if (OFF == lnksts) */ {
+ if (speed) {
+ *speed = NTB_SPEED_NONE;
+ }
+ if (width) {
+ *width = NTB_WIDTH_NONE;
+ }
+ }
+
+ return lnksts;
+}
+
+/*
+ * NTB bus callback - enable the link on the secondary side of the ntb
+ *
+ * NOTE Since there can be more than one pair of NTB devices (we use shared
+ * Lookup table) on the Primary port, the link must be always enabled from that
+ * side. So the next function fully works from the Secondary side only.
+ */
+static int idt_ntb_link_enable(struct ntb_dev *ntb, enum ntb_speed speed,
+ enum ntb_width width)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = pdata->cfg_mmio;
+
+ /* Primary port driver enables the link in the initialization method */
+ if (NTB_TOPO_PRI == ntb->topo) {
+ /* Notify the opposite side, that the link is enabled */
+ idt_ntb_link_notify(ndev);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is virtually enabled");
+
+ return -EINVAL;
+ }
+
+ /* Secondary ports can effectively enable the link on the local side */
+ idt_ntb_link_effective_enable(pdata);
+
+ /* Enable the interrupts of message, doorbells, switch and temperature
+ * sensor events. This will generate all the pending interrupts after the
+ * link is effectively enabled */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, NTINT_UNMASK);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is enabled");
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - disable the link on the secondary side of the ntb
+ */
+static int idt_ntb_link_disable(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = pdata->cfg_mmio;
+
+ /* Primary port driver disables the link in the link clear method */
+ if (NTB_TOPO_PRI == ntb->topo) {
+ /* Notify the opposite side, that the link is disabled */
+ idt_ntb_link_clear_notification(ndev);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is virtually disabled");
+
+ return -EINVAL;
+ }
+
+ /* Disable the interrupts of message, doorbells, switch and temperature
+ * sensor events. This will stop generateing interrupts while link is
+ * down */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, NTINT_MASK);
+
+ /* Secondary ports can effectively disable the link on the local side */
+ idt_ntb_link_effective_disable(pdata);
+
+ dev_dbg_ndev(ndev, "IDT NT-function link is disabled");
+
+ return SUCCESS;
+}
+
+/*===========================================================================
+ * 4. Memory Window subsystem
+ *===========================================================================*/
+
+/*
+ * Find the Secondary port serial number (id) by the passed primary and
+ * secondary ports
+ */
+static inline unsigned char idt_ntb_findid(struct idt_ntb_topo *topo,
+ unsigned char pri, unsigned char sec)
+{
+ return hweight32(topo->secports[pri] & (((u32)1 << sec) - 1));
+}
+
+/*
+ * Initialize the PCI device BAR2(3:x64) setup register
+ */
+static int idt_ntb_setup_bar2(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ phys_addr_t limit;
+ int ret;
+
+ /* Request the PCI resources for the BAR2(3) */
+ ret = pci_request_region(pdata->pdev, BAR2, NTB_NAME);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Failed to request the PCI BAR2(3) resources");
+ return ret;
+ }
+
+ /* Retrieve the physical address of the mapped by the Lookup table
+ * shared memory - BAR2(3) */
+ pdata->mw_base = pci_resource_start(pdata->pdev, BAR2);
+
+ /* Limit the BAR2 address with resepect to the Lookup table boundary */
+ /* Calculate the size of just one Memory Window */
+ pdata->mw_size = pci_resource_len(pdata->pdev, BAR2)/32;
+
+ /* Find the limit address */
+ limit = pdata->mw_base + IDT_NTB_MW_CNT * pdata->mw_size - 1;
+
+ /* Set the BAR size limiting register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT2, (u32)limit);
+#ifdef CONFIG_64BIT
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT3, (u32)(limit >> 32));
+#endif /* CONFIG_64BIT */
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the PCI device BAR2(3:x64) setup register
+ */
+static void idt_ntb_clean_bar2(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 limit = -1;
+
+ /* Set the BAR size limiting register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT2, limit);
+#ifdef CONFIG_64BIT
+ idt_ntb_writereg(cfg, IDT_NT_PCI_BARLIMIT3, limit);
+#endif /* CONFIG_64BIT */
+
+ /* Just write the disabled BARSETUP0 */
+ pci_release_region(pdata->pdev, BAR2);
+}
+
+/*
+ * Set the Memory Window translation address for the passed peer NTB device
+ */
+static int idt_ntb_setmw(struct idt_ntb_dev *ndev, const int mwindx,
+ const dma_addr_t addr)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 lut_indxbar = 0, lut_partval = 0;
+ unsigned long irqflags;
+
+ /* Return error if the passed memory window index is out of range */
+ if (mwindx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid Memory Window index specified to set");
+ return -EINVAL;
+ }
+
+ /* Return error if the passed address is not aligned with the four
+ * bytes */
+ if (!IS_ALIGNED(addr, IDT_NTB_TRANSALIGN)) {
+ dev_err_ndev(ndev, "Translated base address is not aligned");
+ return -EINVAL;
+ }
+
+ /* Collect the Lookup table offset */
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_INDEX,
+ ndev->mw_self_offset + mwindx);
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_BAR, BAR2);
+
+ /* Collect the Lookup table entry partition and valid bits */
+ idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_PART, ndev->part);
+ idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_VALID, ON);
+
+ /* Start critical section writing to the local port Lookup table */
+ spin_lock_irqsave(&pdata->lut_lock, irqflags);
+ /* Write the data to the Lookup table registers of the peer */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTOFFSET, lut_indxbar);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTLDATA, (u32)addr);
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)(addr >> 32));
+#else
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)0);
+#endif /* !CONFIG_ARCH_DMA_ADDR_T_64BIT */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTUDATA, lut_partval);
+ /* Finally unlock the Lookup table */
+ spin_unlock_irqrestore(&pdata->lut_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * Set the Memory Window translation address for the passed peer NTB device
+ */
+static int idt_ntb_unsetmw(struct idt_ntb_dev *ndev, const int mwindx)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 lut_indxbar = 0, lut_partval = 0;
+ unsigned long irqflags;
+
+ /* Return Error if the passed Memory Window index is out of range */
+ if (mwindx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid Memory Window index specified to unset");
+ return -EINVAL;
+ }
+
+ /* Collect the Lookup table offset */
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_INDEX,
+ ndev->mw_self_offset + mwindx);
+ idt_ntb_writefld_var(&lut_indxbar, IDT_NT_LUT_BAR, BAR2);
+
+ /* Collect the Lookup table entry partition and valid bits */
+ idt_ntb_writefld_var(&lut_partval, IDT_NT_LUT_VALID, OFF);
+
+ /* Start critical section writing to the Lookup table */
+ spin_lock_irqsave(&pdata->lut_lock, irqflags);
+ /* Write the data to the Lookup table registers of the peer */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTOFFSET, lut_indxbar);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTLDATA, (u32)0);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTMDATA, (u32)0);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_LUTUDATA, lut_partval);
+ /* Finally unlock the Lookup table */
+ spin_unlock_irqrestore(&pdata->lut_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * Cleanup the local Lookup table
+ */
+static int idt_ntb_cleanlut(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_dev *ndev;
+ unsigned char id, mw;
+ int ret;
+
+ /* Walk through all the available peers */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ ndev = &pdata->ndevs[id];
+
+ /* Unset all the local memory windows */
+ for (mw = 0; mw < ndev->mw_self_cnt; mw++) {
+ ret = idt_ntb_unsetmw(ndev, mw);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+ }
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Initialize the Memory Windows for the current NT-function with respect to the
+ * topologically predefined NTB pairs
+ *
+ * NOTE The first NTB pairs are lucky to have the extended set of Memory Windows
+ */
+static int idt_ntb_init_mws(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_topo *topo = &pdata->topo;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ unsigned char id, mwcnt, luckies, curoffset;
+ int ret;
+
+ /* Calculate the number of Memory Windows per NTB */
+ mwcnt = IDT_NTB_MW_CNT / topo->paircnt;
+ luckies = IDT_NTB_MW_CNT % topo->paircnt;
+
+ /* Find the memory windows local and peer parameters */
+ if (NTB_TOPO_PRI == pdata->role) {
+ /* Loop over all the locally available peers */
+ curoffset = 0;
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Find the memory windows offset and count */
+ ndevs[id].mw_self_offset = curoffset;
+ ndevs[id].mw_self_cnt = mwcnt + (luckies > id ? 1 : 0);
+ ndevs[id].mw_peer_cnt = IDT_NTB_MW_CNT;
+
+ /* Get the offset for the next Memory Windows */
+ curoffset += ndevs[id].mw_self_cnt;
+ }
+ } else /* if (NTB_TOPO_SEC == pdata->role) */ {
+ id = ndevs[0].pairid;
+ ndevs[0].mw_self_offset = 0;
+ ndevs[0].mw_self_cnt = IDT_NTB_MW_CNT;
+ ndevs[0].mw_peer_cnt = mwcnt + (luckies > id ? 1 : 0);
+ }
+
+ /* Initialize the BAR2(3) related registers and data fields */
+ ret = idt_ntb_setup_bar2(pdata);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+
+ /* Initialize the Lookup table spinlock*/
+ spin_lock_init(&pdata->lut_lock);
+
+ /* Cleanup the Lookup table */
+ (void)idt_ntb_cleanlut(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB device memory windows redistributed");
+
+ return SUCCESS;
+}
+
+/*
+ * Clean the Memory Windows initialized for the current NT-function
+ */
+static void idt_ntb_clean_mws(struct idt_ntb_data *pdata)
+{
+ /* Cleanup the peers Lookup tables */
+ (void)idt_ntb_cleanlut(pdata);
+
+ /* Clean the BAR2(3) */
+ idt_ntb_clean_bar2(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB function memory windows cleaned");
+}
+
+/*
+ * NTB bus callback - local memory windows count
+ */
+static int idt_ntb_mw_count(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Return the number of available local memory windows */
+ return ndev->mw_self_cnt;
+}
+
+/*
+ * NTB bus callback - get the map resource of a memory window
+ */
+static int idt_ntb_mw_get_maprsc(struct ntb_dev *ntb, int idx, phys_addr_t *base,
+ resource_size_t *size)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+
+ /* It's error to pass the out of range Memory Window index */
+ if (idx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid memory window index passed to get map res");
+ return -EINVAL;
+ }
+
+ /* The base address is determined with respect to the Lookup table
+ * table offset */
+ if (base)
+ *base = pdata->mw_base +
+ (ndev->mw_self_offset + idx) * pdata->mw_size;
+ if (size)
+ *size = pdata->mw_size;
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - get the local memory windows alignments
+ */
+static int idt_ntb_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+
+ /* It's error to pass the out of range Memory Window index */
+ if (idx >= ndev->mw_self_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid memory window index passed to get alignment");
+ return -EINVAL;
+ }
+
+ /* According to standard the address should be alignment within 4KB */
+ if (addr_align)
+ *addr_align = SZ_4K;
+ /* Size alignment and max size effectively make the size fixed to
+ * size_max */
+ if (size_align)
+ *size_align = pdata->mw_size;
+ if (size_max)
+ *size_max = pdata->mw_size;
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - set the translation of a Memory Window
+ */
+static int idt_ntb_mw_set_trans(struct ntb_dev *ntb, int idx, dma_addr_t addr,
+ resource_size_t size)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ int ret;
+
+ /* Although the passed size is not used anywhere, we need to make sure
+ * the size fits the memory window */
+ if (0 != size && size != pdata->mw_size) {
+ dev_err_ndev(ndev,
+ "Invalid translated address size was specified");
+ return -EINVAL;
+ }
+
+ /* Set the passed memory window or unset it if the size is zero */
+ if (0 != size) {
+ ret = idt_ntb_setmw(ndev, idx, addr);
+ } else /* if (0 == size) */ {
+ ret = idt_ntb_unsetmw(ndev, idx);
+ }
+
+ return ret;
+}
+
+/*
+ * NTB bus callback - peer memory windows count
+ */
+static int idt_ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Return the number of available peer memory windows */
+ return ndev->mw_peer_cnt;
+}
+
+/*
+ * NTB bus callback - get the peer memory windows alignments
+ */
+static int idt_ntb_peer_mw_get_align(struct ntb_dev *ntb, int idx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+
+ /* It's error to pass the out of range Memory Window index */
+ if (idx >= ndev->mw_peer_cnt) {
+ dev_err_ndev(ndev,
+ "Invalid memory window index passed to get "
+ "peer alignment");
+ return -EINVAL;
+ }
+
+ /* Although there are only two unmodifiable LS-bits in lookup table
+ * entries, according to standard the address should be aligned
+ * within 4KB */
+ if (addr_align)
+ *addr_align = SZ_4K;
+ /* Size alignment and max size effectively make the size fixed to
+ * size_max */
+ if (size_align)
+ *size_align = pdata->mw_size;
+ if (size_max)
+ *size_max = pdata->mw_size;
+
+ return SUCCESS;
+}
+
+/*===========================================================================
+ * 5. Doorbells subsystem
+ *===========================================================================*/
+
+static void idt_ntb_db_tasklet(unsigned long data);
+
+/*
+ * Initialize the Global Doorbell Mask
+ *
+ * NOTE Initialize the Inbound Doorbell mask so the local event can
+ * be rised by the self Doorbells bits only. The Outbound
+ * Doorbell is setup so the local port could set both self
+ * and peer Doorbells. Due to the self and peer masks swap
+ * the following loops should work well on the both sides
+ */
+static void idt_ntb_init_gdbellmsk(struct idt_ntb_data *pdata, unsigned char id)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ u32 selfpartbits, peerpartbits;
+ int setbit;
+
+ /* There is a bug if the passed id exceeds the total number of peers */
+ BUG_ON(id >= pdata->peer_cnt);
+
+ /* Get the self and peer partition masks */
+ selfpartbits = ~((u32)1 << pdata->part);
+ peerpartbits = ~((u32)1 << ndevs[id].part);
+
+ /* Init the self Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_self_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ selfpartbits);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ selfpartbits & peerpartbits);
+ }
+
+ /* Init the peer Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_peer_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ peerpartbits);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ selfpartbits & peerpartbits);
+ }
+}
+
+/*
+ * Deinitialize the Global Doorbell Mask
+ *
+ * Function is unused to make sure the NTB devices can be unloaded without
+ * any serious consequences for the peer device.
+ */
+static void __maybe_unused idt_ntb_clean_gdbellmsk(struct idt_ntb_data *pdata,
+ unsigned char id)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ int setbit;
+
+ /* There is a bug if the passed id exceeds the total number of peers */
+ BUG_ON(id >= pdata->peer_cnt);
+
+ /* Deinit the self Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_self_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ (u32)0);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ (u32)0);
+ }
+ /* Deinit the peer Doorbell masks */
+ for_each_set_bit_u32(ndevs[id].db_peer_mask, setbit) {
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GIDBELLMSK0 + setbit,
+ (u32)0);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_GODBELLMSK0 + setbit,
+ (u32)0);
+ }
+}
+
+/*
+ * Initialize the Doorbells for the current NT-function with respect to the
+ * topologically predefined NTB pairs
+ *
+ * NOTE The first NTB pairs are lucky to have the extended set of Doorbells
+ */
+static void idt_ntb_init_db(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_topo *topo = &pdata->topo;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ unsigned char id, dbcntstd, dbcntext, dbleft, luckies, pairid, dboffset;
+ u32 pridbmask, secdbmask;
+
+ /* Calculate the number of Doorbells per pair and the leftovers */
+ dbcntstd = IDT_NTB_DBELL_CNT / topo->paircnt;
+ dbleft = IDT_NTB_DBELL_CNT % topo->paircnt + (dbcntstd % 2) * topo->paircnt;
+ /* Alter the db count to be even */
+ dbcntstd = (dbcntstd / 2) * 2;
+ dbcntext = dbcntstd + 2;
+
+ /* Number of the lucky pairs having additional Doorbells */
+ luckies = dbleft / 2;
+
+ /* Loop over all the locally available peers */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Current pair ID */
+ pairid = ndevs[id].pairid;
+
+ /* Retrieve the doorbells count and the doorbells offset for the
+ * current pair ID (the first luckies have extended doorbells) */
+ if (luckies > pairid) {
+ ndevs[id].db_cnt = dbcntext / 2;
+ dboffset = dbcntext * pairid;
+ } else {
+ ndevs[id].db_cnt = dbcntstd / 2;
+ dboffset = dbcntext * luckies +
+ dbcntstd * (pairid - luckies);
+ }
+
+ /* Calculate the valid Doorbells mask for the corresponding
+ * ports */
+ ndevs[id].db_valid_mask = ((u32)1 << ndevs[id].db_cnt) - 1;
+ pridbmask = ndevs[id].db_valid_mask << dboffset;
+ secdbmask = pridbmask << ndevs[id].db_cnt;
+
+ /* Initialize the corresponding Device structure fields */
+ if (NTB_TOPO_PRI == pdata->role) {
+ ndevs[id].db_self_mask = pridbmask;
+ ndevs[id].db_self_offset = dboffset;
+ ndevs[id].db_peer_mask = secdbmask;
+ ndevs[id].db_peer_offset = dboffset + ndevs[id].db_cnt;
+ } else /* if (NTB_TOPO_SEC == pdata->role) */ {
+ ndevs[id].db_self_mask = secdbmask;
+ ndevs[id].db_self_offset = dboffset + ndevs[id].db_cnt;
+ ndevs[id].db_peer_mask = pridbmask;
+ ndevs[id].db_peer_offset = dboffset;
+ }
+
+ /* Initialize the corresponding Global Doorbell masks. It can be
+ * done by both Primary and Secondary ports */
+ idt_ntb_init_gdbellmsk(pdata, id);
+ }
+
+ /* Initialize the spin lock to sync access to the self doorbell status
+ * and mask variables */
+ pdata->db_sts = 0;
+ pdata->db_msk = (u32)-1;
+ /* In fact db_lock is used at most at tasklet so BH lock would be enough,
+ * but the critical section can be accessed in the db event handler,
+ * which is protected by the context irqsave spin lock. So calling BH
+ * spin locker/unlocker function would cause the OOPS Warning of
+ * local_bh_enable_ip method. Therefore the irqsave/irqrestore methods
+ * are used to synchronize access to the db_sts and db_msk fields*/
+ spin_lock_init(&pdata->db_lock);
+
+ /* Initialize the doorbells tasklet */
+ tasklet_init(&pdata->db_tasklet, idt_ntb_db_tasklet,
+ (unsigned long)pdata);
+
+ /* Unmask the inbound doorbell interrupts */
+ idt_ntb_writereg(pdata->cfg_mmio, IDT_NT_PCI_INDBELLMSK, INDB_UNMASK);
+
+ dev_dbg_data(pdata, "IDT NTB device doorbells initialized");
+}
+
+/*
+ * Clean the Doorbells initialized for the pairs of NT-functions
+ *
+ * It just makes all the NT-functions being able to use the self and peer
+ * Doorbells
+ */
+static void idt_ntb_clean_db(struct idt_ntb_data *pdata)
+{
+ /*unsigned char id;*/
+
+ /* Just kill the tasklet */
+ tasklet_kill(&pdata->db_tasklet);
+
+ /* Just clean the Doorbell masks for all the peers as they must have
+ * initially been. Do it by the Primary side only */
+ /*if (NTB_TOPO_PRI == pdata->role) {
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ idt_ntb_clean_gdbellmsk(pdata, id);
+ }
+ }*/
+
+ dev_dbg_data(pdata, "IDT NTB device doorbells deinitilized");
+}
+
+/*
+ * Doorbells event tasklet
+ */
+static void idt_ntb_db_tasklet(unsigned long data)
+{
+ struct idt_ntb_data *pdata = (struct idt_ntb_data *)data;
+ struct idt_ntb_dev *ndevs = pdata->ndevs;
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 db_sts, db_self, db_sts_prev;
+ unsigned long setbit, irqflags;
+ unsigned char id;
+
+ /* NOTE All doorbells are masked to generate the interrupt by the IRQ
+ * handler until the cause of the interrupt is handled */
+ db_sts = idt_ntb_readreg(cfg, IDT_NT_PCI_INDBELLSTS);
+ /* Clear all the retrieved doorbell bits */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_INDBELLSTS, db_sts);
+ /* Finally unmask the doorbells interrupt. The next action shall rise
+ * the interrupt if any doorbell bit was set after the register had
+ * been read and cleared */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_INDBELLMSK, INDB_UNMASK);
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Retrieve the current doorbell status bits */
+ db_sts_prev = pdata->db_sts;
+ /* Set the new doorbell status */
+ pdata->db_sts |= db_sts;
+ /* There are going to be handled only the doorbell bits, which have not
+ * been set before and also have not been masked */
+ db_sts &= ~db_sts_prev & ~pdata->db_msk;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* If the new doorbell status bits are masked then do nothing */
+ if (!db_sts) {
+ dev_dbg_data(pdata, "Got masked doorbell interrupt");
+ return;
+ }
+
+ /* Walk through all the peers looking for the relevant one to handle
+ * new doorbells */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Invoke the context callback if there are doorbells set for
+ * the current NTB device */
+ db_self = (db_sts & ndevs[id].db_self_mask);
+ db_self >>= ndevs[id].db_self_offset;
+ for_each_set_bit_u32(db_self, setbit) {
+ ntb_db_event(&ndevs[id].ntb, (int)setbit);
+ }
+ }
+}
+
+/*
+ * NTB bus callback - get a mask of doorbell bits supported by the ntb
+ */
+static u64 idt_ntb_db_valid_mask(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Return the valid doorbell bits mask */
+ return ndev->db_valid_mask;
+}
+
+/*
+ * NTB bus callback - get the number of doorbell interrupt vectors
+ */
+static int idt_ntb_db_vector_count(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ /* Number of doorbell vectors equal to the doorbell bits count */
+ return ndev->db_cnt;
+}
+
+/*
+ * NTB bus callback - get a mask of doorbell bits serviced by a vector
+ */
+static u64 idt_ntb_db_vector_mask(struct ntb_dev *ntb, int db_vec)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+
+ if (db_vec < 0 || ndev->db_cnt <= db_vec) {
+ return 0;
+ }
+
+ /* Each doorbell bit corresponds to the vector so the mask is just one
+ * shifted bit */
+ return ((u64)1 << db_vec);
+}
+
+/*
+ * NTB bus callback - read the local doorbell register
+ */
+static u64 idt_ntb_db_read(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+ u32 db_sts;
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Read the current doorbell status */
+ db_sts = pdata->db_sts;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* Return the accordingly shifted doorbell bits */
+ return (db_sts & ndev->db_self_mask) >> ndev->db_self_offset;
+}
+
+/*
+ * NTB bus callback - set bits in the local doorbell register
+ *
+ * NOTE It must be done using the doorbell register io to generate the
+ * interrupt and invoke the doorbell event handler set by the client
+ * driver
+ */
+static int idt_ntb_db_set(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid doorbell bits are passed to locally set");
+ return -EINVAL;
+ }
+
+ /* Set the corresponding bits in the doorbell register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_OUTDBELLSET,
+ ((u32)db_bits << ndev->db_self_offset));
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - clear bits in the local doorbell register
+ */
+static int idt_ntb_db_clear(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid doorbell bits are passed to locally clear");
+ return -EINVAL;
+ }
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Read the current doorbell status */
+ pdata->db_sts &= ~((u32)db_bits << ndev->db_self_offset);
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - read the local doorbell mask
+ */
+static u64 idt_ntb_db_read_mask(struct ntb_dev *ntb)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+ u32 db_msk;
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Read the current doorbell mask */
+ db_msk = pdata->db_msk;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* Return the accordingly shifted doorbell bits */
+ return (db_msk & ndev->db_self_mask) >> ndev->db_self_offset;
+}
+
+/*
+ * NTB bus callback - set bits in the local doorbell mask
+ */
+static int idt_ntb_db_set_mask(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ unsigned long irqflags;
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid field is passed to set the doorbell mask");
+ return -EINVAL;
+ }
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Set the corresponding bits in the local mask */
+ pdata->db_msk |= ((u32)db_bits << ndev->db_self_offset);
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - clear bits in the local doorbell mask
+ */
+static int idt_ntb_db_clear_mask(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ u32 db_sts, unmask_bits;
+ unsigned long setbit, irqflags;
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid field is passed to clear the doorbell mask");
+ return -EINVAL;
+ }
+
+ /* Calculate the unmaskable bits first */
+ unmask_bits = ((u32)db_bits << ndev->db_self_offset);
+
+ /** START Sync access to the doorbell variables */
+ spin_lock_irqsave(&pdata->db_lock, irqflags);
+ /* Retrieve the doorbell status bits, which have been masked, but are
+ * going to be unmasked now */
+ db_sts = pdata->db_sts & pdata->db_msk & unmask_bits;
+ /* Clear the corresponding bits in the local mask */
+ pdata->db_msk &= ~unmask_bits;
+ /** END The critical section of access to the doorbell variables */
+ spin_unlock_irqrestore(&pdata->db_lock, irqflags);
+
+ /* Invoke the context callback if there are set doorbells, which have
+ * just been unmasked */
+ db_sts = (db_sts & ndev->db_self_mask) >> ndev->db_self_offset;
+ for_each_set_bit_u32(db_sts, setbit) {
+ ntb_db_event(&ndev->ntb, (int)setbit);
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - set bits in the peer doorbell register
+ */
+static int idt_ntb_peer_db_set(struct ntb_dev *ntb, u64 db_bits)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+
+ /* Return error if invalid bits are set */
+ if (db_bits & ~ndev->db_valid_mask) {
+ dev_dbg_ndev(ndev,
+ "Invalid doorbell bits are passed to remotely set");
+ return -EINVAL;
+ }
+
+ /* Set the corresponding bits in the doorbell register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_OUTDBELLSET,
+ ((u32)db_bits << ndev->db_peer_offset));
+
+ return SUCCESS;
+}
+
+/*===========================================================================
+ * 6. Messaging subsystem
+ *===========================================================================*/
+
+static void idt_ntb_inmsg_work(struct work_struct *work);
+
+static void idt_ntb_outmsg_work(struct work_struct *work);
+
+static void idt_ntb_msg_tasklet(unsigned long data);
+
+/*
+ * Constructor is used initialize the allocated message structure
+ */
+static inline void idt_ntb_msg_ctor(struct idt_ntb_msg *msg)
+{
+ /* Set initial message retry count */
+ msg->retry = IDT_NTB_SENDMSG_RETRY;
+
+ /* Init the queue entry */
+ INIT_LIST_HEAD(&msg->entry);
+}
+
+/*
+ * Initialize the messaging subsystem
+ */
+static int idt_ntb_init_msg(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndev;
+ unsigned char id;
+
+ /* Allocate the IDT messages cache without alignment and flags with no
+ * constructor */
+ pdata->msg_cache = kmem_cache_create(NTB_CACHENAME,
+ sizeof(struct idt_ntb_msg), 0, 0, NULL);
+ if (NULL == pdata->msg_cache) {
+ dev_err_data(pdata,
+ "IDT NTB failed to allocate the messages cache");
+ return -ENOMEM;
+ }
+
+ /* Init the messages routing spin lock */
+ spin_lock_init(&pdata->msg_lock);
+
+ /* Walk through all the device initializing the message related
+ * structures */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Get the current NTB device structure */
+ ndev = &pdata->ndevs[id];
+
+ /* Initialize the incoming messages queue */
+ atomic_queue_init(&ndev->qinmsg);
+ /* Setup the incoming message work thread (it's not delayed) */
+ INIT_WORK(&ndev->inmsg_work, idt_ntb_inmsg_work);
+
+ /* Initialize the outgoing messages queue */
+ atomic_queue_init(&ndev->qoutmsg);
+ /* Setup the outgoing message work thread (it can be
+ * delayed) */
+ INIT_DELAYED_WORK(&ndev->outmsg_work, idt_ntb_outmsg_work);
+ }
+
+ /* Setup the messages tasklet - bh handler of incoming messages */
+ tasklet_init(&pdata->msg_tasklet, idt_ntb_msg_tasklet,
+ (unsigned long)pdata);
+
+ /* Clear the outbound and inbound Messages status */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, MSG_MASK);
+
+ /* Unmask the message interrupts only for the first incoming message
+ * register */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_UNMASK);
+
+ dev_dbg_data(pdata, "IDT NTB device messaging subsystem initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the messaging subsystem
+ */
+static void idt_ntb_deinit_msg(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_dev *ndev;
+ struct list_head *entry;
+ unsigned char id;
+
+ /* Just kill the tasklet */
+ tasklet_kill(&pdata->db_tasklet);
+
+ /* Walk through all the devices deinitializing the message related
+ * structures */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Get the current NTB device structure */
+ ndev = &pdata->ndevs[id];
+
+ /* Stop the incoming message work thread */
+ cancel_work_sync(&ndev->inmsg_work);
+ /* Free all the allocated incoming message objects */
+ while (!atomic_queue_empty(&ndev->qinmsg)) {
+ entry = atomic_queue_get(&ndev->qinmsg);
+ kmem_cache_free(pdata->msg_cache,
+ to_msg_list_entry(entry));
+ }
+
+ /* Stop the outgoing message work thread */
+ cancel_delayed_work_sync(&ndev->outmsg_work);
+ /* Free all the allocated outgoing message objects */
+ while (!atomic_queue_empty(&ndev->qoutmsg)) {
+ entry = atomic_queue_get(&ndev->qoutmsg);
+ kmem_cache_free(pdata->msg_cache,
+ to_msg_list_entry(entry));
+ }
+ }
+
+ /* Mask the message interrupts */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_MASK);
+
+ /* Clear the outbound and inbound messages status */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, MSG_MASK);
+
+ /* Destroy the IDT messages cache */
+ kmem_cache_destroy(pdata->msg_cache);
+
+ dev_dbg_data(pdata,
+ "IDT NTB function messaging subsystem deinitialized");
+}
+
+/*
+ * Write message to the specified peer
+ */
+static int idt_ntb_writemsg(struct idt_ntb_dev *ndev, const struct ntb_msg *msg)
+{
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ void __iomem *cfg = to_cfg_ndev(ndev);
+ u32 stat, swpmsgctl[IDT_NTB_MSG_CNT];
+ int regid;
+
+ /* Initialize the message control register so the local outbound message
+ * registers would be connected with the peers inbound ones */
+ for (regid = 0; regid < IDT_NTB_MSG_CNT; regid++) {
+ /* Init switch partition message control registers variable */
+ swpmsgctl[regid] = 0;
+ idt_ntb_writefld_var(&swpmsgctl[regid], IDT_SW_MSGROUTE_REG,
+ regid);
+ idt_ntb_writefld_var(&swpmsgctl[regid], IDT_SW_MSGROUTE_PART,
+ ndev->part);
+ }
+
+ /* Use spin lock to synchronize just thirteen IO operations. It's used
+ * just among the kernel threads so we don't need to disable IRQs/bh */
+ spin_lock(&pdata->msg_lock);
+ /* Route to the local outbound message to the inbound one of the peer
+ * and send the data to there starting from the data because the
+ * interrupts are enabled for the first message register only */
+ for (regid = (IDT_NTB_MSG_CNT - 1); 0 <= regid; regid--) {
+ /* Set the route and send the data */
+ idt_ntb_writereg(cfg, partdata_tbl[pdata->part].msgctl[regid],
+ swpmsgctl[regid]);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_OUTMSG0 + regid,
+ msg->data[regid]);
+ /* Read the status of the previous operation */
+ stat = idt_ntb_readfld_mem(cfg, IDT_NT_OUTMSGSTS);
+ if (SUCCESS != stat) {
+ dev_dbg_ndev(ndev,
+ "Failed to send message to peer %hhd", regid);
+ break;
+ }
+ }
+ /* Immedietly clear the outbound message status if it has been set */
+ if (SUCCESS != stat) {
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, OUTMSG_MASK);
+ }
+ /* Finally unlock the message routing subsystem */
+ spin_unlock(&pdata->msg_lock);
+
+ /* If the write operation was not successful then the peer inbound
+ * register must be full so return -EBUSY error */
+ if (SUCCESS != stat) {
+ return -EBUSY;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Read the message
+ */
+static int idt_ntb_readmsg(struct idt_ntb_data *pdata, unsigned char *part,
+ struct ntb_msg *msg)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 msgsts, msgsrc;
+ unsigned char regid;
+
+ /* Read the inbound messages status */
+ msgsts = idt_ntb_readfld_mem(cfg, IDT_NT_INMSGSTS);
+ if (INMSG_STS != msgsts) {
+ dev_err_data(pdata, "Invalid status %#80x to read msg", msgsts);
+ BUG();
+ return -EINVAL;
+ }
+
+ /* Read data from the inbound message registers. It doesn't need to be
+ * synchronized since the read operation is performed from the tasklet
+ * only, that is non-reentrant */
+ *part = idt_ntb_readreg(cfg, IDT_NT_PCI_INMSGSRC0);
+ for (regid = 0; regid < IDT_NTB_MSG_CNT; regid++) {
+ msg->data[regid] =
+ idt_ntb_readreg(cfg, IDT_NT_PCI_INMSG0 + regid);
+ /* Read the source of the message checking whether the message
+ * data has come from the same partition */
+ msgsrc = idt_ntb_readreg(cfg, IDT_NT_PCI_INMSGSRC0 + regid);
+ if (msgsrc != *part) {
+ dev_err_data(pdata,
+ "Message data is inconsistent, src: %u != %u",
+ *part, msgsrc);
+ BUG();
+ return -EINVAL;
+ }
+ }
+
+ /* Clear the inbound message status */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTS, INMSG_MASK);
+
+ return SUCCESS;
+}
+
+/*
+ * Work thread handling the inbound messages events
+ */
+static void idt_ntb_inmsg_work(struct work_struct *work)
+{
+ struct idt_ntb_dev *ndev = to_ndev_inmsg_work(work);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ struct list_head *entry;
+ struct idt_ntb_msg *msgwrap;
+
+ /* Retrieve the last received message. It's bug to have inbound message
+ * queue empty at this point since the tasklet has just added one in
+ * there */
+ entry = atomic_queue_get(&ndev->qinmsg);
+ BUG_ON(NULL == entry);
+ msgwrap = to_msg_list_entry(entry);
+
+ /* Call the client driver message event handler */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_NEW, &msgwrap->msg);
+
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+}
+
+/*
+ * Work thread handling the outgoing messages
+ */
+static void idt_ntb_outmsg_work(struct work_struct *work)
+{
+ struct idt_ntb_dev *ndev = to_ndev_outmsg_work(work);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ struct list_head *entry;
+ struct idt_ntb_msg *msgwrap;
+ int ret;
+
+ /* Retrieve a message from the top of the queue. It's bug to have
+ * inbound message queue empty at this point since the client driver
+ * has just added one in there */
+ entry = atomic_queue_get(&ndev->qoutmsg);
+ BUG_ON(NULL == entry);
+ msgwrap = to_msg_list_entry(entry);
+
+ /* If link is not up it is useless to send any data */
+ if (OFF == idt_ntb_link_status(ndev)) {
+ dev_dbg_ndev(ndev,
+ "Link got suddenly down while sending a message");
+ /* Link got down so rise the fail event */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_FAIL, &msgwrap->msg);
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+ /* If some messages are left then reschedule the worker */
+ goto outmsg_work_requeue;
+ } /* else of (ON == idt_ntb_link_status(ndev)) */
+
+ /* Try to send the message */
+ ret = idt_ntb_writemsg(ndev, &msgwrap->msg);
+ if (SUCCESS == ret) {
+ /* The message has been successfully sent so rise the event */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_SENT, &msgwrap->msg);
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+ /* May need to reschedule the worker */
+ goto outmsg_work_requeue;
+ } /* else if (SUCCESS != ret) {} */
+
+ /* Could not send message. Rise the error if it has been the last
+ * attempt. If it hasn't get the message back into the queue and
+ * restart the worker */
+ msgwrap->retry--;
+ if (likely(0 != msgwrap->retry)) {
+ atomic_queue_add(&ndev->qoutmsg, &msgwrap->entry);
+ } else /* if (0 == msgwrap->retry) */ {
+ dev_err_ndev(ndev, "Run out of attempt to send a message");
+ /* Rise the error in this case */
+ ntb_msg_event(&ndev->ntb, NTB_MSG_FAIL, &msgwrap->msg);
+ /* Message memory can be freed */
+ kmem_cache_free(pdata->msg_cache, msgwrap);
+ }
+
+ /* If there is something left to send then queue the handler again */
+outmsg_work_requeue:
+ if (!atomic_queue_empty(&ndev->qoutmsg)) {
+ (void)queue_delayed_work(pdata->idt_wq, &ndev->outmsg_work,
+ IDT_NTB_SENDMSG_TOUT);
+ }
+}
+
+/*
+ * Tasklet handling inbound messages
+ */
+static void idt_ntb_msg_tasklet(unsigned long data)
+{
+ struct idt_ntb_data *pdata = (struct idt_ntb_data *)data;
+ struct idt_ntb_dev *ndev, *tndev = NULL;
+ struct idt_ntb_msg *msgwrap;
+ void __iomem *cfg = pdata->cfg_mmio;
+ unsigned char part, id;
+
+ /* Allocate the memory for the new message */
+ msgwrap = kmem_cache_alloc(pdata->msg_cache, GFP_KERNEL);
+ if (NULL == msgwrap) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for incoming message");
+ return;
+ }
+ /* Initializet the allocated message wrap structure although it's not
+ * necessary here */
+ idt_ntb_msg_ctor(msgwrap);
+
+ /* Read the message from the inbound registers. Don't need to check
+ * the return value since error would be asserted anyway */
+ (void)idt_ntb_readmsg(pdata, &part, &msgwrap->msg);
+
+ /* Finally unmask the message IRQs so the next message can be
+ * retrieved */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_UNMASK);
+
+ /* Find device the message has been sent to */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Retrieve the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Break the loop if target device is found */
+ if (ndev->part == part) {
+ tndev = ndev;
+ break;
+ }
+ }
+ /* Assert bug if message was received from invalid partition */
+ BUG_ON(NULL == tndev);
+
+ /* Add the new message to the tail of incoming queue of the target
+ * device */
+ atomic_queue_add_tail(&ndev->qinmsg, &msgwrap->entry);
+
+ /* Schedule the inbound message worker straight away */
+ (void)queue_work(pdata->idt_wq, &ndev->inmsg_work);
+}
+
+/*
+ * NTB bus callback - post the message to the peer
+ */
+static int idt_ntb_msg_post(struct ntb_dev *ntb, struct ntb_msg *msg)
+{
+ struct idt_ntb_dev *ndev = to_ndev_ntb(ntb);
+ struct idt_ntb_data *pdata = to_data_ndev(ndev);
+ struct idt_ntb_msg *msgwrap;
+ unsigned char idx;
+
+ /* If the link is down then don't post any message */
+ if (OFF == idt_ntb_link_status(ndev)) {
+ dev_dbg_ndev(ndev,
+ "Can't post a message since link is down");
+ return -EINVAL;
+ }
+
+ /* Allocate memory for message wrap structure */
+ msgwrap = kmem_cache_alloc(pdata->msg_cache, GFP_KERNEL);
+ if (NULL == msgwrap) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for outgoing message");
+ return -ENOMEM;
+ }
+ /* Initializet the allocated message wrap structure */
+ idt_ntb_msg_ctor(msgwrap);
+
+ /* Fill in the message wrapper with data */
+ for (idx = 0; idx < IDT_NTB_MSG_CNT; idx++) {
+ msgwrap->msg.data[idx] = msg->data[idx];
+ }
+
+ /* Add the initialized wrap to the queue of outgoing messages */
+ atomic_queue_add_tail(&ndev->qoutmsg, &msgwrap->entry);
+
+ /* Start the outgoing messages worker with no timeout */
+ (void)queue_delayed_work(pdata->idt_wq, &ndev->outmsg_work, 0);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB bus callback - size of the message data
+ */
+static int idt_ntb_msg_size(struct ntb_dev *ntb)
+{
+ /* Just return the number of messages registers */
+ return IDT_NTB_MSG_CNT;
+}
+
+/*===========================================================================
+ * 7. IRQ-related functions
+ *===========================================================================*/
+
+static irqreturn_t idt_ntb_isr(int irq, void *dev);
+
+/*
+ * Convert the temperature field to the value and fraction
+ */
+static inline void idt_ntb_convert_temp(const u32 temp,
+ unsigned char *val, unsigned char *frac)
+{
+ *val = temp >> 1;
+ *frac = ((temp & 0x1) ? 5 : 0);
+}
+
+/*
+ * Initialize the IDT IRQ sources
+ */
+static void idt_ntb_init_irqsrc(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 tempctl = 0;
+
+ /* Set the temperature sensor alarms */
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_LTH, IDT_NTB_TEMP_LTH << 1);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_HTH, IDT_NTB_TEMP_HTH << 1);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_BLTH_EN, ON);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_AHTH_EN, ON);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_PDOWN, OFF);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_TMPCTL, tempctl);
+
+ /* Interrupts are enabled by default only for Primary side since there
+ * can be more than one device */
+ if (NTB_TOPO_PRI == pdata->role) {
+ /* Enable the interrupts of message, doorbells, switch and
+ * temperature sensor events. This will generate all the
+ * pending interrupts after the link is effectively enabled */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, NTINT_UNMASK);
+ } else /* if (NTB_TOPO_SEC == pdata->role) */ {
+ /* Disable all the interrupts. NTB device enable callback will
+ * enable the necessary message, doorbells, switch and
+ * temperature sensor events */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, ALLINT_MASK);
+ }
+}
+
+/*
+ * Clear the IDT IRQs
+ */
+static void idt_ntb_clear_irqsrc(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 tempctl = 0;
+
+ /* Unset the temperature sensor alarm and disable the sensor */
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_BLTH_EN, OFF);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_AHTH_EN, OFF);
+ idt_ntb_writefld_var(&tempctl, IDT_SW_TMP_PDOWN, ON);
+ idt_ntb_writereg(cfg, IDT_SW_PCI_TMPCTL, tempctl);
+
+ /* Mask all the interrupts */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTMSK, ALLINT_MASK);
+}
+
+/*
+ * Initialize the PCIe interrupt handler
+ *
+ * NOTE The code is gotoed a bit, but still it's pretty obvious. First
+ * we try to enable MSI interrupt. If it fails we initiate the INTx interrupt.
+ * In any successful case the IDT NTB interrupts need to be enabled.
+ */
+static int idt_ntb_init_isr(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+ int ret;
+
+ /* Enable the MSI interrupts */
+ ret = pci_enable_msi(pdev);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "IDT failed to enable MSI interrupt");
+ goto err_try_intx;
+ }
+
+ /* Request correspondig IRQ number */
+ ret = request_irq(pdev->irq, idt_ntb_isr, 0, NTB_IRQNAME, pdata);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "IDT failed to set MSI IRQ handler");
+ goto err_disable_msi;
+ }
+
+ /* From now on the MSI interrupt is used */
+ dev_dbg_data(pdata, "IDT NTB is using MSI interrupts");
+
+ /* Need to enable the corresponding IDT NTB interrupts */
+ goto idt_init_irqs;
+
+err_disable_msi:
+ pci_disable_msi(pdev);
+
+err_try_intx:
+ /* Enable INTx interrutps since MSI can't be used */
+ pci_intx(pdev, ON);
+
+ ret = request_irq(pdev->irq, idt_ntb_isr, IRQF_SHARED,
+ NTB_IRQNAME, pdata);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "IDT failed to enable INTx interrupt");
+ goto err_pci_indx;
+ }
+
+ /* From now on the INTx interrupt is used */
+ dev_dbg_data(pdata, "IDT NTB is using INTx interrupts");
+
+ /* Need to enable the corresponding IDT NTB interrupts */
+idt_init_irqs:
+ idt_ntb_init_irqsrc(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB function IRQs initilized");
+
+ return SUCCESS;
+
+err_pci_indx:
+ pci_intx(pdev, OFF);
+
+ return ret;
+}
+
+/*
+ * Deinitialize the PCIe interrupt handler
+ */
+static void idt_ntb_clear_isr(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+
+ /* Clear the IDT NTB interrupt sources by masking them */
+ idt_ntb_clear_irqsrc(pdata);
+
+ /* Stop the interrupt handling */
+ free_irq(pdev->irq, pdata);
+ if (pci_dev_msi_enabled(pdev)) {
+ pci_disable_msi(pdev);
+ } else /* if (!pci_dev_msi_enabled(pdev)) */ {
+ pci_intx(pdev, OFF);
+ }
+
+ dev_dbg_data(pdata, "IDT NTB function interrupts are disabled");
+}
+
+/*
+ * Switch events ISR
+ */
+static void idt_ntb_se_isr(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntintsts = 0, sests;
+
+ /* Clean the corresponding interrupt bit */
+ idt_ntb_writefld_var(&ntintsts, IDT_NT_SEINT_STS, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTSTS, ntintsts);
+
+ /* Just print we got the switch event */
+ sests = idt_ntb_readreg(cfg, IDT_SW_PCI_SESTS);
+ dev_dbg_data(pdata, "Got switch event IRQ %#08x", sests);
+}
+
+/*
+ * Temperature sensor event ISR
+ */
+static void idt_ntb_temp_isr(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntintsts = 0, curtemp;
+ unsigned char val, frac;
+
+ /* Clean the corresponding interrupt bit */
+ idt_ntb_writefld_var(&ntintsts, IDT_NT_TMPINT_STS, ON);
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTINTSTS, ntintsts);
+
+ /* Read the temperature status */
+ curtemp = idt_ntb_readfld_mem(cfg, IDT_SW_TMP_CURTEMP);
+ idt_ntb_convert_temp(curtemp, &val, &frac);
+
+ /* Print the current temperature */
+ dev_warn_data(pdata,
+ "IDT temperature sensor alarm: %hhu.%hhu, valid space [%d;%d]",
+ val, frac, IDT_NTB_TEMP_LTH, IDT_NTB_TEMP_HTH);
+
+ /* Read the temperature alarm to clear the value out */
+ (void)idt_ntb_readreg(cfg, IDT_SW_PCI_TMPALARM);
+}
+
+/*
+ * IDT PCIe-swtich NTB-function interrupts handler
+ */
+static irqreturn_t idt_ntb_isr(int irq, void *dev)
+{
+ struct idt_ntb_data *pdata = dev;
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 ntintsts;
+ unsigned long setbit;
+ irqreturn_t status = IRQ_NONE;
+
+ /* Read the NTINTSTS register to determine the source of the
+ * interrupt.
+ * NOTE In order to make sure the deferred handlers are executed
+ * only when the corresponding interrupt really happens, the
+ * message/boorbell interrupt is temporarily masked. Additionally
+ * the interrupts status register must be filtered with the interrupts
+ * mask since the correposnding bit may be set even when the interrupt
+ * is masked */
+ ntintsts = idt_ntb_readreg(cfg, IDT_NT_PCI_NTINTSTS) &
+ ~idt_ntb_readreg(cfg, IDT_NT_PCI_NTINTMSK);
+ for_each_set_bit_u32(ntintsts, setbit) {
+ /* Handle the cause of the interrupt */
+ switch (setbit) {
+ case MSGINT_BIT:
+ /* Mask the message IRQs until the data is handled. It
+ * must be unmasked within the tasklet right after the
+ * data is read so the next message can be retrieved */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_MSGSTSMSK, MSG_MASK);
+ /* Schedule the tasklet to handle the new message */
+ tasklet_schedule(&pdata->msg_tasklet);
+ break;
+ case DBINT_BIT:
+ /* Mask the doorbell IRQs until the data is handled. It
+ * must be unmasked within the tasklet right after the
+ * doorbell status bits are read and clear so the next
+ * doorbell event can be raised */
+ idt_ntb_writereg(cfg, IDT_NT_PCI_INDBELLMSK, INDB_MASK);
+ /* Schedule the tasklet to handle the set doorbell bits */
+ tasklet_schedule(&pdata->db_tasklet);
+ break;
+ case SEINT_BIT:
+ /* Just call the switch event handler. It doesn't do
+ * much work */
+ idt_ntb_se_isr(pdata);
+ break;
+ case TEMPINT_BIT:
+ /* Just call the temperature sensor event handler.
+ * It doesn't do much work */
+ idt_ntb_temp_isr(pdata);
+ break;
+ default:
+ dev_err_data(pdata,
+ "Invalid IDT IQR status bit is set");
+ break;
+ }
+ /* If there is any interrupt bit is set then we handle it */
+ status = IRQ_HANDLED;
+ }
+
+ return status;
+}
+
+/*===========================================================================
+ * 8. NTB bus initialization
+ *===========================================================================*/
+
+/*
+ * NTB KAPI operations
+ *
+ * NOTE This driver implements the synchronous interface only.
+ */
+static const struct ntb_dev_ops idt_ntb_ops = {
+ .link_is_up = idt_ntb_link_is_up,
+ .link_enable = idt_ntb_link_enable,
+ .link_disable = idt_ntb_link_disable,
+ .mw_count = idt_ntb_mw_count,
+ .mw_get_maprsc = idt_ntb_mw_get_maprsc,
+ .mw_get_align = idt_ntb_mw_get_align,
+ .mw_set_trans = idt_ntb_mw_set_trans,
+ .peer_mw_count = idt_ntb_peer_mw_count,
+ .peer_mw_get_align = idt_ntb_peer_mw_get_align,
+ .db_valid_mask = idt_ntb_db_valid_mask,
+ .db_vector_count = idt_ntb_db_vector_count,
+ .db_vector_mask = idt_ntb_db_vector_mask,
+ .db_read = idt_ntb_db_read,
+ .db_set = idt_ntb_db_set,
+ .db_clear = idt_ntb_db_clear,
+ .db_read_mask = idt_ntb_db_read_mask,
+ .db_set_mask = idt_ntb_db_set_mask,
+ .db_clear_mask = idt_ntb_db_clear_mask,
+ .peer_db_set = idt_ntb_peer_db_set,
+ .msg_post = idt_ntb_msg_post,
+ .msg_size = idt_ntb_msg_size
+};
+
+/*
+ * NTB devices registration function
+ */
+static int idt_ntb_register_devs(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_dev *ndev;
+ int id, ret;
+
+ /* Loop over all the NTB devices initializing the necessary fields */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Retrieve the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Set the device operation callbacks */
+ ndev->ntb.ops = &idt_ntb_ops;
+
+ /* Register the device */
+ ret = ntb_register_device(&ndev->ntb);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to register NTB device");
+ goto err_unregister_device;
+ }
+ }
+
+ dev_dbg_data(pdata, "IDT NTB device(s) successfully registered");
+
+ return SUCCESS;
+
+err_unregister_device:
+ for (id--; 0 <= id; id--) {
+ ndev = &pdata->ndevs[id];
+ ntb_unregister_device(&ndev->ntb);
+ }
+
+ return ret;
+}
+
+/*
+ * NTB devices unregistration function
+ */
+static void idt_ntb_unregister_devs(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_dev *ndev;
+ int id;
+
+ /* Loop over all the NTB devices initializing the necessary fields */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ /* Retrieve the current NTB device */
+ ndev = &pdata->ndevs[id];
+
+ /* Just unregister the device */
+ ntb_unregister_device(&ndev->ntb);
+ }
+
+ dev_dbg_data(pdata, "IDT NTB devices are practically unregistered");
+}
+
+/*===========================================================================
+ * 9. IDT NT-functions topology
+ *===========================================================================*/
+
+/*
+ * Add the NT-function pair of Primary and Secondary ports to the topology
+ */
+static inline void idt_ntb_addntb(struct idt_ntb_topo *topo,
+ const unsigned char pri,
+ const unsigned char sec)
+{
+ topo->priports |= ((u32)1 << pri);
+ topo->secports[pri] |= ((u32)1 << sec);
+}
+
+/*
+ * Retrieve the port role
+ */
+static inline enum ntb_topo idt_ntb_portrole(const struct idt_ntb_topo *topo,
+ const unsigned char port)
+{
+ return ((topo->priports & ((u32)1 << port)) ?
+ NTB_TOPO_PRI : NTB_TOPO_SEC);
+}
+
+/*
+ * Function first checks whether the port can have an NT-function then whether
+ * the function is activated on the port
+ */
+static int idt_ntb_checkport(const struct idt_ntb_data *pdata,
+ const unsigned char port)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ unsigned char pid;
+ u32 sts, mode;
+ int stat = -EINVAL;
+
+ /* Check whether the port can have the NT-function */
+ for (pid = 0; pid < pdata->swcfg->port_cnt; pid++) {
+ if (pdata->swcfg->ports[pid] == port) {
+ stat = SUCCESS;
+ break;
+ }
+ }
+ /* Return -EINVAL if it can't */
+ if (SUCCESS != stat) {
+ return -EINVAL;
+ }
+
+ /* Get the port status so to determine the port mode */
+ sts = idt_ntb_readreg(cfg, portdata_tbl[port].sts);
+ mode = idt_ntb_readfld_var(sts, IDT_SW_PORT_MODE);
+
+ /* Check whther the port has the NT-function */
+ if (PORTMODE_NT != mode && PORTMODE_USNT != mode &&
+ PORTMODE_USNTDMA != mode) {
+ return -EINVAL;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Scan the IDT NT-function topology by reading the NTSDATA register
+ * That register is initialized with the Primary port number of the
+ * corresponding secondary ports. Of course the algorithm doesn't permit the
+ * two Primary ports pointing to each other.
+ */
+static int idt_ntb_scantopo(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_topo *topo = &pdata->topo;
+ unsigned char pid;
+ unsigned long port;
+ u32 priport;
+ int ret;
+
+ /* Clean the topo structure */
+ memset(topo, 0, sizeof(*topo));
+
+ /* Walk through all the available ports checking whether the
+ * NT-function enabled on them. If so retrieve its Primary side port */
+ for (pid = 0; pid < pdata->swcfg->port_cnt; pid++) {
+ /* Retrieve the port number */
+ port = pdata->swcfg->ports[pid];
+
+ /* Check whether the port has the NT-function
+ * NOTE Within this loop we are sure it can */
+ if (SUCCESS == idt_ntb_checkport(pdata, port)) {
+ /* If it does then read it's NTSDATA interpreting its
+ * value as the Primary port number */
+ priport = idt_ntb_readreg(cfg,
+ portdata_tbl[port].ntsdata);
+
+ /* Add the NTB to the topology only if the retrieved
+ * primary port can have NT-function and have it
+ * activated */
+ ret = idt_ntb_checkport(pdata, priport);
+ if (SUCCESS == ret && port != priport) {
+ idt_ntb_addntb(topo, priport, port);
+ /* Increment the number of NTB pairs */
+ topo->paircnt++;
+ }
+
+ /* If the retrieved port either can't have the
+ * NT-function or doesn't have NT-function activated
+ * then the topology is corrupted */
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Invalid primary NT port %u was read",
+ priport);
+ return -EINVAL;
+ }
+ } /* else { just skip it }*/
+ }
+
+ /* Check the topology consistency to make sure it is just downwards
+ * directional tree graph with two levels: one primary root and
+ * a number of secondary lists (can be none) */
+ for_each_set_bit_u32(topo->priports, port) {
+ /* Check whether there is no any Primary port amongst the
+ * Secondary ports */
+ if (topo->secports[port] & topo->priports) {
+ dev_err_data(pdata,
+ "Port %lu has Primary and Secondary roles,"
+ "IDT NTB topology is inconsistent", port);
+ return -EINVAL;
+ }
+ }
+
+ dev_dbg_data(pdata, "IDT NTB functions topology has been scanned");
+
+ return SUCCESS;
+}
+
+/*
+ * Create set of Secondary sided peer devices of the topology
+ * The function is used by the Primary side of the topology
+ */
+static int idt_ntb_secpeers(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_topo *topo = &pdata->topo;
+ u32 secports, portsts;
+ unsigned char id = 0;
+ unsigned long port;
+ int node;
+
+ /* Get the set of the Secondary ports of the current Primary port */
+ secports = topo->secports[pdata->port];
+
+ /* Calculate the number of peers */
+ pdata->peer_cnt = hweight32(secports);
+
+ /* Allocate the memory for all the peers IDT NTB device structures */
+ node = dev_to_node(to_dev_data(pdata));
+ pdata->ndevs = kzalloc_node(pdata->peer_cnt*sizeof(*pdata->ndevs),
+ GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(pdata->ndevs)) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for Secondary peer devices");
+ return -ENOMEM;
+ }
+
+ /* Walk through all the secondary ports initializing the
+ * corresponding NTB device and data fields */
+ for_each_set_bit_u32(secports, port) {
+ /* Read the port status register to retrieve the partition */
+ portsts = idt_ntb_readreg(cfg, portdata_tbl[port].sts);
+
+ /* Save the port and partition numbers */
+ pdata->ndevs[id].port = port;
+ pdata->ndevs[id].part =
+ idt_ntb_readfld_var(portsts, IDT_SW_PORT_SWPART);
+
+ /* Initialize the local topology and PCI device fields */
+ pdata->ndevs[id].ntb.topo = pdata->role;
+ pdata->ndevs[id].ntb.pdev = pdata->pdev;
+
+ /* Increment the device id number */
+ id++;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Create Primary sided peer device of the topology
+ * The function is used by the Secondary side of the topology
+ */
+static int idt_ntb_pripeer(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ u32 priport, portsts;
+ int node;
+
+ /* Get the Primary port of the current port */
+ priport = idt_ntb_readreg(cfg, portdata_tbl[pdata->port].ntsdata);
+
+ /* There is going to be just one peer */
+ pdata->peer_cnt = 1;
+
+ /* Allocate the memory for IDT NTB device structure of just one peer */
+ node = dev_to_node(to_dev_data(pdata));
+ pdata->ndevs = kzalloc_node(sizeof(*pdata->ndevs), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(pdata->ndevs)) {
+ dev_err_data(pdata,
+ "Failed to allocate memory for Primary peer device");
+ return -ENOMEM;
+ }
+
+ /* Read the port status register to retrieve the partition */
+ portsts = idt_ntb_readreg(cfg, portdata_tbl[priport].sts);
+
+ /* Save the peer id, port and partition numbers */
+ pdata->ndevs->port = priport;
+ pdata->ndevs->part = idt_ntb_readfld_var(portsts, IDT_SW_PORT_SWPART);
+
+ /* Initialize the local topology and PCI device fields */
+ pdata->ndevs->ntb.topo = pdata->role;
+ pdata->ndevs->ntb.pdev = pdata->pdev;
+
+ return SUCCESS;
+}
+
+/*
+ * Enumerate the peer pairs
+ *
+ * Basically the pairid is just the order number of the corresponding
+ * Secondary side port. So the function just loop over the Primary ports.
+ * If the local port is Primary then just linearly enumerate its peers
+ * starting from the corresponding number.
+ * If the local port is Secondary then the function walks through
+ * all the Secondary port of the corresponding Primary port looking
+ * for the current one to assign the simultaniously incremented id.
+ */
+static void idt_ntb_enumpairs(struct idt_ntb_data *pdata)
+{
+ struct idt_ntb_topo *topo = &pdata->topo;
+ unsigned char id, pairid = 0;
+ unsigned long priport, secport;
+ u32 secports;
+
+ /* Loop over all the Primary ports calculating the pairids */
+ for_each_set_bit_u32(topo->priports, priport) {
+ /* Retrieve the Secondary ports connected to the current
+ * Primary one */
+ secports = topo->secports[priport];
+
+ /* Enumerate the current port related pairs */
+ /* If current port is Primary then enumerate its peers */
+ if (NTB_TOPO_PRI == pdata->role && priport == pdata->port) {
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ pdata->ndevs[id].pairid = pairid + id;
+ }
+ /* Stop looping, the job is done */
+ break;
+ }
+ /* If the current port is Secondary then retrieve its peer id
+ * within the corresponding Primary port */
+ else if (NTB_TOPO_SEC == pdata->role &&
+ priport == pdata->ndevs[0].port) {
+ id = 0;
+ for_each_set_bit_u32(secports, secport) {
+ if (secport == pdata->port) {
+ pdata->ndevs[0].pairid = pairid + id;
+ break;
+ }
+ id++;
+ }
+ /* Stop looping, the job is done */
+ break;
+ }
+
+ /* Increment the pairid with the number of the related Secondary
+ * ports */
+ pairid += hweight32(secports);
+ }
+}
+
+/*
+ * Create the NTB devices with respect to the topology
+ */
+static int idt_ntb_addpeers(struct idt_ntb_data *pdata)
+{
+ void __iomem *cfg = pdata->cfg_mmio;
+ struct idt_ntb_topo *topo = &pdata->topo;
+ u32 portsts;
+ int ret;
+
+ /* Retrieve the current port number */
+ pdata->port = idt_ntb_readfld_mem(cfg, IDT_NT_PORTNUM);
+
+ /* Read the current port partition number */
+ portsts = idt_ntb_readreg(cfg, portdata_tbl[pdata->port].sts);
+ pdata->part = idt_ntb_readfld_var(portsts, IDT_SW_PORT_SWPART);
+
+ /* Check whether the current port role is Primary or Secondary */
+ pdata->role = idt_ntb_portrole(topo, pdata->port);
+
+ /* Create either the Primary or Secondary side peers set */
+ ret = (NTB_TOPO_PRI == pdata->role) ?
+ idt_ntb_secpeers(pdata) : idt_ntb_pripeer(pdata);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+
+ /* Enumerate all the NTB connected pairs */
+ idt_ntb_enumpairs(pdata);
+
+ dev_dbg_data(pdata, "IDT NTB peer devices created");
+
+ return SUCCESS;
+}
+
+/*
+ * Remove the peer NTB devices added to the data structure
+ */
+static void idt_ntb_delpeers(struct idt_ntb_data *pdata)
+{
+ /* Release the memory occupied by the */
+ kfree(pdata->ndevs);
+
+ dev_dbg_data(pdata, "IDT NTB peer devices discarded");
+}
+
+/*===========================================================================
+ * 10. Basic initialization functions
+ *===========================================================================*/
+
+/*
+ * Check whether the device is properly pre-initialized
+ */
+static int idt_ntb_check_quirks(struct pci_dev *pdev)
+{
+ u32 data, fld;
+ int ret;
+
+ /* Read the BARSETUP0 */
+ ret = pci_read_config_dword(pdev, BARSETUP0_OFF, &data);
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to read BARSETUP0 configuration register");
+ return ret;
+ }
+
+ /* Check whether the BAR0 register is enabled */
+ if (OFF == idt_ntb_readfld_var(data, IDT_NT_BARSTP_EN)) {
+ dev_err(&pdev->dev,
+ "BAR0 isn't enabled");
+ return -EINVAL;
+ }
+
+ /* Check whether the BAR0 maps the registers configuration space */
+ fld = idt_ntb_readfld_var(data, IDT_NT_BARSTP_MODE);
+ if (BARSTP_MODE_CFGSPC != fld) {
+ dev_err(&pdev->dev,
+ "BAR0 isn't configured to map the configuration space");
+ return -EINVAL;
+ }
+
+ /* Read the BARSETUP2 */
+ ret = pci_read_config_dword(pdev, BARSETUP2_OFF, &data);
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to read BARSETUP2 configuration register");
+ return ret;
+ }
+
+ /* Check whether the BAR2 register is enabled */
+ if (OFF == idt_ntb_readfld_var(data, IDT_NT_BARSTP_EN)) {
+ dev_err(&pdev->dev,
+ "BAR2 isn't enabled");
+ return -EINVAL;
+ }
+
+ /* Check whether the BAR2 maps memory windows */
+ fld = idt_ntb_readfld_var(data, IDT_NT_BARSTP_MODE);
+ if (BARSTP_MODE_WNDW != fld) {
+ dev_err(&pdev->dev,
+ "BAR2 isn't configured to map memory windows");
+ return -EINVAL;
+ }
+
+ /* Check whether the BAR2 maps the 24-entries lookup table */
+ fld = idt_ntb_readfld_var(data, IDT_NT_BARSTP_ATRAN);
+ if (BARSTP_ATRAN_LU24 != fld) {
+ dev_err(&pdev->dev,
+ "BAR2 isn't configured to map 24-entries lookup table");
+ return -EINVAL;
+ }
+
+ return SUCCESS;
+}
+
+/*
+ * Create the IDT PCIe-swtich driver data structure performing the basic
+ * initialization
+ */
+static struct idt_ntb_data *idt_ntb_create_data(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct idt_ntb_data *pdata;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&pdev->dev);
+ pdata = kzalloc_node(sizeof(*pdata), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(pdata)) {
+ dev_err(&pdev->dev,
+ "Failed to allocate memory for IDT NTB driver data");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Create the workqueue used by the driver */
+ pdata->idt_wq = create_workqueue(NTB_WQNAME);
+ if (IS_ERR_OR_NULL(pdata->idt_wq)) {
+ dev_err(&pdev->dev, "Failed to create workqueue");
+ goto err_kfree;
+ }
+
+ /* Put the IDT driver data pointer to the PCI-device private pointer */
+ pci_set_drvdata(pdev, pdata);
+ /* Save the PCI-device pointer inside the data structure */
+ pdata->pdev = pdev;
+ /* Save the IDT PCIe-switch ports configuration */
+ pdata->swcfg = (struct idt_89hpes_pdata *)id->driver_data;
+
+ dev_dbg_data(pdata, "IDT NTB device data created");
+
+ return pdata;
+
+err_kfree:
+ kfree(pdata);
+
+ return NULL;
+}
+
+/*
+ * Free the IDT PCie-swtich driver data structure
+ */
+static void idt_ntb_free_data(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+
+ /* Flush and destroy the workqueue */
+ flush_workqueue(pdata->idt_wq);
+ destroy_workqueue(pdata->idt_wq);
+
+ /* Clean the private data pointer of the PCI-device structure */
+ pci_set_drvdata(pdev, NULL);
+
+ /* Free the memory allocated for the IDT NTB driver data */
+ kfree(pdata);
+
+ dev_dbg(&pdev->dev, "IDT NTB device data discarded");
+}
+
+/*
+ * Initialize the basic PCI-related subsystem
+ */
+static int idt_ntb_init_pci(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+ int ret;
+
+ /* Enable the device advanced error reporting. Don't check the return
+ * value since the service might be disabled from the kernel */
+ ret = pci_enable_pcie_error_reporting(pdev);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to enable AER capability of IDT NTB");
+ }
+ /* Cleanup the uncorrectable error status before starting the rest of
+ * initialization */
+ pci_cleanup_aer_uncorrect_error_status(pdev);
+
+ /* First enable the PCI device */
+ ret = pci_enable_device(pdev);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to enable the PCI device");
+ goto err_disable_aer;
+ }
+
+ /* Reguest the PCI device resources like the BAR memory mapping, etc
+ * It's done for BAR0 for now */
+ ret = pci_request_region(pdev, BAR0, NTB_NAME);
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Failed to request the PCI BAR0 resources");
+ goto err_disable_device;
+ }
+
+ /* Initialize the bit mask of DMA although I don't see where it can be
+ * used for now */
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (SUCCESS != ret) {
+ ret = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (SUCCESS != ret) {
+ dev_err_data(pdata, "Failed to set any DMA bit mask\n");
+ goto err_release_region;
+ }
+ dev_warn_data(pdata, "Cannot set the DMA highmem bit mask\n");
+ }
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (SUCCESS != ret) {
+ ret = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (SUCCESS != ret) {
+ dev_err_data(pdata,
+ "Failed to set any consistent DMA bit mask\n");
+ goto err_release_region;
+ }
+ dev_warn_data(pdata,
+ "Cannot set the consistent DMA highmem bit mask\n");
+ }
+
+ /* Retrieve the virtual address of the PCI configuration space */
+ pdata->cfg_mmio = pci_iomap(pdev, BAR0, 0);
+ if (IS_ERR_OR_NULL(pdata->cfg_mmio)) {
+ dev_err_data(pdata,
+ "Failed to map the IDT NT-function config space\n");
+ ret = -EIO;
+ goto err_release_region;
+ }
+
+ dev_dbg_data(pdata, "IDT NTB function PCI interface was initialized");
+
+ return SUCCESS;
+
+err_disable_aer:
+ (void)pci_disable_pcie_error_reporting(pdev);
+err_release_region:
+ pci_release_region(pdev, BAR0);
+err_disable_device:
+ pci_disable_device(pdev);
+
+ return ret;
+}
+
+/*
+ * Deinitialize the basic PCI-related subsystem
+ */
+static void idt_ntb_deinit_pci(struct idt_ntb_data *pdata)
+{
+ struct pci_dev *pdev = pdata->pdev;
+
+ /* Disable the AER capability */
+ (void)pci_disable_pcie_error_reporting(pdev);
+
+ /* Unmap the IDT PCIe-switch configuration space */
+ pci_iounmap(pdev, pdata->cfg_mmio);
+
+ /* Release the PCI-device BAR0 resources */
+ pci_release_region(pdev, BAR0);
+
+ /* Finally disable the PCI device */
+ pci_disable_device(pdev);
+
+ dev_dbg_data(pdata, "IDT NTB function PCI interface was cleaned");
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t idt_ntb_dbgfs_info_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+static ssize_t idt_ntb_dbgfs_ntregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+static ssize_t idt_ntb_dbgfs_swregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+/*
+ * Driver DebugFS info file operations
+ */
+static const struct file_operations idt_ntb_dbgfs_info_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = idt_ntb_dbgfs_info_read
+};
+
+/*
+ * Driver DebugFS NT registers file operations
+ */
+static const struct file_operations idt_ntb_dbgfs_ntregs_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = idt_ntb_dbgfs_ntregs_read
+};
+
+/*
+ * Driver DebugFS IDT PCIe-swtich global registers file operations
+ */
+static const struct file_operations idt_ntb_dbgfs_swregs_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = idt_ntb_dbgfs_swregs_read
+};
+
+/*
+ * DebugFS read info node callback
+ */
+static ssize_t idt_ntb_dbgfs_info_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ struct idt_ntb_data *pdata = filp->private_data;
+ void __iomem *cfg = pdata->cfg_mmio;
+ enum ntb_speed speed;
+ enum ntb_width width;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+ u32 var;
+ int id, sts, part, bdf, port;
+ unsigned char temp, frac;
+
+ /* Lets limit the buffer size the way the Intel/AMD drivers do */
+ size = min_t(size_t, count, 0x1000U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tIDT PCIe-switch NT-function Information:\n\n");
+
+ /* General device configurations */
+ off += scnprintf(strbuf + off, size - off,
+ "Switch port\t\t\t- %hhu\n", pdata->port);
+ off += scnprintf(strbuf + off, size - off,
+ "Port partition\t\t\t- %hhu\n", pdata->part);
+ off += scnprintf(strbuf + off, size - off,
+ "Number of peers\t\t\t- %hhu\n", pdata->peer_cnt);
+
+ /* Local switch NT-function role topology and available port to
+ * communicate to */
+ off += scnprintf(strbuf + off, size - off,
+ "NT-function role\t\t- %s\n", ntb_topo_string(pdata->role));
+ off += scnprintf(strbuf + off, size - off,
+ "Peer Port:Partition available\t- ");
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "%hhd:%hhd ",
+ pdata->ndevs[id].port, pdata->ndevs[id].part);
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+ /* Links status */
+ var = idt_ntb_readreg(cfg, portdata_tbl[pdata->port].sts);
+ if (idt_ntb_readfld_var(var, IDT_SW_PORT_LNKUP)) {
+ off += scnprintf(strbuf + off, size - off,
+ "Local Port Link status\t\t- ");
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_PCIELSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "PCIe Gen %u ",
+ idt_ntb_readfld_var(var, IDT_NT_CURLNKSPD));
+ off += scnprintf(strbuf + off, size - off,
+ "x%u lanes\n",
+ idt_ntb_readfld_var(var, IDT_NT_CURLNKWDTH));
+ } else {
+ off += scnprintf(strbuf + off, size - off,
+ "Local port link status\t\t- Down (Weird)\n");
+ }
+ off += scnprintf(strbuf + off, size - off,
+ "Peer ports link status\t\t- ");
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ sts = idt_ntb_link_is_up(&pdata->ndevs[id].ntb, &speed, &width);
+ if (ON == sts) {
+ off += scnprintf(strbuf + off, size - off,
+ "%hhd:Gen %u x%u, ", pdata->ndevs[id].port, speed, width);
+ } else /* if (OFF == sts) */ {
+ off += scnprintf(strbuf + off, size - off,
+ "%hhd:Down, ", pdata->ndevs[id].port);
+ }
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+ /* General resources information */
+ off += scnprintf(strbuf + off, size - off,
+ "Total doorbells count\t\t- %u\n", IDT_NTB_DBELL_CNT);
+ off += scnprintf(strbuf + off, size - off,
+ "Total memory windows count\t- %u\n", IDT_NTB_MW_CNT);
+ off += scnprintf(strbuf + off, size - off,
+ "Total message registers count\t- %u\n", IDT_NTB_MSG_CNT);
+
+ /* Common resources state */
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_GDBELLSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Global doorbells status\t\t- %#010x\n", var);
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_INDBELLSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Local doorbells status\t\t- %#010x\n", var);
+ off += scnprintf(strbuf + off, size - off,
+ "Mirror doorbells value\t\t- %#010x\n",
+ pdata->db_sts);
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_INDBELLMSK);
+ off += scnprintf(strbuf + off, size - off,
+ "Local doorbells mask\t\t- %#010x\n", var);
+ off += scnprintf(strbuf + off, size - off,
+ "Mirror doorbells mask value\t- %#010x\n",
+ pdata->db_msk);
+
+ /* Per-device resources */
+ for (id = 0; id < pdata->peer_cnt; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "Port %hhd (pair id %hhd)\n",
+ pdata->ndevs[id].port, pdata->ndevs[id].pairid);
+ off += scnprintf(strbuf + off, size - off,
+ "\tDoorbells share\t- "
+ "local %#010x offset %hhu, peer %#010x offset %hhu\n",
+ pdata->ndevs[id].db_self_mask,
+ pdata->ndevs[id].db_self_offset,
+ pdata->ndevs[id].db_peer_mask,
+ pdata->ndevs[id].db_peer_offset);
+ off += scnprintf(strbuf + off, size - off,
+ "\tDoorbells\t- count %hhu, valid mask: %#010x,\n",
+ pdata->ndevs[id].db_cnt,
+ pdata->ndevs[id].db_valid_mask);
+ off += scnprintf(strbuf + off, size - off,
+ "\tMemory windows\t- local/peer count %hhu/%hhu, "
+ "size %u bytes, local offset: %hhu\n",
+ pdata->ndevs[id].mw_self_cnt,
+ pdata->ndevs[id].mw_peer_cnt,
+ (unsigned int)pdata->mw_size,
+ pdata->ndevs[id].mw_self_offset);
+ }
+
+ /* Doorbells mapping */
+ off += scnprintf(strbuf + off, size - off,
+ "\nInbound db:part mapping\n\t");
+ for (id = 0; id < IDT_NTB_DBELL_CNT; id++) {
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_GIDBELLMSK0 + id);
+ off += scnprintf(strbuf + off, size - off, "%02d:", id);
+ for_each_set_bit_u32(~var & 0xFF, part) {
+ off += scnprintf(strbuf + off, size - off, "%d,", part);
+ }
+ off += scnprintf(strbuf + off, size - off, "\b; ");
+ if (0 == ((id + 1) % 10)) {
+ off += scnprintf(strbuf + off, size - off, "\n\t");
+ }
+ }
+ off += scnprintf(strbuf + off, size - off,
+ "\nOutbound db:part mapping\n\t");
+ for (id = 0; id < IDT_NTB_DBELL_CNT; id++) {
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_GODBELLMSK0 + id);
+ off += scnprintf(strbuf + off, size - off, "%02d:", id);
+ for_each_set_bit_u32(~var & 0xFF, part) {
+ off += scnprintf(strbuf + off, size - off,
+ "%d,", part);
+ }
+ off += scnprintf(strbuf + off, size - off, "\b; ");
+ if (0 == ((id + 1) % 10)) {
+ off += scnprintf(strbuf + off, size - off, "\n\t");
+ }
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+ /* NTB control register */
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_NTCTL);
+ off += scnprintf(strbuf + off, size - off,
+ "\nNTB control register\t- %#010x\n", var);
+
+ /* NTB Mapping table */
+ off += scnprintf(strbuf + off, size - off,
+ "NTB mapping table\n");
+ for (id = 0; id < IDT_NTB_MTBL_ENTRY_CNT; id++) {
+ idt_ntb_writereg(cfg, IDT_NT_PCI_NTMTBLADDR, (u32)id);
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_NTMTBLDATA);
+ if (ON == idt_ntb_readfld_var(var, IDT_NT_MTBL_VALID)) {
+ bdf = idt_ntb_readfld_var(var, IDT_NT_MTBL_BDF);
+ off += scnprintf(strbuf + off, size - off,
+ "\t%02d: part %d, bus %d, dev %d, func %d\n",
+ id, idt_ntb_readfld_var(var, IDT_NT_MTBL_PART),
+ (bdf >> 8) & 0xFF, (bdf >> 3) & 0x1F, bdf & 7);
+ }
+ }
+
+ /* Currently enabled IRQs */
+ off += scnprintf(strbuf + off, size - off, "\nNTB interrupts status\n");
+ var = idt_ntb_readreg(cfg, IDT_NT_PCI_NTINTMSK);
+ for_each_set_bit_u32(ALLINT_MASK, id) {
+ switch (id) {
+ case MSGINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tMessage interrupts\t\t\t\t- ");
+ break;
+ case DBINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tDoorbell interrupts\t\t\t\t- ");
+ break;
+ case SEINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tSwitch event interrupts\t\t\t\t- ");
+ break;
+ case FMCI_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tFailover mode change initiated IRQ\t\t- ");
+ break;
+ case FMCC_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tFailover mode change completed IRQ\t\t- ");
+ break;
+ case TEMPINT_BIT:
+ off += scnprintf(strbuf + off, size - off,
+ "\tTemperature sensor IRQ (T < %d || %d < T)\t- ",
+ IDT_NTB_TEMP_LTH, IDT_NTB_TEMP_HTH);
+ break;
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "\tWarning! Invalid bit is set in the NTINTMSK register\n");
+ break;
+ }
+
+ if (0x0 == (var & BIT_MASK(id))) {
+ off += scnprintf(strbuf + off, size - off,
+ "enabled\n");
+ } else {
+ off += scnprintf(strbuf + off, size - off,
+ "disabled\n");
+ }
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tIDT PCIe-switch general configuration:\n\n");
+
+ /* Boot configuration vector status */
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_BCVSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Switch boot mode\n\t");
+ switch (idt_ntb_readfld_var(var, IDT_SW_SWMODE)) {
+ case (0x0):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition\n");
+ break;
+ case (0x1):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition with Serial EEPROM\n");
+ break;
+ case (0x2):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition with Serial EEPROM Jump 0 "
+ "Initialization\n");
+ break;
+ case (0x3):
+ off += scnprintf(strbuf + off, size - off,
+ "Single Partition with Serial EEPROM Jump 1 "
+ "Initialization\n");
+ break;
+ case (0x8):
+ off += scnprintf(strbuf + off, size - off,
+ "Single partition with reduced latency\n");
+ break;
+ case (0x9):
+ off += scnprintf(strbuf + off, size - off,
+ "Single partition with Serial EEPROM initialization "
+ "and reduced latency\n");
+ break;
+ case (0xA):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports\n");
+ break;
+ case (0xB):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports and i2c Reset\n");
+ break;
+ case (0xC):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports and Serial EEPROM "
+ "initialization\n");
+ break;
+ case (0xD):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Unattached ports with i2c Reset "
+ "and Serial EEPROM initialization\n");
+ break;
+ case (0xE):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Disabled ports\n");
+ break;
+ case (0xF):
+ off += scnprintf(strbuf + off, size - off,
+ "Multi-partition with Disabled ports and Serial EEPROM "
+ "initialization\n");
+ break;
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "Unknown\n");
+ break;
+ }
+ off += scnprintf(strbuf + off, size - off,
+ "Switch boot clock mode\n\t");
+ switch (idt_ntb_readfld_var(var, IDT_SW_CLKMODE)) {
+ case (0x0):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- non-common global clocked\n"
+ "\tOther ports\t- non-common global clocked\n");
+ break;
+ case (0x1):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- Common global clocked\n"
+ "\tOther ports\t- non-common global clocked\n");
+ break;
+ case (0x2):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- non-common global clocked\n"
+ "\tOther ports\t- common global clocked\n");
+ break;
+ case (0x3):
+ off += scnprintf(strbuf + off, size - off,
+ "Port 0\t\t- common global clocked\n"
+ "\tOther ports\t- common global clocked\n");
+ break;
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "Unknown\n");
+ break;
+ }
+
+ /* Per-port link status and clock configuration */
+ off += scnprintf(strbuf + off, size - off,
+ "Ports clocking status\n");
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_PCLKMODE);
+ for (id = 0; id < pdata->swcfg->port_cnt; id++) {
+ port = pdata->swcfg->ports[id];
+ sts = idt_ntb_readreg(cfg, portdata_tbl[port].pcielsts);
+ off += scnprintf(strbuf + off, size - off,
+ "\tPort %d\t- %s %s mode\n", port,
+ idt_ntb_readfld_var(sts, IDT_NT_SCLK) ?
+ "common" : "non-common",
+ idt_ntb_readfld_var(var, IDT_SW_P0CLKMODE + id) ?
+ "local" : "global");
+ }
+
+ /* SMBus configuration */
+ var = idt_ntb_readreg(cfg, IDT_SW_PCI_SMBUSSTS);
+ off += scnprintf(strbuf + off, size - off,
+ "Slave SMBus address\t- %#04x\n",
+ idt_ntb_readfld_var(var, IDT_SW_SSMBADDR));
+ off += scnprintf(strbuf + off, size - off,
+ "Master SMBus address\t- %#04x\n",
+ idt_ntb_readfld_var(var, IDT_SW_MSMBADDR));
+
+ /* Current temperature */
+ var = idt_ntb_readfld_mem(cfg, IDT_SW_TMP_CURTEMP);
+ idt_ntb_convert_temp(var, &temp, &frac);
+ off += scnprintf(strbuf + off, size - off,
+ "Switch temperature\t- %d.%dC\n", temp, (0 != frac) ? 5 : 0);
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, count, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * Read passed set of registers method for DebugFS nodes
+ */
+static ssize_t idt_ntb_dbgfs_regs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp,
+ enum idt_ntb_cfgreg start,
+ enum idt_ntb_cfgreg end,
+ const char *title)
+{
+ struct idt_ntb_data *pdata = filp->private_data;
+ void __iomem *cfg = pdata->cfg_mmio;
+ enum idt_ntb_cfgreg reg;
+ enum idt_ntb_regtype regtype;
+ ptrdiff_t regoffset;
+ enum idt_ntb_regsize regsize;
+ const char *regdesc;
+ u32 data;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+
+ /* Lets limit the buffer size the way the Intel/AMD drivers do */
+ size = min_t(size_t, count, 0x4000U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ return -ENOMEM;
+ }
+
+ /* Put the title first */
+ off += scnprintf(strbuf + off, size - off, "\n\t\t%s\n\n", title);
+
+ /* Print the header of the registers */
+ off += scnprintf(strbuf + off, size - off, " 03 02 01 00\n");
+
+ /* Scan through the whole passed range reading the addresses, values
+ * and description and printing it to the buffer */
+ for (reg = start; reg < end; reg++) {
+ /* Retrieve the register type, offset, size and description */
+ idt_ntb_regparams(reg, ®type, ®offset, ®size, ®desc);
+
+ /* Read the value of the corresponding register */
+ data = idt_ntb_readreg(cfg, reg);
+
+ /* Print the register offset */
+ off += scnprintf(strbuf + off, size - off,
+ "0x%05lX: ", (unsigned long)regoffset);
+
+ /* Then print the value of the register in compliance with the
+ * register size */
+ switch (regsize) {
+ case REGBYTE:
+ off += scnprintf(strbuf + off, size - off,
+ " %02hhX", data);
+ break;
+ case REGWORD:
+ off += scnprintf(strbuf + off, size - off,
+ " %02hhX %02hhX", (data >> 8), data);
+ break;
+ case REGDWORD:
+ default:
+ off += scnprintf(strbuf + off, size - off,
+ "%02hhX %02hhX %02hhX %02hhX",
+ (data >> 24), (data >> 16), (data >> 8), data);
+ break;
+ }
+
+
+ /* Then description if going to be the last */
+ off += scnprintf(strbuf + off, size - off,
+ " - %s\n", regdesc);
+ }
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, count, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS read NT-function registers node callback
+ */
+static ssize_t idt_ntb_dbgfs_ntregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ ssize_t size;
+
+ /* Read the values of the NT-related registers */
+ size = idt_ntb_dbgfs_regs_read(filp, ubuf, count, offp,
+ 0, IDT_NTB_CFGREGS_SPLIT, "NT-function registers raw values");
+
+ return size;
+}
+
+/*
+ * DebugFS read IDT PCIe-switch registers node info callback
+ */
+static ssize_t idt_ntb_dbgfs_swregs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ ssize_t size;
+
+ /* Read the values of the IDT PCIe-swtich global registers */
+ size = idt_ntb_dbgfs_regs_read(filp, ubuf, count, offp,
+ IDT_NTB_CFGREGS_SPLIT + 1, IDT_NTB_CFGREGS_END,
+ "IDT PCIe-switch global registers raw values");
+
+ return size;
+}
+
+/*
+ * Driver DebugFS initialization function
+ */
+static int idt_ntb_init_dbgfs(struct idt_ntb_data *pdata)
+{
+ const char *devname;
+ struct dentry *dbgfs_info, *dbgfs_ntregs, *dbgfs_swregs;
+ int ret = 0;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_topdir)) {
+ dev_info_data(pdata,
+ "Top DebugFS directory has not been created for "
+ NTB_NAME);
+ return PTR_ERR(dbgfs_topdir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(to_dev_data(pdata));
+
+ /* Create the top directory of the device */
+ pdata->dbgfs_dir = debugfs_create_dir(devname, dbgfs_topdir);
+ if (IS_ERR(pdata->dbgfs_dir)) {
+ dev_dbg_data(pdata, "Could not create the DebugFS dir %s for %s",
+ devname, NTB_NAME);
+ return PTR_ERR(pdata->dbgfs_dir);
+ }
+
+ /* Create the info file node */
+ dbgfs_info = debugfs_create_file("info", S_IRUSR,
+ pdata->dbgfs_dir, pdata, &idt_ntb_dbgfs_info_ops);
+ if (IS_ERR(dbgfs_info)) {
+ dev_dbg_data(pdata, "Could not create the DebugFS info node");
+ ret = PTR_ERR(dbgfs_info);
+ goto err_rm_dir;
+ }
+
+ /* Create the NT-registers file node */
+ dbgfs_ntregs = debugfs_create_file("ntregs", S_IRUSR,
+ pdata->dbgfs_dir, pdata, &idt_ntb_dbgfs_ntregs_ops);
+ if (IS_ERR(dbgfs_ntregs)) {
+ dev_dbg_data(pdata,
+ "Could not create the DebugFS NT-registers node");
+ ret = PTR_ERR(dbgfs_ntregs);
+ goto err_rm_dir;
+ }
+
+ /* Create the NT-registers file node */
+ dbgfs_swregs = debugfs_create_file("swregs", S_IRUSR,
+ pdata->dbgfs_dir, pdata, &idt_ntb_dbgfs_swregs_ops);
+ if (IS_ERR(dbgfs_swregs)) {
+ dev_dbg_data(pdata,
+ "Could not create the DebugFS global registers node");
+ ret = PTR_ERR(dbgfs_swregs);
+ goto err_rm_dir;
+ }
+
+ dev_dbg_data(pdata, "IDT NTB device DebugFS nodes created");
+
+ return SUCCESS;
+
+ /* Following call will remove all the subfiles in the directory */
+err_rm_dir:
+ debugfs_remove_recursive(pdata->dbgfs_dir);
+
+ return ret;
+}
+
+/*
+ * Driver DebugFS deinitialization function
+ */
+static void idt_ntb_deinit_dbgfs(struct idt_ntb_data *pdata)
+{
+ debugfs_remove_recursive(pdata->dbgfs_dir);
+
+ dev_dbg_data(pdata, "IDT NTB device DebugFS nodes discarded");
+}
+
+/*===========================================================================
+ * 12. PCI bus callback functions
+ *===========================================================================*/
+
+/*
+ * PCI device probe() callback function
+ */
+static int idt_ntb_pci_probe(struct pci_dev *pdev,
+ const struct pci_device_id *id)
+{
+ struct idt_ntb_data *pdata;
+ int ret;
+
+ /* Check whether the kernel has properly fixed the IDT NTB
+ * function up */
+ ret = idt_ntb_check_quirks(pdev);
+ if (SUCCESS != ret) {
+ return ret;
+ }
+
+ /* Allocate the memory for the IDT PCIe-swtich NTB driver data */
+ pdata = idt_ntb_create_data(pdev, id);
+ if (IS_ERR_OR_NULL(pdata)) {
+ return PTR_ERR(pdata);
+ }
+
+ /* Initialize the basic PCI subsystem of the device */
+ ret = idt_ntb_init_pci(pdata);
+ if (SUCCESS != ret) {
+ goto err_free_data;
+ }
+
+ /* Determine the ports NT-functions predefined topology */
+ ret = idt_ntb_scantopo(pdata);
+ if (SUCCESS != ret) {
+ goto err_deinit_pci;
+ }
+
+ /* Add all the peers */
+ ret = idt_ntb_addpeers(pdata);
+ if (SUCCESS != ret) {
+ goto err_deinit_pci;
+ }
+
+ /* Initialize the doorbells */
+ idt_ntb_init_db(pdata);
+
+ /* Allocate the Memory Window resources */
+ ret = idt_ntb_init_mws(pdata);
+ if (SUCCESS != ret) {
+ goto err_freedb;
+ }
+
+ /* Init messaging subsystem */
+ ret = idt_ntb_init_msg(pdata);
+ if (SUCCESS != ret) {
+ goto err_clean_mws;
+ }
+
+ /* Start the link polling subsystem */
+ idt_ntb_init_link(pdata);
+
+ /* Initialize the PCIe interrupts */
+ ret = idt_ntb_init_isr(pdata);
+ if (SUCCESS != ret) {
+ goto err_clear_link;
+ }
+
+ /* Register all the devices on the NTB bus */
+ ret = idt_ntb_register_devs(pdata);
+ if (SUCCESS != ret) {
+ goto err_clear_isr;
+ }
+
+ /* Initialize the DebugFS node of the IDT PCIe-switch NTB driver.
+ * Don't pay much attention to this even if it failed */
+ (void)idt_ntb_init_dbgfs(pdata);
+
+ /* IDT PCIe-switch NTB driver is finally initialized */
+ dev_info_data(pdata, "IDT PCIe-swtich NTB devices are ready");
+
+ /* May the force be with us... */
+ return SUCCESS;
+
+err_clear_isr:
+ idt_ntb_clear_isr(pdata);
+err_clear_link:
+ idt_ntb_clear_link(pdata);
+/*err_deinit_msg:*/
+ idt_ntb_deinit_msg(pdata);
+err_clean_mws:
+ idt_ntb_clean_mws(pdata);
+err_freedb:
+ idt_ntb_clean_db(pdata);
+/*err_delpeers:*/
+ idt_ntb_delpeers(pdata);
+err_deinit_pci:
+ idt_ntb_deinit_pci(pdata);
+err_free_data:
+ idt_ntb_free_data(pdata);
+
+ return ret;
+}
+
+/*
+ * PCI device remove() callback function
+ */
+static void idt_ntb_pci_remove(struct pci_dev *pdev)
+{
+ struct idt_ntb_data *pdata = pci_get_drvdata(pdev);
+
+ /* Deinit the DebugFS node */
+ idt_ntb_deinit_dbgfs(pdata);
+
+ /* Unregister the devices from the NTB bus */
+ idt_ntb_unregister_devs(pdata);
+
+ /* Stop the interrupt handler */
+ idt_ntb_clear_isr(pdata);
+
+ /* Stop the link polling subsystem */
+ idt_ntb_clear_link(pdata);
+
+ /* Deinitialize the messaging subsystem */
+ idt_ntb_deinit_msg(pdata);
+
+ /* Clear the memory windows */
+ idt_ntb_clean_mws(pdata);
+
+ /* Free the allocated Doorbells */
+ idt_ntb_clean_db(pdata);
+
+ /* Delete the added peer devices */
+ idt_ntb_delpeers(pdata);
+
+ /* Deinit the basic PCI subsystem */
+ idt_ntb_deinit_pci(pdata);
+
+ /* Free the memory occupied by the data */
+ idt_ntb_free_data(pdata);
+
+ /* IDT PCIe-switch NTB driver is finally initialized */
+ dev_info(&pdev->dev, "IDT PCIe-swtich NTB devices are unloaded");
+
+ /* Sayonara... */
+}
+
+/*
+ * IDT PCIe-switch models ports configuration structures
+ */
+static struct idt_89hpes_pdata idt_89hpes24nt6ag2_config = {
+ .port_cnt = 6, .ports = {0, 2, 4, 6, 8, 12}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt8ag2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt8bg2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes12nt12g2_config = {
+ .port_cnt = 3, .ports = {0, 8, 16}
+};
+static struct idt_89hpes_pdata idt_89hpes16nt16g2_config = {
+ .port_cnt = 4, .ports = {0, 8, 12, 16}
+};
+static struct idt_89hpes_pdata idt_89hpes24nt24g2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt24ag2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+static struct idt_89hpes_pdata idt_89hpes32nt24bg2_config = {
+ .port_cnt = 8, .ports = {0, 2, 4, 6, 8, 12, 16, 20}
+};
+
+/*
+ * PCI-ids table of the supported IDT PCIe-switch devices
+ */
+static const struct pci_device_id idt_ntb_pci_tbl[] = {
+ {IDT_PCI_DEVICE_IDS(89HPES24NT6AG2, idt_89hpes24nt6ag2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT8AG2, idt_89hpes32nt8ag2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT8BG2, idt_89hpes32nt8bg2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES12NT12G2, idt_89hpes12nt12g2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES16NT16G2, idt_89hpes16nt16g2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES24NT24G2, idt_89hpes24nt24g2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT24AG2, idt_89hpes32nt24ag2_config)},
+ {IDT_PCI_DEVICE_IDS(89HPES32NT24BG2, idt_89hpes32nt24bg2_config)},
+ {0}
+};
+MODULE_DEVICE_TABLE(pci, idt_ntb_pci_tbl);
+
+/*
+ * IDT PCIe-switch NT-function device driver structure definition
+ */
+static struct pci_driver idt_ntb_pci_driver = {
+ .name = KBUILD_MODNAME,
+ .probe = idt_ntb_pci_probe,
+ .remove = idt_ntb_pci_remove,
+ .id_table = idt_ntb_pci_tbl,
+};
+
+static int __init idt_ntb_pci_driver_init(void)
+{
+ pr_info("%s %s\n", NTB_DESC, NTB_VER);
+
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_topdir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Register the NTB hardware driver to handle the PCI device */
+ return pci_register_driver(&idt_ntb_pci_driver);
+}
+module_init(idt_ntb_pci_driver_init);
+
+static void __exit idt_ntb_pci_driver_exit(void)
+{
+ /* Unregister the NTB hardware driver */
+ pci_unregister_driver(&idt_ntb_pci_driver);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_topdir);
+}
+module_exit(idt_ntb_pci_driver_exit);
+
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt.h b/drivers/ntb/hw/idt/ntb_hw_idt.h
new file mode 100644
index 0000000..ffe3327
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt.h
@@ -0,0 +1,390 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+#ifndef NTB_HW_IDT_H
+#define NTB_HW_IDT_H
+
+#include <linux/types.h>
+#include <linux/ntb.h>
+#include <linux/pci.h>
+#include <linux/pci_ids.h>
+#include <linux/interrupt.h>
+#include <linux/spinlock.h>
+#include <linux/list.h>
+
+#include "ntb_hw_idt_regmap.h"
+
+/*
+ * Macro is used to create the struct pci_device_id that matches
+ * the supported IDT PCIe-switches
+ * @devname: Capitalized name of the particular device
+ * @data: Variable passed to the driver of the particular device
+ */
+#define IDT_PCI_DEVICE_IDS(devname, data) \
+ .vendor = PCI_VENDOR_ID_IDT, .device = PCI_DEVICE_ID_IDT_##devname, \
+ .subvendor = PCI_ANY_ID, .subdevice = PCI_ANY_ID, \
+ .class = (PCI_CLASS_BRIDGE_OTHER << 8), .class_mask = (0xFFFF00), \
+ .driver_data = (kernel_ulong_t)&data
+
+/*
+ * IDT PCIe-switches device IDs
+ */
+#define PCI_DEVICE_ID_IDT_89HPES24NT6AG2 0x8091
+#define PCI_DEVICE_ID_IDT_89HPES32NT8AG2 0x808F
+#define PCI_DEVICE_ID_IDT_89HPES32NT8BG2 0x8088
+#define PCI_DEVICE_ID_IDT_89HPES12NT12G2 0x8092
+#define PCI_DEVICE_ID_IDT_89HPES16NT16G2 0x8090
+#define PCI_DEVICE_ID_IDT_89HPES24NT24G2 0x808E
+#define PCI_DEVICE_ID_IDT_89HPES32NT24AG2 0x808C
+#define PCI_DEVICE_ID_IDT_89HPES32NT24BG2 0x808A
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ * @BAR0: Operation with BAR0
+ * @BAR2: Operation with BAR2
+ * @BAR4: Operation with BAR4
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+#define BAR0 0
+#define BAR2 2
+#define BAR4 4
+
+/*
+ * Inline helper function to perform the for each set bit looping.
+ *
+ * NOTE We don't use the standard for_each_set_bit because it's unsigned
+ * long aligned, but our registers are u32 sized.
+ */
+static __always_inline int next_bit(u32 var, int bit)
+{
+ int pos;
+
+ pos = ffs(var & ~(((u32)1 << bit) - 1));
+ return (0 == pos || 32 <= bit) ? 32 : (pos - 1);
+}
+
+/*
+ * Perform loop for each set bit of a u32 variable.
+ *
+ * NOTE Size of integer is supposed to be 32-bits or greater so this
+ * "for each"-macro would work.
+ */
+#define for_each_set_bit_u32(var, bit) \
+ for ((bit) = next_bit(var, 0); \
+ bit < 32; \
+ (bit) = next_bit(var, (bit) + 1))
+
+/*
+ * Create a contiguous bitmask starting at bit position @l and ending at
+ * position @h. For example
+ * GENMASK_ULL(39, 21) gives us the 64bit vector 0x000000ffffe00000.
+ */
+#ifndef GENMASK
+#define GENMASK(h, l) \
+ (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+#endif /* !GENMASK */
+
+/*
+ * Number of NTB resource like Doorbell bits, Memory windows
+ * and Message registers
+ */
+#define IDT_NTB_DBELL_CNT 32
+#define IDT_NTB_MW_CNT 24
+#define IDT_NTB_MSG_CNT 4
+#define IDT_NTB_MTBL_ENTRY_CNT 64
+
+/*
+ * General IDT PCIe-switch constant
+ * @IDT_NTB_MAXPORTS_CNT: Maximum number of ports per IDT PCIe-switch
+ * @IDT_NTB_MAXPARTS_CNT: Maximum number of partitions per IDT PCIe-switch
+ * @IDT_PCIE_REGSIZE: Size of the registers in bytes
+ * @IDT_NTB_TRANSALIGN: Alignment of the translated base address
+ * @IDT_NTB_LNKPOLL_TOUT: Timeout of the link polling kernel thread
+ * @IDT_NTB_SENDMSG_TOUT: Timeout of sending the next message to a peer
+ * @IDT_NTB_TEMP_LTH: Lower threshold of the IDT temperature sensor
+ * @IDT_NTB_TEMP_HTH: Higher threshold of the IDT temperature sensor
+ */
+#define IDT_NTB_MAXPORTS_CNT 24
+#define IDT_NTB_MAXPARTS_CNT 8
+#define IDT_PCIE_REGSIZE 4
+#define IDT_NTB_TRANSALIGN 4
+#define IDT_NTB_LNKPOLL_TOUT msecs_to_jiffies(1000)
+#define IDT_NTB_SENDMSG_TOUT msecs_to_jiffies(100)
+#define IDT_NTB_SENDMSG_RETRY 50
+#define IDT_NTB_TEMP_LTH (u32)10
+#define IDT_NTB_TEMP_HTH (u32)85
+
+/*
+ * u32 data atomic structure
+ */
+typedef struct {
+ spinlock_t lock;
+ u32 data;
+} atomic_u32_t;
+
+/*
+ * Queue head with atomic access
+ */
+typedef struct {
+ struct list_head head;
+ spinlock_t lock;
+} queue_atomic_t;
+
+/*
+ * Messages list container
+ * @msg: Message structure
+ * @retry: Number of retries left
+ * @entry: Queue entry
+ */
+struct idt_ntb_msg {
+ struct ntb_msg msg;
+ int retry;
+ struct list_head entry;
+};
+#define to_msg_list_entry(pentry) \
+ (list_entry(pentry, struct idt_ntb_msg, entry))
+
+/*
+ * IDT PCIe-switch model private data
+ * @port_cnt: Total number of NT endpoint ports
+ * @ports: Port ids
+ */
+struct idt_89hpes_pdata {
+ unsigned char port_cnt;
+ unsigned char ports[];
+};
+
+/*
+ * NTB-bus device structure
+ * @ntb: NTB-bus device related structure
+ *
+ * @port: Remote NT-function port
+ * @part: Remote NT-function partition
+ *
+ * @pairid: Global Identifier of Primary-Secondary ports pair
+ *
+ * @lnk_sts: Peer side link status
+ *
+ * @mw_self_cnt: Number of memory windows locally available
+ * @mw_self_offset: Offset of the first memory window in the Lookup table
+ * @mw_peer_cnt: Number of peer memory windows
+ *
+ * @db_cnt: Number of Doorbells for communications with the
+ * peer NT-function
+ * @db_self_offset: Bits offset of the self Doorbells
+ * @db_peer_offset: Bits offset of the peer Doorbells
+ * @db_valid_mask: Doorbell valid mask
+ * @db_self_mask: Mask of the shifted by self_offset doorbells
+ * @db_peer_mask: Mask of the shifted by peer_offset doorbells
+ *
+ * @qinmsg: Queue of inbound messages received from the peer
+ * @inmsg_work: Work thread rising event of new message arrival
+ * @qoutmsg: Queue of outbound messages posted to send to the peer
+ * @outmsg_work: Work thread sending messages
+ */
+struct idt_ntb_dev {
+ struct ntb_dev ntb;
+
+ unsigned char port;
+ unsigned char part;
+
+ unsigned char pairid;
+
+ u32 lnk_sts;
+
+ unsigned char mw_self_cnt;
+ unsigned char mw_self_offset;
+ unsigned char mw_peer_cnt;
+
+ unsigned char db_cnt;
+ unsigned char db_self_offset;
+ unsigned char db_peer_offset;
+ u32 db_valid_mask;
+ u32 db_self_mask;
+ u32 db_peer_mask;
+
+ queue_atomic_t qinmsg;
+ struct work_struct inmsg_work;
+ queue_atomic_t qoutmsg;
+ struct delayed_work outmsg_work;
+};
+#define to_ndev_ntb(pntb) container_of(pntb, struct idt_ntb_dev, ntb)
+#define to_pdev_ndev(ndev) ((ndev)->ntb.pdev)
+#define to_dev_ndev(ndev) (&ndev->ntb.dev)
+#define to_data_ndev(ndev) \
+ ((struct idt_ntb_data *)(pci_get_drvdata(to_pdev_ndev(ndev))))
+#define to_cfg_ndev(ndev) (to_data_ndev(ndev)->cfg_mmio)
+#define to_ndev_inmsg_work(work) \
+ container_of(work, struct idt_ntb_dev, inmsg_work)
+#define to_ndev_outmsg_work(work) \
+ container_of(to_delayed_work(work), struct idt_ntb_dev, outmsg_work)
+
+/*
+ * IDT PCIe-switch NTB bus topology structure
+ * @paircnt: Total number of the NTB pair in the current topology
+ * (it's just the number of Secondary ports)
+ *
+ * @priports: Bitset of Primary ports
+ * @secports: Array of Secondary ports bitsets related to the corresponding
+ * Primary ports
+ */
+struct idt_ntb_topo {
+ unsigned char paircnt;
+
+ u32 priports;
+ u32 secports[IDT_NTB_MAXPORTS_CNT];
+};
+
+/*
+ * Structure related to the local IDT PCIe-switch NT-function
+ * @pdev: Pointer to the PCI-bus device
+ * @swcfg: Pointer to the struct idt_89hpes_pdata related to the current
+ * IDT PCIe-switch
+ *
+ * @port: Local NT-function port
+ * @part: Local NT-function partition
+ *
+ * @topo: Topology of the NT-function ports
+ * @role: Local port role in the IDT topology
+ *
+ * @peer_cnt: Number of possible remote peers
+ * @ndevs: Array of the device-related structures
+ *
+ * @cfg_mmio: Virtual address of the memory mapped configuration space
+ * of the NT-function
+ *
+ * @idt_wq: IDT driver workqueue to setup the link poll and messages
+ * delivery operations
+ *
+ * @lnk_work: Link status polling kernel thread
+ *
+ * @mw_base: Physical address of the memory mapped base address of the
+ * Memory Windows
+ * @mw_size: Size of one Memory Window
+ * @lut_lock: Lookup table access spin lock
+ *
+ * @db_sts: Doorbell status atomic variable
+ * @db_msk: Doorbell mask atomic variable
+ * @db_lock: Doorbell status and mask spin lock
+ * @db_tasklet: Tasklet to handle the doorbell events
+ *
+ * @msg_lock: Messages routing table lock
+ * @msg_cache: Slab cache of the message structures
+ * @msg_tasklet: Tasklet - handler of the incoming messages
+ *
+ * @dbgfs_dir: DebugFS directory to place the driver debug file
+ */
+struct idt_ntb_data {
+ struct pci_dev *pdev;
+ struct idt_89hpes_pdata *swcfg;
+
+ unsigned char port;
+ unsigned char part;
+
+ struct idt_ntb_topo topo;
+ enum ntb_topo role;
+
+ unsigned char peer_cnt;
+ struct idt_ntb_dev *ndevs;
+
+ void __iomem *cfg_mmio;
+
+ struct workqueue_struct *idt_wq;
+
+ struct delayed_work lnk_work;
+
+ phys_addr_t mw_base;
+ resource_size_t mw_size;
+ spinlock_t lut_lock;
+
+ u32 db_sts;
+ u32 db_msk;
+ spinlock_t db_lock;
+ struct tasklet_struct db_tasklet;
+
+ spinlock_t msg_lock;
+ struct kmem_cache *msg_cache;
+ struct tasklet_struct msg_tasklet;
+
+ struct dentry *dbgfs_dir;
+};
+#define to_dev_data(data) (&(data->pdev->dev))
+#define to_data_lnkwork(work) \
+ container_of(to_delayed_work(work), struct idt_ntb_data, lnk_work)
+
+/*
+ * Descriptor of the IDT PCIe-switch port specific parameters in the
+ * Global Configuration Space
+ * @pcicmd: PCI command register
+ * @pcielsts: PCIe link status
+ * @ntsdata: NT signal data
+ * @ntgsignal: NT global signal
+ *
+ * @ctl: Port control register
+ * @sts: Port status register
+ */
+struct idt_ntb_port {
+ enum idt_ntb_cfgreg pcicmd;
+ enum idt_ntb_cfgreg pcielsts;
+ enum idt_ntb_cfgreg ntsdata;
+ enum idt_ntb_cfgreg ntgsignal;
+
+ enum idt_ntb_cfgreg ctl;
+ enum idt_ntb_cfgreg sts;
+};
+
+/*
+ * Descriptor of the IDT PCIe-switch partition specific parameters.
+ * @ctl: Partition control register in the Global Address Space
+ * @sts: Partition status register in the Global Address Space
+ */
+struct idt_ntb_part {
+ enum idt_ntb_cfgreg ctl;
+ enum idt_ntb_cfgreg sts;
+ enum idt_ntb_cfgreg msgctl[IDT_NTB_MSG_CNT];
+};
+
+#endif /* NTB_HW_IDT_H */
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt_quirks.c b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
new file mode 100644
index 0000000..57c7ccf
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.c
@@ -0,0 +1,163 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*#define DEBUG*/
+
+#include <linux/kernel.h>
+#include <linux/moduleparam.h>
+#include <linux/pci.h>
+
+#include "ntb_hw_idt.h"
+#include "ntb_hw_idt_quirks.h"
+
+/*
+ * Module parameters:
+ * @mw_aprt: Memory Windows aperture (x86: 9 - 26, x64: 9 - 32)
+ */
+static unsigned char mw_aprt = DEFAULT_MW_APRT;
+module_param(mw_aprt, byte, 0000);
+MODULE_PARM_DESC(mw_aprt,
+ "IDT NTB memory windows aperture. The actual memory windows size is "
+ "limited with 2^mw_aprt. It is initially set to 20 so the upper "
+ "boundary of the memory windows size would be 1 MB."
+ "Both sides, local node and peer MUST set the same value!");
+
+/*
+ * Alter the passed driver paremeters
+ */
+static void idt_ntb_alter_params(struct pci_dev *pdev)
+{
+ unsigned char mw_aprt_bak = mw_aprt;
+
+ /* Clamp the memory windows aperture parameter */
+#ifdef CONFIG_64BIT
+ mw_aprt = clamp(mw_aprt, MIN_MW_APRT, MAX_X64_MW_APRT);
+#else
+ mw_aprt = clamp(mw_aprt, MIN_MW_APRT, MAX_X86_MW_APRT);
+#endif /* !CONFIG_64BIT */
+ if (mw_aprt_bak != mw_aprt) {
+ dev_warn(&pdev->dev,
+ "IDT NTB memory windows aperture has been clamped "
+ "from %hhu to %hhu", mw_aprt_bak, mw_aprt);
+ }
+
+ dev_dbg(&pdev->dev, "IDT NTB HW-driver parameter has been verified");
+}
+
+/*
+ * IDT PCIe-swtich NTB function BARs pre-initializer
+ */
+static void idt_ntb_quirks(struct pci_dev *pdev)
+{
+ int ret;
+ u32 lubar_aprt = 0, dirbar_aprt = 0;
+
+ /* Alter the memory windows aperture parameter first */
+ idt_ntb_alter_params(pdev);
+
+ /* Calculate memory windows related BAR aperture */
+ lubar_aprt = (mw_aprt + MWLUTBL_APRT) << MWAPRT_OFF;
+ dirbar_aprt = mw_aprt << MWAPRT_OFF;
+
+ /* Pre-initialize the maximum possible BAR's so don't worry about them
+ * anymore */
+ /* BAR0 - Memory mapped Configuration space - x32 Non-prefetchable
+ * memory mapped space. Since it is the registers space then it must be
+ * non-prefetchable, which permits the 32-bits address only according
+ * to the PCI specs. Even though PCIe bridges doesn't do any prefetching
+ * whether prefetch bit is set or not, We'll set the bit as a matter of
+ * legacy */
+ ret = pci_write_config_dword(pdev, BARSETUP0_OFF, BARSETUP_CFG_32BIT);
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to activate registers configuration space (BAR0)");
+ return;
+ }
+
+ /* BAR2(+ x64:3) - Memory mapped shared memory with address translation
+ * based on lookup table - x32/x64 Non-prefetchable/prefetchable memory
+ * mapped space with aperture of 2^(mw_aprt + MWLUTBL_APRT), which
+ * effectively gives 2^mw_aprt bytes of memory space per each memory
+ * window */
+#ifdef CONFIG_64BIT
+ ret = pci_write_config_dword(pdev, BARSETUP2_OFF,
+ BARSETUP_24LUMW_64BIT | lubar_aprt);
+
+#else
+ ret = pci_write_config_dword(pdev, BARSETUP2_OFF,
+ BARSETUP_24LUMW_32BIT | lubar_aprt);
+
+#endif /* !CONFIG_64BIT */
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to activate lookup table based memory window (BAR2)");
+ return;
+ }
+
+ /* BAR4(+ x64:5) - Memory mapped shared memory with direct address
+ * translation - x32/x64 Non-prefetchable/prefetchable memory
+ * mapped space with aperture of 2^(mw_aprt + MWLUTBL_APRT) */
+#ifdef CONFIG_64BIT
+ ret = pci_write_config_dword(pdev, BARSETUP4_OFF,
+ BARSETUP_DIRMW_64BIT | dirbar_aprt);
+
+#else
+ ret = pci_write_config_dword(pdev, BARSETUP4_OFF,
+ BARSETUP_DIRMW_32BIT | dirbar_aprt);
+
+#endif /* !CONFIG_64BIT */
+ if (SUCCESS != ret) {
+ dev_err(&pdev->dev,
+ "Failed to activate directly mapped memory window (BAR4)");
+ return;
+ }
+
+ dev_dbg(&pdev->dev, "IDT NTB BAR's enabled");
+}
+IDT_NTB_PCI_FIXUP_EARLY(89HPES24NT6AG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT8AG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT8BG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES12NT12G2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES16NT16G2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES24NT24G2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT24AG2, idt_ntb_quirks);
+IDT_NTB_PCI_FIXUP_EARLY(89HPES32NT24BG2, idt_ntb_quirks);
+
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt_quirks.h b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
new file mode 100644
index 0000000..e633b7c
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt_quirks.h
@@ -0,0 +1,114 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+#ifndef NTB_HW_IDT_QUIRKS_H
+#define NTB_HW_IDT_QUIRKS_H
+
+#include <linux/pci.h>
+#include <linux/pci_ids.h>
+
+#include "ntb_hw_idt.h"
+
+/*
+ * Macro is used to create the struct pci_fixup that matches the supported
+ * IDT PCIe-switches
+ * @devname: Capitalized name of the particular device
+ * @hook: Fixup hook function name
+ */
+#define IDT_NTB_PCI_FIXUP_EARLY(devname, hook) \
+ DECLARE_PCI_FIXUP_CLASS_EARLY(PCI_VENDOR_ID_IDT, \
+ PCI_DEVICE_ID_IDT_##devname, PCI_CLASS_BRIDGE_OTHER, 8U, hook)
+
+/*
+ * IDT PCIe-switch NTB function BAR setup parameters:
+ * @BARSETUP{N}_OFF: BAR{N} setup register offset
+ * @BARSETUP_CFG_32BIT: 32-bits addressable non-prefetchable memory
+ * mapped registers configuration space
+ * @BARSETUP_CFG_64BIT: 64-bits addressable prefetchable memory
+ * mapped registers configuration space
+ * @BARSETUP_DIRMW_32BIT: 32-bits addresable non-prefetchable direct
+ * address translatable memory window
+ * @BARSETUP_DIRMW_64BIT: 64-bits addresable prefetchable direct
+ * address translatable memory window
+ * @BARSETUP_12LUMW_32BIT: 32-bits addresable non-prefetchable 12-entries
+ * lookup table memory window
+ * @BARSETUP_12LUMW_64BIT: 64-bits addresable prefetchable 12-entries
+ * lookup table memory window
+ * @BARSETUP_24LUMW_32BIT: 32-bits addresable non-prefetchable 24-entries
+ * lookup table memory window
+ * @BARSETUP_24LUMW_64BIT: 64-bits addresable prefetchable 24-entries
+ * lookup table memory window
+ *
+ */
+#define BARSETUP0_OFF 0x00470
+#define BARSETUP1_OFF 0x00480
+#define BARSETUP2_OFF 0x00490
+#define BARSETUP3_OFF 0x004A0
+#define BARSETUP4_OFF 0x004B0
+#define BARSETUP5_OFF 0x004C0
+#define BARSETUP_CFG_32BIT ((u32)0x800004C0U)
+#define BARSETUP_CFG_64BIT ((u32)0x800004CCU)
+#define BARSETUP_DIRMW_32BIT ((u32)0x80000000U)
+#define BARSETUP_DIRMW_64BIT ((u32)0x8000000CU)
+#define BARSETUP_12LUMW_32BIT ((u32)0x80000800U)
+#define BARSETUP_12LUMW_64BIT ((u32)0x8000080CU)
+#define BARSETUP_24LUMW_32BIT ((u32)0x80001000U)
+#define BARSETUP_24LUMW_64BIT ((u32)0x8000100CU)
+#define MWAPRT_OFF 4
+
+/*
+ * IDT PCIe-switch NTB function related parameters:
+ * @DEFAULT_MW_APRT: Default aperture of the memory windows (that is
+ * maximum size of the memory windows)
+ * @MIN_MW_APRT: Minimum possible aperture of the memory windows
+ * @MAX_X86_MW_APRT: Maximum aperture for x86 architecture
+ * @MAX_X64_MW_APRT: Maximum aperture for x64 architecture
+ * @MWLUTBL_APRT: Additional value to translate the per memory
+ * windows specific aperture to the aperture of
+ * the whole lookup table
+ */
+#define DEFAULT_MW_APRT (unsigned char)20
+#define MIN_MW_APRT (unsigned char)9
+#define MAX_X86_MW_APRT (unsigned char)26
+#define MAX_X64_MW_APRT (unsigned char)32
+#define MWLUTBL_APRT (unsigned char)5
+
+#endif /* NTB_HW_IDT_QUIRKS_H */
diff --git a/drivers/ntb/hw/idt/ntb_hw_idt_regmap.h b/drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
new file mode 100644
index 0000000..fa9aaf2
--- /dev/null
+++ b/drivers/ntb/hw/idt/ntb_hw_idt_regmap.h
@@ -0,0 +1,877 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * IDT PCIe-switch NTB Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+#ifndef NTB_HW_IDT_REGMAP_H
+#define NTB_HW_IDT_REGMAP_H
+
+#include <linux/compiler.h>
+#include <linux/spinlock.h>
+
+/*
+ * Helper macros to enumerate the the registers and fields tables
+ * identifications.
+ * It's used in conjunction with the IDT_NT_REGFLDS(), IDT_SW_REGFLDS(),
+ * IDT_NT_CFGREGS() and IDT_SW_CFGREGS() macroses
+ */
+#define PAIR_ID_ENUM(ID, reg, mask, offset) ID,
+
+/*
+ * Helper macros to pair the Field identification with the corresponding
+ * register, mask and the offset
+ * It's used in conjunction with the IDT_NT_REGFLDS() and
+ * IDT_SW_REGFLDS() macroses
+ */
+#define PAIR_FLDID_ACCESS(ID, reg, mask, offset, retreg, retmask, retoffset) \
+ case ID: \
+ retreg = reg; \
+ retmask = mask; \
+ retoffset = offset; \
+ break;
+
+/*
+ * Helper macros to pair the registers identifications with the corresponding
+ * offset and the size
+ * It's used in conjunction with the IDT_NT_CFGREGS() and
+ * IDT_SW_CFGREGS() macroses
+ */
+#define PAIR_REGID_ACCESS(ID, addr, size, desc, retaddr, retsize, retdesc) \
+ case ID: \
+ retaddr = addr; \
+ retsize = size; \
+ retdesc = desc; \
+ break;
+
+/*
+ * IDT PCIe-swtich registers related constants:
+ *
+ * @BARSTP_MEMMAP: Memory mapped BAR select
+ * @BARSTP_TYPE_32: 32-bits addressable BAR select
+ * @BARSTP_TYPE_64: 64-bits addressable BAR select
+ * @BARSTP_NONPREF: Non-prefetchable memory
+ * @BARSTP_PREF: Prefetchable memory
+ * @BARSTP_MINSIZE: Minimum BAR aperture (2^SIZE) for Lookup table
+ * @BARSTP_MAXSIZE_32: Maximum BAR aperture for Lookup table and x86 CPU
+ * @BARSTP_MAXSIZE_64: Maximum BAR aperture for Lookup table and x64 CPU
+ * @BARSTP_MODE_WNDW: Memory Window mode of BAR
+ * @BARSTP_MODE_CFGSPC: Configuration space mode of BAR
+ * @BARSTP_ATRAN_DRCT: Direct address translation
+ * @BARSTP_ATRAN_LU12: 12-entries Lookup table for address translation
+ * @BARSTP_ATRAN_LU24: 24-entries Lookup table for address translation
+ *
+ * @GASAADDR_OFFSET: GASAADDR register offset in the NT-function config space
+ * @GASADATA_OFFSET: GASADATA register offset in the NT-function config space
+ *
+ * @PORTMODE_NT: Port mode - just one NT function
+ * @PORTMODE_USNT: Port mode - upstream switch port with NT function
+ * @PORTMODE_USNTDMA: Port mode - upstream switch port with NT and DMA
+ * functions
+ *
+ * @NTINT_MASK: Mask the NT interrupts (Msg, DB, SE, Temp)
+ * @NTINT_UNMASK: Unmask the NT interrupts (Msg, DB, SE, Temp)
+ * @ALLINT_MASK: Mask all the available interrupts
+ * @ALLINT_UNMASK: Unmask all the available unterrupts
+ * @MSGINT_BIT: Message interrupt serial bit
+ * @DBINT_BIT: Doorbell interrupt serial bit
+ * @SEINT_BIT: Switch events interrupt serial bit
+ * @FMCI_BIT: Failover mode change initiated
+ * @FMCC_BIT: Failover mode change completed
+ * @TEMPINT_BIT: Temperature sensor interrupt serial bit
+ *
+ * @INDB_MASK: Mask the inbound doorbells interrupt
+ * @INDB_UNMASK: Unmask the inbound doorbells interrupt
+ * @OUTMSG_MASK: Mask all the outbound message bits
+ * @INMSG_MASK: Mask all the inbound message bits
+ * @INMSG_STS: Valid inbound message status
+ * @MSG_MASK: Mask all the message interrupts
+ * @MSG_UNMASK: Unmask the first inbound message register interrupt
+ */
+#define BARSTP_MEMMAP ((u32)0x0)
+#define BARSTP_TYPE_32 ((u32)0x0)
+#define BARSTP_TYPE_64 ((u32)0x2)
+#define BARSTP_NONPREF ((u32)0x0)
+#define BARSTP_PREF ((u32)0x1)
+#define BARSTP_MINSIZE ((u32)14)
+#define BARSTP_MAXSIZE_32 ((u32)16)
+#define BARSTP_MAXSIZE_64 ((u32)37)
+#define BARSTP_MODE_WNDW ((u32)0x0)
+#define BARSTP_MODE_CFGSPC ((u32)0x1)
+#define BARSTP_ATRAN_DRCT ((u32)0x0)
+#define BARSTP_ATRAN_LU12 ((u32)0x1)
+#define BARSTP_ATRAN_LU24 ((u32)0x2)
+#define GASAADDR_OFFSET ((ptrdiff_t)0x00FF8)
+#define GASADATA_OFFSET ((ptrdiff_t)0x00FFC)
+#define PORTMODE_NT ((u32)0x3)
+#define PORTMODE_USNT ((u32)0x4)
+#define PORTMODE_USNTDMA ((u32)0x7)
+#define NTINT_MASK ((u32)0x8B)
+#define NTINT_UNMASK (~NTINT_MASK)
+#define ALLINT_MASK ((u32)0xBB)
+#define ALLINT_UNMASK (~ALLINT_MASK)
+#define MSGINT_BIT ((u32)0)
+#define DBINT_BIT ((u32)1)
+#define SEINT_BIT ((u32)3)
+#define FMCI_BIT ((u32)4)
+#define FMCC_BIT ((u32)5)
+#define TEMPINT_BIT ((u32)7)
+#define INDB_MASK ((u32)-1)
+#define INDB_UNMASK ((u32)0x0)
+#define OUTMSG_MASK ((u32)0xF)
+#define INMSG_MASK ((u32)0xF0000)
+#define INMSG_STS ((u32)0xF)
+#define MSG_MASK ((u32)0xF000F)
+#define MSG_UNMASK ((u32)0xE000F)
+
+/*
+ * Table of the register fields accessed over either the NT-function
+ * memory mapped registers or IDT PCIe-switch Global registers.
+ * This table is then translated into the switch-case statement
+ * so to get the proper "Name"->{reg addr, mask, fld offset}
+ * pairs.
+ */
+#define IDT_NT_REGFLDS(X, args...) \
+ /* PCI command register */ \
+ X(IDT_NT_IOAE, IDT_NT_PCI_CMD, 0x1, 0, ## args) \
+ X(IDT_NT_MAE, IDT_NT_PCI_CMD, 0x1, 1, ## args) \
+ X(IDT_NT_BME, IDT_NT_PCI_CMD, 0x1, 2, ## args) \
+ /* Link status registers */ \
+ X(IDT_NT_MAXLNKSPD, IDT_NT_PCI_PCIELCAP, 0xF, 0, ## args) \
+ X(IDT_NT_MAXLNKWDTH, IDT_NT_PCI_PCIELCAP, 0x3F, 4, ## args) \
+ X(IDT_NT_PORTNUM, IDT_NT_PCI_PCIELCAP, 0xFF, 24, ## args) \
+ X(IDT_NT_CURLNKSPD, IDT_NT_PCI_PCIELSTS, 0xF, 0, ## args) \
+ X(IDT_NT_CURLNKWDTH, IDT_NT_PCI_PCIELSTS, 0x3F, 4, ## args) \
+ X(IDT_NT_SCLK, IDT_NT_PCI_PCIELSTS, 0x1, 12, ## args) \
+ /* SSVID/SSID registers */\
+ X(IDT_NT_SSVID, IDT_NT_PCI_SSIDSSVID, 0xFFFF, 0, ## args) \
+ X(IDT_NT_SSID, IDT_NT_PCI_SSIDSSVID, 0xFFFF, 16, ## args) \
+ /* General NT-function registers */\
+ X(IDT_NT_IDPROTDIS, IDT_NT_PCI_NTCTL, 0x1, 0, ## args) \
+ X(IDT_NT_CPEN, IDT_NT_PCI_NTCTL, 0x1, 1, ## args) \
+ /* NT interrupts related registers */ \
+ X(IDT_NT_INTSTS, IDT_NT_PCI_NTINTSTS, 0xBB, 0, ## args) \
+ X(IDT_NT_MSGINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 0, ## args) \
+ X(IDT_NT_DBINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 1, ## args) \
+ X(IDT_NT_SEINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 3, ## args) \
+ X(IDT_NT_FMCIINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 4, ## args) \
+ X(IDT_NT_FMCCINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 5, ## args) \
+ X(IDT_NT_TMPINT_STS, IDT_NT_PCI_NTINTSTS, 0x1, 7, ## args) \
+ X(IDT_NT_INTMSK, IDT_NT_PCI_NTINTMSK, 0xBB, 0, ## args) \
+ X(IDT_NT_MSGINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 0, ## args) \
+ X(IDT_NT_DBINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 1, ## args) \
+ X(IDT_NT_SEINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 3, ## args) \
+ X(IDT_NT_FMCIINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 4, ## args) \
+ X(IDT_NT_FMCCINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 5, ## args) \
+ X(IDT_NT_TMPINT_MSK, IDT_NT_PCI_NTINTMSK, 0x1, 7, ## args) \
+ X(IDT_NT_GSIGNAL, IDT_NT_PCI_NTGSIGNAL, 0x1, 0, ## args) \
+ /* Message registers status and masks */ \
+ X(IDT_NT_OUTMSGSTS, IDT_NT_PCI_MSGSTS, 0xF, 0, ## args) \
+ X(IDT_NT_INMSGSTS, IDT_NT_PCI_MSGSTS, 0xF, 16, ## args) \
+ X(IDT_NT_OUTMSG0STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 0, ## args) \
+ X(IDT_NT_OUTMSG1STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 1, ## args) \
+ X(IDT_NT_OUTMSG2STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 2, ## args) \
+ X(IDT_NT_OUTMSG3STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 3, ## args) \
+ X(IDT_NT_INMSG0STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 16, ## args) \
+ X(IDT_NT_INMSG1STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 17, ## args) \
+ X(IDT_NT_INMSG2STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 18, ## args) \
+ X(IDT_NT_INMSG3STSMSK, IDT_NT_PCI_MSGSTSMSK, 0x1, 19, ## args) \
+ /* BARSETUPx register (default BARSETUP0) */ \
+ X(IDT_NT_BARSTP_MEMSI, IDT_NT_PCI_BARSETUP0, 0x1, 0, ## args) \
+ X(IDT_NT_BARSTP_TYPE, IDT_NT_PCI_BARSETUP0, 0x3, 1, ## args) \
+ X(IDT_NT_BARSTP_PREF, IDT_NT_PCI_BARSETUP0, 0x1, 3, ## args) \
+ X(IDT_NT_BARSTP_SIZE, IDT_NT_PCI_BARSETUP0, 0x3F, 4, ## args) \
+ X(IDT_NT_BARSTP_MODE, IDT_NT_PCI_BARSETUP0, 0x1, 10, ## args) \
+ X(IDT_NT_BARSTP_ATRAN, IDT_NT_PCI_BARSETUP0, 0x3, 11, ## args) \
+ X(IDT_NT_BARSTP_TPART, IDT_NT_PCI_BARSETUP0, 0x7, 13, ## args) \
+ X(IDT_NT_BARSTP_EN, IDT_NT_PCI_BARSETUP0, 0x1, 31, ## args) \
+ /* NT mapping table registers */ \
+ X(IDT_NT_MTBL_ADDR, IDT_NT_PCI_NTMTBLADDR, 0x7F, 0, ## args) \
+ X(IDT_NT_MTBL_ERR, IDT_NT_PCI_NTMTBLSTS, 0x1, 0, ## args) \
+ X(IDT_NT_MTBL_VALID, IDT_NT_PCI_NTMTBLDATA, 0x1, 0, ## args) \
+ X(IDT_NT_MTBL_BDF, IDT_NT_PCI_NTMTBLDATA, 0xFFFF, 1, ## args) \
+ X(IDT_NT_MTBL_PART, IDT_NT_PCI_NTMTBLDATA, 0x7, 17, ## args) \
+ X(IDT_NT_MTBL_ATP, IDT_NT_PCI_NTMTBLDATA, 0x1, 29, ## args) \
+ X(IDT_NT_MTBL_CNS, IDT_NT_PCI_NTMTBLDATA, 0x1, 30, ## args) \
+ X(IDT_NT_MTBL_RNS, IDT_NT_PCI_NTMTBLDATA, 0x1, 31, ## args) \
+ X(IDT_NT_MTBL_REQID, IDT_NT_PCI_REQIDCAP, 0xFFFF, 0, ## args) \
+ /* Lookup table registers */ \
+ X(IDT_NT_LUT_INDEX, IDT_NT_PCI_LUTOFFSET, 0x1F, 0, ## args) \
+ X(IDT_NT_LUT_BAR, IDT_NT_PCI_LUTOFFSET, 0x7, 8, ## args) \
+ X(IDT_NT_LUT_PART, IDT_NT_PCI_LUTUDATA, 0xF, 0, ## args) \
+ X(IDT_NT_LUT_VALID, IDT_NT_PCI_LUTUDATA, 0x1, 31, ## args)
+
+/*
+ * Table of the fields accessed over the global switch registers
+ * This table is then translated into the switch-case statement
+ * so to get the proper "Name"->{reg addr, mask, fld offset}
+ * pairs.
+ */
+#define IDT_SW_REGFLDS(X, args...) \
+ /* Boot configuration vector status */ \
+ X(IDT_SW_SWMODE, IDT_SW_PCI_BCVSTS, 0xF, 0, ## args) \
+ X(IDT_SW_GCLKFSEL, IDT_SW_PCI_BCVSTS, 0x1, 5, ## args) \
+ X(IDT_SW_SSMB_ADDRSET, IDT_SW_PCI_BCVSTS, 0x3, 7, ## args) \
+ X(IDT_SW_CLKMODE, IDT_SW_PCI_BCVSTS, 0x3, 14, ## args) \
+ /* Ports clocking mode */ \
+ X(IDT_SW_P0CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 0, ## args) \
+ X(IDT_SW_P2CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 2, ## args) \
+ X(IDT_SW_P4CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 4, ## args) \
+ X(IDT_SW_P6CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 6, ## args) \
+ X(IDT_SW_P8CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 8, ## args) \
+ X(IDT_SW_P12CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 10, ## args) \
+ X(IDT_SW_P16CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 12, ## args) \
+ X(IDT_SW_P20CLKMODE, IDT_SW_PCI_PCLKMODE, 0x2, 14, ## args) \
+ /* Switch Ports Status register (default, Port 0) */ \
+ X(IDT_SW_PORT_LNKUP, IDT_SW_PCI_SWPORT0STS, 0x1, 4, ## args) \
+ X(IDT_SW_PORT_LNKMODE, IDT_SW_PCI_SWPORT0STS, 0x1, 5, ## args) \
+ X(IDT_SW_PORT_MODE, IDT_SW_PCI_SWPORT0STS, 0xF, 6, ## args) \
+ X(IDT_SW_PORT_SWPART, IDT_SW_PCI_SWPORT0STS, 0x7, 10, ## args) \
+ /* Switch Event registers */ \
+ X(IDT_SW_LNKUP_GSTS, IDT_SW_PCI_SESTS, 0x1, 0, ## args) \
+ X(IDT_SW_LNKDN_GSTS, IDT_SW_PCI_SESTS, 0x1, 1, ## args) \
+ X(IDT_SW_FRST_GSTS, IDT_SW_PCI_SESTS, 0x1, 2, ## args) \
+ X(IDT_SW_HRST_GSTS, IDT_SW_PCI_SESTS, 0x1, 3, ## args) \
+ X(IDT_SW_FOVER_GSTS, IDT_SW_PCI_SESTS, 0x1, 4, ## args) \
+ X(IDT_SW_GSIG_GSTS, IDT_SW_PCI_SESTS, 0x1, 5, ## args) \
+ X(IDT_SW_LNKUP_GMSK, IDT_SW_PCI_SEMSK, 0x1, 0, ## args) \
+ X(IDT_SW_LNKDN_GMSK, IDT_SW_PCI_SEMSK, 0x1, 1, ## args) \
+ X(IDT_SW_FRST_GMSK, IDT_SW_PCI_SEMSK, 0x1, 2, ## args) \
+ X(IDT_SW_HRST_GMSK, IDT_SW_PCI_SEMSK, 0x1, 3, ## args) \
+ X(IDT_SW_FOVER_GMSK, IDT_SW_PCI_SEMSK, 0x1, 4, ## args) \
+ X(IDT_SW_GSIG_GMSK, IDT_SW_PCI_SEMSK, 0x1, 5, ## args) \
+ X(IDT_SW_SEPART_GMSK, IDT_SW_PCI_SEPMSK, 0xFF, 0, ## args) \
+ X(IDT_SW_PORTLNKUP_STS, IDT_SW_PCI_SELINKUPSTS, 0xFFF, 0, ## args) \
+ X(IDT_SW_PORTLNKUP_MSK, IDT_SW_PCI_SELINKUPMSK, 0xFFF, 0, ## args) \
+ X(IDT_SW_PORTLNKDN_STS, IDT_SW_PCI_SELINKDNSTS, 0xFFF, 0, ## args) \
+ X(IDT_SW_PORTLNKDN_MSK, IDT_SW_PCI_SELINKDNMSK, 0xFFF, 0, ## args) \
+ X(IDT_SW_PARTFRST_STS, IDT_SW_PCI_SEFRSTSTS, 0xF, 0, ## args) \
+ X(IDT_SW_PARTFRST_MSK, IDT_SW_PCI_SEFRSTMSK, 0xF, 0, ## args) \
+ X(IDT_SW_PARTHRST_STS, IDT_SW_PCI_SEHRSTSTS, 0xF, 0, ## args) \
+ X(IDT_SW_PARTHRST_MSK, IDT_SW_PCI_SEHRSTMSK, 0xF, 0, ## args) \
+ X(IDT_SW_PARTGSIG_STS, IDT_SW_PCI_SEGSIGSTS, 0xF, 0, ## args) \
+ X(IDT_SW_PARTGSIG_MSK, IDT_SW_PCI_SEGSIGMSK, 0xF, 0, ## args) \
+ /* Global DoorBell registers (default, Doorbell 0) */ \
+ X(IDT_SW_PART_GODBELLMSK, IDT_SW_PCI_GODBELLMSK0, 0xF, 0, ## args) \
+ X(IDT_SW_PART_GIDBELLMSK, IDT_SW_PCI_GIDBELLMSK0, 0xF, 0, ## args) \
+ /* Message register (default, Partition 0 Message Control 0) */ \
+ X(IDT_SW_MSGROUTE_REG, IDT_SW_PCI_SWP0MSGCTL0, 0x3, 0, ## args) \
+ X(IDT_SW_MSGROUTE_PART, IDT_SW_PCI_SWP0MSGCTL0, 0x7, 4, ## args) \
+ /* SMBUS status */ \
+ X(IDT_SW_SSMBADDR, IDT_SW_PCI_SMBUSSTS, 0x7F, 1, ## args) \
+ X(IDT_SW_MSMBADDR, IDT_SW_PCI_SMBUSSTS, 0x7F, 9, ## args) \
+ /* Temperature sensor register */ \
+ X(IDT_SW_TMP_LTH, IDT_SW_PCI_TMPCTL, 0xFF, 0, ## args) \
+ X(IDT_SW_TMP_HTH, IDT_SW_PCI_TMPCTL, 0xFF, 16, ## args) \
+ X(IDT_SW_TMP_BLTH_EN, IDT_SW_PCI_TMPCTL, 0x1, 26, ## args) \
+ X(IDT_SW_TMP_AHTH_EN, IDT_SW_PCI_TMPCTL, 0x1, 29, ## args) \
+ X(IDT_SW_TMP_PDOWN, IDT_SW_PCI_TMPCTL, 0x1, 31, ## args) \
+ X(IDT_SW_TMP_CURTEMP, IDT_SW_PCI_TMPSTS, 0xFF, 0, ## args) \
+ X(IDT_SW_TMP_BLTH_STS, IDT_SW_PCI_TMPSTS, 0x1, 24, ## args) \
+ X(IDT_SW_TMP_AHTH_STS, IDT_SW_PCI_TMPSTS, 0x1, 29, ## args) \
+ X(IDT_SW_TMP_BLTH_CLR, IDT_SW_PCI_TMPALARM, 0x1, 24, ## args) \
+ X(IDT_SW_TMP_AHTH_CLR, IDT_SW_PCI_TMPALARM, 0x1, 29, ## args)
+
+/*
+ * Enumeration of the IDT PCIe-switch registers access fields
+ */
+enum idt_ntb_regfld {
+ IDT_NT_REGFLDS(PAIR_ID_ENUM)
+ IDT_NTB_REGFLDS_SPLIT,
+ IDT_SW_REGFLDS(PAIR_ID_ENUM)
+ IDT_NTB_REGFLDS_END
+};
+
+/*
+ * Enumeration of the possible registers size
+ */
+enum idt_ntb_regsize {
+ REGBYTE = 1,
+ REGWORD = 2,
+ REGDWORD = 4
+};
+
+/*
+ * Enumeration of the NT-function Configuration Space registers
+ * NOTE 1) The IDT PCIe-switch internal data is littel-endian
+ * so it must be taken into account in the driver
+ * internals.
+ * 2) Additionally the registers should be accessed either
+ * with byte-enables corresponding to their native size or
+ * the size of one DWORD
+ */
+#define IDT_NT_CFGREGS(X, args...) \
+ /* PCI Express Configuration Space */ \
+ /* Type 0 configuration header */ \
+ X(IDT_NT_PCI_VID, 0x00000, REGWORD, "Vendor Identification", ## args) \
+ X(IDT_NT_PCI_DID, 0x00002, REGWORD, "Device Identification", ## args) \
+ X(IDT_NT_PCI_CMD, 0x00004, REGWORD, "PCI Command", ## args) \
+ X(IDT_NT_PCI_STS, 0x00006, REGWORD, "PCI Status", ## args) \
+ X(IDT_NT_PCI_RID, 0x00008, REGBYTE, "Revision Identification", ## args) \
+ X(IDT_NT_PCI_PROGIF, 0x00009, REGBYTE, "Program Interface", ## args) \
+ X(IDT_NT_PCI_SCCLASS, 0x0000A, REGBYTE, "Sub Class Code", ## args) \
+ X(IDT_NT_PCI_CLASS, 0x0000B, REGBYTE, "Class Code", ## args) \
+ X(IDT_NT_PCI_CLS, 0x0000C, REGBYTE, "Cache Line Size", ## args) \
+ X(IDT_NT_PCI_LTIMER, 0x0000D, REGBYTE, "Latency Time", ## args) \
+ X(IDT_NT_PCI_HDR, 0x0000E, REGBYTE, "Header Type", ## args) \
+ X(IDT_NT_PCI_BIST, 0x0000F, REGBYTE, "Built-in Self Test Register", ## args) \
+ X(IDT_NT_PCI_BAR0, 0x00010, REGDWORD, "Base Address Register 0", ## args) \
+ X(IDT_NT_PCI_BAR1, 0x00014, REGDWORD, "Base Address Register 1", ## args) \
+ X(IDT_NT_PCI_BAR2, 0x00018, REGDWORD, "Base Address Register 2", ## args) \
+ X(IDT_NT_PCI_BAR3, 0x0001C, REGDWORD, "Base Address Register 3", ## args) \
+ X(IDT_NT_PCI_BAR4, 0x00020, REGDWORD, "Base Address Register 4", ## args) \
+ X(IDT_NT_PCI_BAR5, 0x00024, REGDWORD, "Base Address Register 5", ## args) \
+ X(IDT_NT_PCI_CCISPTR, 0x00028, REGDWORD, "CardBus CIS Pointer", ## args) \
+ X(IDT_NT_PCI_SUBVID, 0x0002C, REGWORD, "Subsystem Vendor ID Pointer", ## args) \
+ X(IDT_NT_PCI_SUBID, 0x0002E, REGWORD, "Subsystem ID Pointer", ## args) \
+ X(IDT_NT_PCI_EROMBASE, 0x00030, REGWORD, "Expansion ROM Base", ## args) \
+ X(IDT_NT_PCI_CAPPTR, 0x00034, REGBYTE, "Capabilities Pointer", ## args) \
+ X(IDT_NT_PCI_INTRLINE, 0x0003C, REGBYTE, "Interrupt Line", ## args) \
+ X(IDT_NT_PCI_INTRPIN, 0x0003D, REGBYTE, "Interrupt PIN", ## args) \
+ X(IDT_NT_PCI_MINGNT, 0x0003E, REGBYTE, "Minimum Grant", ## args) \
+ X(IDT_NT_PCI_MAXLAT, 0x0003F, REGBYTE, "Maximum Latency", ## args) \
+ /* PCI Express capablity structure */ \
+ X(IDT_NT_PCI_PCIECAP, 0x00040, REGDWORD, "PCI Express Capability", ## args) \
+ X(IDT_NT_PCI_PCIEDCAP, 0x00044, REGDWORD, "PCI Express Device Capabilities", ## args) \
+ X(IDT_NT_PCI_PCIEDCTL, 0x00048, REGWORD, "PCI Express Device Control", ## args) \
+ X(IDT_NT_PCI_PCIEDSTS, 0x0004A, REGWORD, "PCI Express Device Status", ## args) \
+ X(IDT_NT_PCI_PCIELCAP, 0x0004C, REGDWORD, "PCI Express Link Capabilities", ## args) \
+ X(IDT_NT_PCI_PCIELCTL, 0x00050, REGWORD, "PCI Express Link Control", ## args) \
+ X(IDT_NT_PCI_PCIELSTS, 0x00052, REGWORD, "PCI Express Link Status", ## args) \
+ X(IDT_NT_PCI_PCIEDCAP2, 0x00064, REGDWORD, "PCI Express Device Capabilities 2", ## args) \
+ X(IDT_NT_PCI_PCIEDCTL2, 0x00068, REGWORD, "PCI Express Device Control 2", ## args) \
+ X(IDT_NT_PCI_PCIEDSTS2, 0x0006A, REGWORD, "PCI Express Device Status 2", ## args) \
+ X(IDT_NT_PCI_PCIELCAP2, 0x0006C, REGDWORD, "PCI Express Link Capabilities 2", ## args) \
+ X(IDT_NT_PCI_PCIELCTL2, 0x00070, REGWORD, "PCI Express Link Control 2", ## args) \
+ X(IDT_NT_PCI_PCIELSTS2, 0x00072, REGWORD, "PCI Express Link Status 2", ## args) \
+ /* PCI Power Management capability structure */ \
+ X(IDT_NT_PCI_PMCAP, 0x000C0, REGDWORD, "PCI Power Management Capabilities", ## args) \
+ X(IDT_NT_PCI_PMCSR, 0x000C4, REGDWORD, "PCI Power Management Control and Status", ## args) \
+ /* MSI Capability structure */ \
+ X(IDT_NT_PCI_MSICAP, 0x000D0, REGDWORD, "Message Signaled Interrupt Capability and Control", ## args) \
+ X(IDT_NT_PCI_MSIADDR, 0x000D4, REGDWORD, "Message Signaled Interrupt Address", ## args) \
+ X(IDT_NT_PCI_MSIUADDR, 0x000D8, REGDWORD, "Message Signaled Interrupt Upper Address", ## args) \
+ X(IDT_NT_PCI_MSIMDATA, 0x000DC, REGDWORD, "Message Signaled Interrupt Message Data", ## args) \
+ /* SSID/SSVID capability structure */ \
+ X(IDT_NT_PCI_SSIDSSVIDCAP, 0x000F0, REGDWORD, "Subsystem ID and Subsystem Vendor ID Capability", ## args) \
+ X(IDT_NT_PCI_SSIDSSVID, 0x000F4, REGDWORD, "Subsystem ID and Subsystem Vendor ID", ## args) \
+ /* Extended access registers */ \
+ X(IDT_NT_PCI_ECFGADDR, 0x000F8, REGDWORD, "Extended Configuration Space Access Address", ## args) \
+ X(IDT_NT_PCI_ECFGDATA, 0x000FC, REGDWORD, "Extended Configuration Space Access Data", ## args) \
+ /*==============64 REGDWORDs ================*/ \
+ /* PCI Express Extended Configuration Space */ \
+ /* Advanced Error Reporting enhanced capability */ \
+ X(IDT_NT_PCI_AERCAP, 0x00100, REGDWORD, "AER Capabilities ", ## args) \
+ X(IDT_NT_PCI_AERUES, 0x00104, REGDWORD, "AER Uncorrectable Error Status", ## args) \
+ X(IDT_NT_PCI_AERUEM, 0x00108, REGDWORD, "AER Uncorrectable Error Mask ", ## args) \
+ X(IDT_NT_PCI_AERUESV, 0x0010C, REGDWORD, "AER Uncorrectable Error Severity ", ## args) \
+ X(IDT_NT_PCI_AERCES, 0x00110, REGDWORD, "AER Correctable Error Status ", ## args) \
+ X(IDT_NT_PCI_AERCEM, 0x00114, REGDWORD, "AER Correctable Error Mask", ## args) \
+ X(IDT_NT_PCI_AERCTL, 0x00118, REGDWORD, "AER Control", ## args) \
+ X(IDT_NT_PCI_AERHL1DW, 0x0011C, REGDWORD, "AER Header Log 1st Doubleword", ## args) \
+ X(IDT_NT_PCI_AERHL2DW, 0x00120, REGDWORD, "AER Header Log 2nd Doubleword", ## args) \
+ X(IDT_NT_PCI_AERHL3DW, 0x00124, REGDWORD, "AER Header Log 3rd Doubleword", ## args) \
+ X(IDT_NT_PCI_AERHL4DW, 0x00128, REGDWORD, "AER Header Log 4th Doubleword", ## args) \
+ /* Device Serial Number enhanced capability */ \
+ X(IDT_NT_PCI_SNUMCAP, 0x00180, REGDWORD, "Serial Number Capabilities", ## args) \
+ X(IDT_NT_PCI_SNUMLDW, 0x00184, REGDWORD, "Serial Number Lower Doubleword", ## args) \
+ X(IDT_NT_PCI_SNUMUDW, 0x00188, REGDWORD, "Serial Number Upper Doubleword", ## args) \
+ /* PCIe Virtual Channel enhanced capability */ \
+ X(IDT_NT_PCI_PCIEVCECAP, 0x00200, REGDWORD, "PCI Express VC Extended Capability Header", ## args) \
+ X(IDT_NT_PCI_PVCCAP1, 0x00204, REGDWORD, "Port VC Capability 1", ## args) \
+ X(IDT_NT_PCI_PVCCAP2, 0x00208, REGDWORD, "Port VC Capability 2", ## args) \
+ X(IDT_NT_PCI_PVCCTL, 0x0020C, REGDWORD, "Port VC Control", ## args) \
+ X(IDT_NT_PCI_PVCSTS, 0x0020E, REGDWORD, "Port VC Status ", ## args) \
+ X(IDT_NT_PCI_VCR0CAP, 0x00210, REGDWORD, "VC Resource 0 Capability", ## args) \
+ X(IDT_NT_PCI_VCR0CTL, 0x00214, REGDWORD, "VC Resource 0 Control", ## args) \
+ X(IDT_NT_PCI_VCR0STS, 0x00218, REGDWORD, "VC Resource 0 Status", ## args) \
+ /* ACS enhanced capability */ \
+ X(IDT_NT_PCI_ACSECAPH, 0x00320, REGDWORD, "ACS Extended Capability Header", ## args) \
+ X(IDT_NT_PCI_ACSCAP, 0x00324, REGWORD, "ACS Capability", ## args) \
+ X(IDT_NT_PCI_ACSCTL, 0x00326, REGWORD, "ACS Control", ## args) \
+ X(IDT_NT_PCI_MCCAPH, 0x00330, REGDWORD, "Multicast Extended Capability Header", ## args) \
+ X(IDT_NT_PCI_MCCAP, 0x00334, REGWORD, "Multicast Capability", ## args) \
+ X(IDT_NT_PCI_MCCTL, 0x00336, REGWORD, "Multicast Control", ## args) \
+ X(IDT_NT_PCI_MCBARL, 0x00338, REGDWORD, "Multicast Base Address Low", ## args) \
+ X(IDT_NT_PCI_MCBARH, 0x0033C, REGDWORD, "Multicast Base Address High", ## args) \
+ X(IDT_NT_PCI_MCRCVL, 0x00340, REGDWORD, "Multicast Receive Low", ## args) \
+ X(IDT_NT_PCI_MCRCVH, 0x00344, REGDWORD, "Multicast Receive High", ## args) \
+ X(IDT_NT_PCI_MCBLKALLL, 0x00348, REGDWORD, "Multicast Block All Low", ## args) \
+ X(IDT_NT_PCI_MCBLKALLH, 0x0034C, REGDWORD, "Multicast Block All High", ## args) \
+ X(IDT_NT_PCI_MCBLKUTL, 0x00350, REGDWORD, "Multicast Block Untranslated Low", ## args) \
+ X(IDT_NT_PCI_MCBLKUTH, 0x00354, REGDWORD, "Multicast Block Untranslated High", ## args) \
+ /*==========================================*/ \
+ /* IDT Proprietary NT-port-specific registers */ \
+ /* NT-function main control registers */ \
+ X(IDT_NT_PCI_NTCTL, 0x00400, REGDWORD, "NT Endpoint Control", ## args) \
+ X(IDT_NT_PCI_NTINTSTS, 0x00404, REGDWORD, "NT Endpoint Interrupt Status", ## args) \
+ X(IDT_NT_PCI_NTINTMSK, 0x00408, REGDWORD, "NT Endpoint Interrupt Mask", ## args) \
+ X(IDT_NT_PCI_NTSDATA, 0x0040C, REGDWORD, "NT Endpoint Signal Data", ## args) \
+ X(IDT_NT_PCI_NTGSIGNAL, 0x00410, REGDWORD, "NT Endpoint Global Signal", ## args) \
+ X(IDT_NT_PCI_NTIERRORMSK0, 0x00414, REGDWORD, "Internal Error Reporting Mask 0", ## args) \
+ X(IDT_NT_PCI_NTIERRORMSK1, 0x00418, REGDWORD, "Internal Error Reporting Mask 1", ## args) \
+ /* Doorbel registers */ \
+ X(IDT_NT_PCI_OUTDBELLSET, 0x00420, REGDWORD, "NT Outbound Doorbell Set", ## args) \
+ X(IDT_NT_PCI_INDBELLSTS, 0x00428, REGDWORD, "NT Inbound Doorbell Status", ## args) \
+ X(IDT_NT_PCI_INDBELLMSK, 0x0042C, REGDWORD, "NT Inbound Doorbell Mask", ## args) \
+ /* Message registers */ \
+ X(IDT_NT_PCI_OUTMSG0, 0x00430, REGDWORD, "Outbound Message 0", ## args) \
+ X(IDT_NT_PCI_OUTMSG1, 0x00434, REGDWORD, "Outbound Message 1", ## args) \
+ X(IDT_NT_PCI_OUTMSG2, 0x00438, REGDWORD, "Outbound Message 2", ## args) \
+ X(IDT_NT_PCI_OUTMSG3, 0x0043C, REGDWORD, "Outbound Message 3", ## args) \
+ X(IDT_NT_PCI_INMSG0, 0x00440, REGDWORD, "Inbound Message 0", ## args) \
+ X(IDT_NT_PCI_INMSG1, 0x00444, REGDWORD, "Inbound Message 1", ## args) \
+ X(IDT_NT_PCI_INMSG2, 0x00448, REGDWORD, "Inbound Message 2", ## args) \
+ X(IDT_NT_PCI_INMSG3, 0x0044C, REGDWORD, "Inbound Message 3", ## args) \
+ X(IDT_NT_PCI_INMSGSRC0, 0x00450, REGDWORD, "Inbound Message Source 0", ## args) \
+ X(IDT_NT_PCI_INMSGSRC1, 0x00454, REGDWORD, "Inbound Message Source 1", ## args) \
+ X(IDT_NT_PCI_INMSGSRC2, 0x00458, REGDWORD, "Inbound Message Source 2", ## args) \
+ X(IDT_NT_PCI_INMSGSRC3, 0x0045C, REGDWORD, "Inbound Message Source 3", ## args) \
+ X(IDT_NT_PCI_MSGSTS, 0x00460, REGDWORD, "Message Status", ## args) \
+ X(IDT_NT_PCI_MSGSTSMSK, 0x00464, REGDWORD, "Message Status Mask", ## args) \
+ /* BAR-setup registers */ \
+ X(IDT_NT_PCI_BARSETUP0, 0x00470, REGDWORD, "BAR 0 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT0, 0x00474, REGDWORD, "BAR 0 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE0, 0x00478, REGDWORD, "BAR 0 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE0, 0x0047C, REGDWORD, "BAR 0 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP1, 0x00480, REGDWORD, "BAR 1 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT1, 0x00484, REGDWORD, "BAR 1 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE1, 0x00488, REGDWORD, "BAR 1 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE1, 0x0048C, REGDWORD, "BAR 1 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP2, 0x00490, REGDWORD, "BAR 2 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT2, 0x00494, REGDWORD, "BAR 2 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE2, 0x00498, REGDWORD, "BAR 2 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE2, 0x0049C, REGDWORD, "BAR 2 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP3, 0x004A0, REGDWORD, "BAR 3 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT3, 0x004A4, REGDWORD, "BAR 3 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE3, 0x004A8, REGDWORD, "BAR 3 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE3, 0x004AC, REGDWORD, "BAR 3 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP4, 0x004B0, REGDWORD, "BAR 4 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT4, 0x004B4, REGDWORD, "BAR 4 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE4, 0x004B8, REGDWORD, "BAR 4 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE4, 0x004BC, REGDWORD, "BAR 4 Upper Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARSETUP5, 0x004C0, REGDWORD, "BAR 5 Setup", ## args) \
+ X(IDT_NT_PCI_BARLIMIT5, 0x004C4, REGDWORD, "BAR 5 Limit Address", ## args) \
+ X(IDT_NT_PCI_BARLTBASE5, 0x004C8, REGDWORD, "BAR 5 Lower Translated Base Address", ## args) \
+ X(IDT_NT_PCI_BARUTBASE5, 0x004CC, REGDWORD, "BAR 5 Upper Translated Base Address", ## args) \
+ /* NT mapping table registers */ \
+ X(IDT_NT_PCI_NTMTBLADDR, 0x004D0, REGDWORD, "NT Mapping Table Address", ## args) \
+ X(IDT_NT_PCI_NTMTBLSTS, 0x004D4, REGDWORD, "NT Mapping Table Status", ## args) \
+ X(IDT_NT_PCI_NTMTBLDATA, 0x004D8, REGDWORD, "NT Mapping Table Data", ## args) \
+ X(IDT_NT_PCI_REQIDCAP, 0x004DC, REGDWORD, "Requester ID Capture", ## args) \
+ /* Memory Windows Lookup table registers */ \
+ X(IDT_NT_PCI_LUTOFFSET, 0x004E0, REGDWORD, "Lookup Table Offset", ## args) \
+ X(IDT_NT_PCI_LUTLDATA, 0x004E4, REGDWORD, "Lookup Table Lower Data", ## args) \
+ X(IDT_NT_PCI_LUTMDATA, 0x004E8, REGDWORD, "Lookup Table Middle Data", ## args) \
+ X(IDT_NT_PCI_LUTUDATA, 0x004EC, REGDWORD, "Lookup Table Upper Data", ## args) \
+ /* NT Endpoint Errors Emulation registers */ \
+ X(IDT_NT_PCI_NTUEEM, 0x004F0, REGDWORD, "NT Endpoint Uncorrectable Error Emulation", ## args) \
+ X(IDT_NT_PCI_NTCEEM, 0x004F4, REGDWORD, "NT Endpoint Correctable Error Emulation", ## args) \
+ /* Punch-through registers */ \
+ X(IDT_NT_PCI_PTCCTL0, 0x00510, REGDWORD, "Punch-Through Configuration Control 0", ## args) \
+ X(IDT_NT_PCI_PTCCTL1, 0x00514, REGDWORD, "Punch-Through Configuration Control 1", ## args) \
+ X(IDT_NT_PCI_PTCDATA, 0x00518, REGDWORD, "Punch-Through Data", ## args) \
+ X(IDT_NT_PCI_PTCSTS, 0x0051C, REGDWORD, "Punch-Through Status", ## args) \
+ /* NT Multicast Group x Port association */ \
+ X(IDT_NT_PCI_NTMCG0PA, 0x00600, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ X(IDT_NT_PCI_NTMCG1PA, 0x00604, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ X(IDT_NT_PCI_NTMCG2PA, 0x00608, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ X(IDT_NT_PCI_NTMCG3PA, 0x0060C, REGDWORD, "NT Multicast Group x Port Association", ## args) \
+ /* Global Address Space Access registers */ \
+ /*X(IDT_NT_PCI_GASAADDR, 0x00FF8, REGDWORD, "Global Address Space Access Address", ## args) \
+ *X(IDT_NT_PCI_GASADATA, 0x00FFC, REGDWORD, "Global Address Space Access Data", ## args)*/
+
+/*
+ * Table of the IDT PCIe-switch Global Configuration and Status
+ * registers, corresponding size and the string name
+ */
+#define IDT_SW_CFGREGS(X, args...) \
+ /* Basic NT-function globally accessed registers */ \
+ /* Port 0 */ \
+ X(IDT_SW_PCI_NTP0_CMD, 0x01004, REGWORD, "Port 0 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP0_PCIELSTS, 0x01052, REGWORD, "Port 0 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP0_NTSDATA, 0x0140C, REGDWORD, "Port 0 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP0_NTGSIGNAL, 0x01410, REGDWORD, "Port 0 NT Global Signal", ## args) \
+ /* Port 2 */ \
+ X(IDT_SW_PCI_NTP2_CMD, 0x05004, REGWORD, "Port 2 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP2_PCIELSTS, 0x05052, REGWORD, "Port 2 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP2_NTSDATA, 0x0540C, REGDWORD, "Port 2 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP2_NTGSIGNAL, 0x05410, REGDWORD, "Port 2 NT Global Signal", ## args) \
+ /* Port 4 */ \
+ X(IDT_SW_PCI_NTP4_CMD, 0x09004, REGWORD, "Port 4 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP4_PCIELSTS, 0x09052, REGWORD, "Port 4 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP4_NTSDATA, 0x0940C, REGDWORD, "Port 4 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP4_NTGSIGNAL, 0x09410, REGDWORD, "Port 4 NT Global Signal", ## args) \
+ /* Port 6 */ \
+ X(IDT_SW_PCI_NTP6_CMD, 0x0D004, REGWORD, "Port 6 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP6_PCIELSTS, 0x0D052, REGWORD, "Port 6 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP6_NTSDATA, 0x0D40C, REGDWORD, "Port 6 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP6_NTGSIGNAL, 0x0D410, REGDWORD, "Port 6 NT Global Signal", ## args) \
+ /* Port 8 */ \
+ X(IDT_SW_PCI_NTP8_CMD, 0x11004, REGWORD, "Port 8 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP8_PCIELSTS, 0x11052, REGWORD, "Port 8 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP8_NTSDATA, 0x1140C, REGDWORD, "Port 8 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP8_NTGSIGNAL, 0x11410, REGDWORD, "Port 8 NT Global Signal", ## args) \
+ /* Port 12 */ \
+ X(IDT_SW_PCI_NTP12_CMD, 0x19004, REGWORD, "Port 12 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP12_PCIELSTS, 0x19052, REGWORD, "Port 12 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP12_NTSDATA, 0x1940C, REGDWORD, "Port 12 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP12_NTGSIGNAL, 0x19410, REGDWORD, "Port 12 NT Global Signal", ## args) \
+ /* Port 16 */ \
+ X(IDT_SW_PCI_NTP16_CMD, 0x21004, REGWORD, "Port 16 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP16_PCIELSTS, 0x21052, REGWORD, "Port 16 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP16_NTSDATA, 0x2140C, REGDWORD, "Port 16 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP16_NTGSIGNAL, 0x21410, REGDWORD, "Port 16 NT Global Signal", ## args) \
+ /* Port 20 */ \
+ X(IDT_SW_PCI_NTP20_CMD, 0x29004, REGWORD, "Port 20 PCI Command", ## args) \
+ X(IDT_SW_PCI_NTP20_PCIELSTS, 0x29052, REGWORD, "Port 20 PCIe link status", ## args) \
+ X(IDT_SW_PCI_NTP20_NTSDATA, 0x2940C, REGDWORD, "Port 20 NT Signal data", ## args) \
+ X(IDT_SW_PCI_NTP20_NTGSIGNAL, 0x29410, REGDWORD, "Port 20 NT Global Signal", ## args) \
+ /* IDT PCIe-switch control registers */ \
+ X(IDT_SW_PCI_SWCTL, 0x3E000, REGDWORD, "Switch Control", ## args) \
+ X(IDT_SW_PCI_BCVSTS, 0x3E004, REGDWORD, "Boot Configuration Vector Status", ## args) \
+ X(IDT_SW_PCI_PCLKMODE, 0x3E008, REGDWORD, "Port Clocking Mode", ## args) \
+ X(IDT_SW_PCI_STK0CFG, 0x3E010, REGDWORD, "Stack 0 Configuration", ## args) \
+ X(IDT_SW_PCI_STK1CFG, 0x3E014, REGDWORD, "Stack 1 Configuration", ## args) \
+ X(IDT_SW_PCI_STK2CFG, 0x3E018, REGDWORD, "Stack 2 Configuration", ## args) \
+ X(IDT_SW_PCI_STK3CFG, 0x3E01C, REGDWORD, "Stack 3 Configuration", ## args) \
+ /* Switch initialization delays */ \
+ X(IDT_SW_PCI_RDRAINDELAY, 0x3E080, REGDWORD, "Reset Drain Delay ", ## args) \
+ X(IDT_SW_PCI_POMCDELAY, 0x3E084, REGDWORD, "Port Operating Mode Change Drain Delay", ## args) \
+ X(IDT_SW_PCI_SEDELAY, 0x3E088, REGDWORD, "Side Effect Delay", ## args) \
+ X(IDT_SW_PCI_USSBRDELAY, 0x3E08C, REGDWORD, "Upstream Secondary Bus Reset Delay", ## args) \
+ /* Switch Partitions control and status registers */ \
+ X(IDT_SW_PCI_SWPART0CTL, 0x3E100, REGDWORD, "Switch Partition 0 Control", ## args) \
+ X(IDT_SW_PCI_SWPART0STS, 0x3E104, REGDWORD, "Switch Partition 0 Status", ## args) \
+ X(IDT_SW_PCI_SWPART0FCTL, 0x3E108, REGDWORD, "Switch Partition 0 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART1CTL, 0x3E120, REGDWORD, "Switch Partition 1 Control", ## args) \
+ X(IDT_SW_PCI_SWPART1STS, 0x3E124, REGDWORD, "Switch Partition 1 Status", ## args) \
+ X(IDT_SW_PCI_SWPART1FCTL, 0x3E128, REGDWORD, "Switch Partition 1 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART2CTL, 0x3E140, REGDWORD, "Switch Partition 2 Control", ## args) \
+ X(IDT_SW_PCI_SWPART2STS, 0x3E144, REGDWORD, "Switch Partition 2 Status", ## args) \
+ X(IDT_SW_PCI_SWPART2FCTL, 0x3E148, REGDWORD, "Switch Partition 2 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART3CTL, 0x3E160, REGDWORD, "Switch Partition 3 Control", ## args) \
+ X(IDT_SW_PCI_SWPART3STS, 0x3E164, REGDWORD, "Switch Partition 3 Status", ## args) \
+ X(IDT_SW_PCI_SWPART3FCTL, 0x3E168, REGDWORD, "Switch Partition 3 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART4CTL, 0x3E180, REGDWORD, "Switch Partition 4 Control", ## args) \
+ X(IDT_SW_PCI_SWPART4STS, 0x3E184, REGDWORD, "Switch Partition 4 Status", ## args) \
+ X(IDT_SW_PCI_SWPART4FCTL, 0x3E188, REGDWORD, "Switch Partition 4 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART5CTL, 0x3E1A0, REGDWORD, "Switch Partition 5 Control", ## args) \
+ X(IDT_SW_PCI_SWPART5STS, 0x3E1A4, REGDWORD, "Switch Partition 5 Status", ## args) \
+ X(IDT_SW_PCI_SWPART5FCTL, 0x3E1A8, REGDWORD, "Switch Partition 5 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART6CTL, 0x3E1C0, REGDWORD, "Switch Partition 6 Control", ## args) \
+ X(IDT_SW_PCI_SWPART6STS, 0x3E1C4, REGDWORD, "Switch Partition 6 Status", ## args) \
+ X(IDT_SW_PCI_SWPART6FCTL, 0x3E1C8, REGDWORD, "Switch Partition 6 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPART7CTL, 0x3E1E0, REGDWORD, "Switch Partition 7 Control", ## args) \
+ X(IDT_SW_PCI_SWPART7STS, 0x3E1E4, REGDWORD, "Switch Partition 7 Status", ## args) \
+ X(IDT_SW_PCI_SWPART7FCTL, 0x3E1E8, REGDWORD, "Switch Partition 7 Failover Control", ## args) \
+ /* Switch Ports control and status registers */ \
+ X(IDT_SW_PCI_SWPORT0CTL, 0x3E200, REGDWORD, "Switch Port 0 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT0STS, 0x3E204, REGDWORD, "Switch Port 0 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT0FCTL, 0x3E208, REGDWORD, "Switch Port 0 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT2CTL, 0x3E240, REGDWORD, "Switch Port 2 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT2STS, 0x3E244, REGDWORD, "Switch Port 2 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT2FCTL, 0x3E248, REGDWORD, "Switch Port 2 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT4CTL, 0x3E280, REGDWORD, "Switch Port 4 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT4STS, 0x3E284, REGDWORD, "Switch Port 4 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT4FCTL, 0x3E288, REGDWORD, "Switch Port 4 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT6CTL, 0x3E2C0, REGDWORD, "Switch Port 6 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT6STS, 0x3E2C4, REGDWORD, "Switch Port 6 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT6FCTL, 0x3E2C8, REGDWORD, "Switch Port 6 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT8CTL, 0x3E300, REGDWORD, "Switch Port 8 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT8STS, 0x3E304, REGDWORD, "Switch Port 8 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT8FCTL, 0x3E308, REGDWORD, "Switch Port 8 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT12CTL, 0x3E380, REGDWORD, "Switch Port 12 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT12STS, 0x3E384, REGDWORD, "Switch Port 12 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT12FCTL, 0x3E388, REGDWORD, "Switch Port 12 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT16CTL, 0x3E400, REGDWORD, "Switch Port 16 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT16STS, 0x3E404, REGDWORD, "Switch Port 16 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT16FCTL, 0x3E408, REGDWORD, "Switch Port 16 Failover Control", ## args) \
+ X(IDT_SW_PCI_SWPORT20CTL, 0x3E480, REGDWORD, "Switch Port 20 Control", ## args) \
+ X(IDT_SW_PCI_SWPORT20STS, 0x3E484, REGDWORD, "Switch Port 20 Status", ## args) \
+ X(IDT_SW_PCI_SWPORT20FCTL, 0x3E488, REGDWORD, "Switch Port 20 Failover Control", ## args) \
+ /* Failover capability control and status registers */ \
+ X(IDT_SW_PCI_FCAP0CTL, 0x3E500, REGDWORD, "Failover Capability 0 Control", ## args) \
+ X(IDT_SW_PCI_FCAP0STS, 0x3E504, REGDWORD, "Failover Capability 0 Status", ## args) \
+ X(IDT_SW_PCI_FCAP0TIMER, 0x3E508, REGDWORD, "Failover Capability 0 Watchdog Timer", ## args) \
+ X(IDT_SW_PCI_FCAP1CTL, 0x3E520, REGDWORD, "Failover Capability 1 Control", ## args) \
+ X(IDT_SW_PCI_FCAP1STS, 0x3E524, REGDWORD, "Failover Capability 1 Status", ## args) \
+ X(IDT_SW_PCI_FCAP1TIMER, 0x3E528, REGDWORD, "Failover Capability 1 Watchdog Timer", ## args) \
+ X(IDT_SW_PCI_FCAP2CTL, 0x3E540, REGDWORD, "Failover Capability 2 Control", ## args) \
+ X(IDT_SW_PCI_FCAP2STS, 0x3E544, REGDWORD, "Failover Capability 2 Status", ## args) \
+ X(IDT_SW_PCI_FCAP2TIMER, 0x3E548, REGDWORD, "Failover Capability 2 Watchdog Timer", ## args) \
+ X(IDT_SW_PCI_FCAP3CTL, 0x3E560, REGDWORD, "Failover Capability 3 Control", ## args) \
+ X(IDT_SW_PCI_FCAP3STS, 0x3E564, REGDWORD, "Failover Capability 3 Status", ## args) \
+ X(IDT_SW_PCI_FCAP3TIMER, 0x3E568, REGDWORD, "Failover Capability 3 Watchdog Timer", ## args) \
+ /* Protection registers */ \
+ X(IDT_SW_PCI_GASAPROT, 0x3E700, REGDWORD, "Global Address Space Access Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT0, 0x3E710, REGDWORD, "Partition 0 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT1, 0x3E714, REGDWORD, "Partition 1 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT2, 0x3E718, REGDWORD, "Partition 2 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT3, 0x3E71C, REGDWORD, "Partition 3 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT4, 0x3E720, REGDWORD, "Partition 4 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT5, 0x3E724, REGDWORD, "Partition 5 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT6, 0x3E728, REGDWORD, "Partition 6 NT Mapping Table Protection", ## args) \
+ X(IDT_SW_PCI_NTMTBLPROT7, 0x3E72C, REGDWORD, "Partition 7 NT Mapping Table Protection", ## args) \
+ /* Switch Event registers */ \
+ X(IDT_SW_PCI_SESTS, 0x3EC00, REGDWORD, "Switch Event Status", ## args) \
+ X(IDT_SW_PCI_SEMSK, 0x3EC04, REGDWORD, "Switch Event Mask", ## args) \
+ X(IDT_SW_PCI_SEPMSK, 0x3EC08, REGDWORD, "Switch Event Partition Mask", ## args) \
+ X(IDT_SW_PCI_SELINKUPSTS, 0x3EC0C, REGDWORD, "Switch Event Link Up Status", ## args) \
+ X(IDT_SW_PCI_SELINKUPMSK, 0x3EC10, REGDWORD, "Switch Event Link Up Mask", ## args) \
+ X(IDT_SW_PCI_SELINKDNSTS, 0x3EC14, REGDWORD, "Switch Event Link Down Status", ## args) \
+ X(IDT_SW_PCI_SELINKDNMSK, 0x3EC18, REGDWORD, "Switch Event Link Down Mask", ## args) \
+ X(IDT_SW_PCI_SEFRSTSTS, 0x3EC1C, REGDWORD, "Switch Event Fundamental Reset Status", ## args) \
+ X(IDT_SW_PCI_SEFRSTMSK, 0x3EC20, REGDWORD, "Switch Event Fundamental Reset Mask", ## args) \
+ X(IDT_SW_PCI_SEHRSTSTS, 0x3EC24, REGDWORD, "Switch Event Hot Reset Status", ## args) \
+ X(IDT_SW_PCI_SEHRSTMSK, 0x3EC28, REGDWORD, "Switch Event Hot Reset Mask", ## args) \
+ X(IDT_SW_PCI_SEFOVRMSK, 0x3EC2C, REGDWORD, "Switch Event Failover Mask", ## args) \
+ X(IDT_SW_PCI_SEGSIGSTS, 0x3EC30, REGDWORD, "Switch Event Global Signal Status", ## args) \
+ X(IDT_SW_PCI_SEGSIGMSK, 0x3EC34, REGDWORD, "Switch Event Global Signal Mask", ## args) \
+ /* Global Doorbell configuration registers */ \
+ X(IDT_SW_PCI_GDBELLSTS, 0x3EC3C, REGDWORD, "NT Global Doorbell Status", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK0, 0x3ED00, REGDWORD, "NT Global Outbound Doorbell 0 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK1, 0x3ED04, REGDWORD, "NT Global Outbound Doorbell 1 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK2, 0x3ED08, REGDWORD, "NT Global Outbound Doorbell 2 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK3, 0x3ED0C, REGDWORD, "NT Global Outbound Doorbell 3 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK4, 0x3ED10, REGDWORD, "NT Global Outbound Doorbell 4 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK5, 0x3ED14, REGDWORD, "NT Global Outbound Doorbell 5 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK6, 0x3ED18, REGDWORD, "NT Global Outbound Doorbell 6 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK7, 0x3ED1C, REGDWORD, "NT Global Outbound Doorbell 7 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK8, 0x3ED20, REGDWORD, "NT Global Outbound Doorbell 8 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK9, 0x3ED24, REGDWORD, "NT Global Outbound Doorbell 9 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK10, 0x3ED28, REGDWORD, "NT Global Outbound Doorbell 10 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK11, 0x3ED2C, REGDWORD, "NT Global Outbound Doorbell 11 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK12, 0x3ED30, REGDWORD, "NT Global Outbound Doorbell 12 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK13, 0x3ED34, REGDWORD, "NT Global Outbound Doorbell 13 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK14, 0x3ED38, REGDWORD, "NT Global Outbound Doorbell 14 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK15, 0x3ED3C, REGDWORD, "NT Global Outbound Doorbell 15 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK16, 0x3ED40, REGDWORD, "NT Global Outbound Doorbell 16 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK17, 0x3ED44, REGDWORD, "NT Global Outbound Doorbell 17 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK18, 0x3ED48, REGDWORD, "NT Global Outbound Doorbell 18 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK19, 0x3ED4C, REGDWORD, "NT Global Outbound Doorbell 19 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK20, 0x3ED50, REGDWORD, "NT Global Outbound Doorbell 20 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK21, 0x3ED54, REGDWORD, "NT Global Outbound Doorbell 21 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK22, 0x3ED58, REGDWORD, "NT Global Outbound Doorbell 22 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK23, 0x3ED5C, REGDWORD, "NT Global Outbound Doorbell 23 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK24, 0x3ED60, REGDWORD, "NT Global Outbound Doorbell 24 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK25, 0x3ED64, REGDWORD, "NT Global Outbound Doorbell 25 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK26, 0x3ED68, REGDWORD, "NT Global Outbound Doorbell 26 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK27, 0x3ED6C, REGDWORD, "NT Global Outbound Doorbell 27 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK28, 0x3ED70, REGDWORD, "NT Global Outbound Doorbell 28 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK29, 0x3ED74, REGDWORD, "NT Global Outbound Doorbell 29 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK30, 0x3ED78, REGDWORD, "NT Global Outbound Doorbell 30 Mask", ## args) \
+ X(IDT_SW_PCI_GODBELLMSK31, 0x3ED7C, REGDWORD, "NT Global Outbound Doorbell 31 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK0, 0x3ED80, REGDWORD, "NT Global Inbound Doorbell 0 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK1, 0x3ED84, REGDWORD, "NT Global Inbound Doorbell 1 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK2, 0x3ED88, REGDWORD, "NT Global Inbound Doorbell 2 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK3, 0x3ED8C, REGDWORD, "NT Global Inbound Doorbell 3 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK4, 0x3ED90, REGDWORD, "NT Global Inbound Doorbell 4 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK5, 0x3ED94, REGDWORD, "NT Global Inbound Doorbell 5 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK6, 0x3ED98, REGDWORD, "NT Global Inbound Doorbell 6 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK7, 0x3ED9C, REGDWORD, "NT Global Inbound Doorbell 7 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK8, 0x3EDA0, REGDWORD, "NT Global Inbound Doorbell 8 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK9, 0x3EDA4, REGDWORD, "NT Global Inbound Doorbell 9 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK10, 0x3EDA8, REGDWORD, "NT Global Inbound Doorbell 10 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK11, 0x3EDAC, REGDWORD, "NT Global Inbound Doorbell 11 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK12, 0x3EDB0, REGDWORD, "NT Global Inbound Doorbell 12 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK13, 0x3EDB4, REGDWORD, "NT Global Inbound Doorbell 13 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK14, 0x3EDB8, REGDWORD, "NT Global Inbound Doorbell 14 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK15, 0x3EDBC, REGDWORD, "NT Global Inbound Doorbell 15 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK16, 0x3EDC0, REGDWORD, "NT Global Inbound Doorbell 16 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK17, 0x3EDC4, REGDWORD, "NT Global Inbound Doorbell 17 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK18, 0x3EDC8, REGDWORD, "NT Global Inbound Doorbell 18 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK19, 0x3EDCC, REGDWORD, "NT Global Inbound Doorbell 19 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK20, 0x3EDD0, REGDWORD, "NT Global Inbound Doorbell 20 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK21, 0x3EDD4, REGDWORD, "NT Global Inbound Doorbell 21 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK22, 0x3EDD8, REGDWORD, "NT Global Inbound Doorbell 22 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK23, 0x3EDDC, REGDWORD, "NT Global Inbound Doorbell 23 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK24, 0x3EDE0, REGDWORD, "NT Global Inbound Doorbell 24 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK25, 0x3EDE4, REGDWORD, "NT Global Inbound Doorbell 25 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK26, 0x3EDE8, REGDWORD, "NT Global Inbound Doorbell 26 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK27, 0x3EDEC, REGDWORD, "NT Global Inbound Doorbell 27 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK28, 0x3EDF0, REGDWORD, "NT Global Inbound Doorbell 28 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK29, 0x3EDF4, REGDWORD, "NT Global Inbound Doorbell 29 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK30, 0x3EDF8, REGDWORD, "NT Global Inbound Doorbell 30 Mask", ## args) \
+ X(IDT_SW_PCI_GIDBELLMSK31, 0x3EDFC, REGDWORD, "NT Global Inbound Doorbell 31 Mask", ## args) \
+ /* Switch partition messages control (msgs routing table) */ \
+ X(IDT_SW_PCI_SWP0MSGCTL0, 0x3EE00, REGDWORD, "Switch Partition 0 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL0, 0x3EE04, REGDWORD, "Switch Partition 1 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL0, 0x3EE08, REGDWORD, "Switch Partition 2 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL0, 0x3EE0C, REGDWORD, "Switch Partition 3 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL0, 0x3EE10, REGDWORD, "Switch Partition 4 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL0, 0x3EE14, REGDWORD, "Switch Partition 5 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL0, 0x3EE18, REGDWORD, "Switch Partition 6 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL0, 0x3EE1C, REGDWORD, "Switch Partition 7 Message Control 0", ## args) \
+ X(IDT_SW_PCI_SWP0MSGCTL1, 0x3EE20, REGDWORD, "Switch Partition 0 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL1, 0x3EE24, REGDWORD, "Switch Partition 1 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL1, 0x3EE28, REGDWORD, "Switch Partition 2 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL1, 0x3EE2C, REGDWORD, "Switch Partition 3 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL1, 0x3EE30, REGDWORD, "Switch Partition 4 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL1, 0x3EE34, REGDWORD, "Switch Partition 5 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL1, 0x3EE38, REGDWORD, "Switch Partition 6 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL1, 0x3EE3C, REGDWORD, "Switch Partition 7 Message Control 1", ## args) \
+ X(IDT_SW_PCI_SWP0MSGCTL2, 0x3EE40, REGDWORD, "Switch Partition 0 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL2, 0x3EE44, REGDWORD, "Switch Partition 1 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL2, 0x3EE48, REGDWORD, "Switch Partition 2 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL2, 0x3EE4C, REGDWORD, "Switch Partition 3 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL2, 0x3EE50, REGDWORD, "Switch Partition 4 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL2, 0x3EE54, REGDWORD, "Switch Partition 5 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL2, 0x3EE58, REGDWORD, "Switch Partition 6 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL2, 0x3EE5C, REGDWORD, "Switch Partition 7 Message Control 2", ## args) \
+ X(IDT_SW_PCI_SWP0MSGCTL3, 0x3EE60, REGDWORD, "Switch Partition 0 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP1MSGCTL3, 0x3EE64, REGDWORD, "Switch Partition 1 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP2MSGCTL3, 0x3EE68, REGDWORD, "Switch Partition 2 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP3MSGCTL3, 0x3EE6C, REGDWORD, "Switch Partition 3 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP4MSGCTL3, 0x3EE70, REGDWORD, "Switch Partition 4 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP5MSGCTL3, 0x3EE74, REGDWORD, "Switch Partition 5 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP6MSGCTL3, 0x3EE78, REGDWORD, "Switch Partition 6 Message Control 3", ## args) \
+ X(IDT_SW_PCI_SWP7MSGCTL3, 0x3EE7C, REGDWORD, "Switch Partition 7 Message Control 3", ## args) \
+ /* SerDes's control registers */ \
+ X(IDT_SW_PCI_S0CTL, 0x3F000, REGDWORD, "SerDes 0 Control", ## args) \
+ X(IDT_SW_PCI_S0TXLCTL0, 0x3F004, REGDWORD, "SerDes 0 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S0TXLCTL1, 0x3F008, REGDWORD, "SerDes 0 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S0RXEQLCTL, 0x3F010, REGDWORD, "SerDes 0 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S1CTL, 0x3F020, REGDWORD, "SerDes 1 Control", ## args) \
+ X(IDT_SW_PCI_S1TXLCTL0, 0x3F024, REGDWORD, "SerDes 1 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S1TXLCTL1, 0x3F028, REGDWORD, "SerDes 1 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S1RXEQLCTL, 0x3F030, REGDWORD, "SerDes 1 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S2CTL, 0x3F040, REGDWORD, "SerDes 2 Control", ## args) \
+ X(IDT_SW_PCI_S2TXLCTL0, 0x3F044, REGDWORD, "SerDes 2 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S2TXLCTL1, 0x3F048, REGDWORD, "SerDes 2 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S2RXEQLCTL, 0x3F050, REGDWORD, "SerDes 2 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S3CTL, 0x3F060, REGDWORD, "SerDes 3 Control", ## args) \
+ X(IDT_SW_PCI_S3TXLCTL0, 0x3F064, REGDWORD, "SerDes 3 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S3TXLCTL1, 0x3F068, REGDWORD, "SerDes 3 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S3RXEQLCTL, 0x3F070, REGDWORD, "SerDes 3 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S4CTL, 0x3F080, REGDWORD, "SerDes 4 Control", ## args) \
+ X(IDT_SW_PCI_S4TXLCTL0, 0x3F084, REGDWORD, "SerDes 4 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S4TXLCTL1, 0x3F088, REGDWORD, "SerDes 4 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S4RXEQLCTL, 0x3F090, REGDWORD, "SerDes 4 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S5CTL, 0x3F0A0, REGDWORD, "SerDes 5 Control", ## args) \
+ X(IDT_SW_PCI_S5TXLCTL0, 0x3F0A4, REGDWORD, "SerDes 5 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S5TXLCTL1, 0x3F0A8, REGDWORD, "SerDes 5 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S5RXEQLCTL, 0x3F0B0, REGDWORD, "SerDes 5 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S6CTL, 0x3F0C0, REGDWORD, "SerDes 6 Control", ## args) \
+ X(IDT_SW_PCI_S6TXLCTL0, 0x3F0C4, REGDWORD, "SerDes 6 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S6TXLCTL1, 0x3F0C8, REGDWORD, "SerDes 6 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S6RXEQLCTL, 0x3F0D0, REGDWORD, "SerDes 6 Receiver Equalization Lane Control", ## args) \
+ X(IDT_SW_PCI_S7CTL, 0x3F0E0, REGDWORD, "SerDes 7 Control", ## args) \
+ X(IDT_SW_PCI_S7TXLCTL0, 0x3F0E4, REGDWORD, "SerDes 7 Transmitter Lane Control 0", ## args) \
+ X(IDT_SW_PCI_S7TXLCTL1, 0x3F0E8, REGDWORD, "SerDes 7 Transmitter Lane Control 1", ## args) \
+ X(IDT_SW_PCI_S7RXEQLCTL, 0x3F0F0, REGDWORD, "SerDes 7 Receiver Equalization Lane Control", ## args) \
+ /* GPIO/Hot-plug control registers */ \
+ X(IDT_SW_PCI_GPIOFUNC, 0x3F16C, REGDWORD, "General Purpose I/O Function", ## args) \
+ X(IDT_SW_PCI_GPIOAFSEL, 0x3F170, REGDWORD, "General Purpose I/O Alternate Function Select", ## args) \
+ X(IDT_SW_PCI_GPIOCFG, 0x3F174, REGDWORD, "General Purpose I/O Configuration", ## args) \
+ X(IDT_SW_PCI_GPIOD, 0x3F178, REGDWORD, "General Purpose I/O Data", ## args) \
+ X(IDT_SW_PCI_HPCFGCTL, 0x3F17C, REGDWORD, "Hot-Plug Configuration Control", ## args) \
+ /* SMBus related registers */ \
+ X(IDT_SW_PCI_SMBUSSTS, 0x3F188, REGDWORD, "SMBus Status", ## args) \
+ X(IDT_SW_PCI_SMBUSCTL, 0x3F18C, REGDWORD, "SMBus Control", ## args) \
+ X(IDT_SW_PCI_EEPROMINTF, 0x3F190, REGDWORD, "Serial EEPROM Interface", ## args) \
+ /* SMBus IO expanders */ \
+ X(IDT_SW_PCI_IOEXPADDR0, 0x3F198, REGDWORD, "SMBus I/O Expander Address 0", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR1, 0x3F19C, REGDWORD, "SMBus I/O Expander Address 1", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR2, 0x3F1A0, REGDWORD, "SMBus I/O Expander Address 2", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR3, 0x3F1A4, REGDWORD, "SMBus I/O Expander Address 3", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR4, 0x3F1A8, REGDWORD, "SMBus I/O Expander Address 4", ## args) \
+ X(IDT_SW_PCI_IOEXPADDR5, 0x3F1AC, REGDWORD, "SMBus I/O Expander Address 5", ## args) \
+ /* General Purpose Events registers */ \
+ X(IDT_SW_PCI_GPECTL, 0x3F1B0, REGDWORD, "General Purpose Event Control", ## args) \
+ X(IDT_SW_PCI_GPESTS, 0x3F1B4, REGDWORD, "General Purpose Event Status", ## args) \
+ /* Temperature sensor */ \
+ X(IDT_SW_PCI_TMPCTL, 0x3F1D4, REGDWORD, "Temperature Sensor Control", ## args) \
+ X(IDT_SW_PCI_TMPSTS, 0x3F1D8, REGDWORD, "Temperature Sensor Status", ## args) \
+ X(IDT_SW_PCI_TMPALARM, 0x3F1DC, REGDWORD, "Temperature Sensor Alarm", ## args) \
+ X(IDT_SW_PCI_TMPADJ, 0x3F1E0, REGDWORD, "Temperature Sensor Adjustment", ## args) \
+ X(IDT_SW_PCI_TSSLOPE, 0x3F1E4, REGDWORD, "Temperature Sensor Slope", ## args) \
+ /* SMBus Configuration Block header log */ \
+ X(IDT_SW_PCI_SMBUSCBHL, 0x3F1E8, REGDWORD, "SMBus Configuration Block Header Log", ## args)
+
+/*
+ * Enumeration of the IDT PCIe-switch NT registers. It's not actual
+ * addresses or offsets, but the numerated names, which are used to find the
+ * necessary values from the tables above. Consenquently the switch-case
+ * shall help to retrieve all the information for the IO operations. Of course,
+ * we are sure the compiler will translate that statement into the jump table
+ * pattern.
+ *
+ * NOTE 1) The IDT PCIe-switch internal data is littel-endian
+ * so it must be taken into account in the driver
+ * internals.
+ * 2) Additionally the registers should be accessed either
+ * with byte-enables corresponding to their native size or
+ * a size of one DWORD
+ * 3) Global registers registers can be accessed by the
+ * GASAADDR and GASADATA registers of NT-functions only
+ */
+enum idt_ntb_cfgreg {
+ IDT_NT_CFGREGS(PAIR_ID_ENUM)
+ IDT_NTB_CFGREGS_SPLIT,
+ IDT_SW_CFGREGS(PAIR_ID_ENUM)
+ IDT_NTB_CFGREGS_END
+};
+
+/*
+ * IDT PCIe-switch register type. It's vital that the types are assigned
+ * with 0 and 1 since those values are used to determine the registers IO
+ * context
+ * @IDT_NT_REGTYPE: NT-function register accessed using the mmio
+ * @IDT_SW_REGTYPE: IDT PCIe-switch Gobal register accessed using GASA regs
+ */
+enum idt_ntb_regtype {
+ IDT_NT_REGTYPE = 0,
+ IDT_SW_REGTYPE = 1
+};
+
+/*
+ * R/W registers operation context structure
+ * @writereg: Register write function
+ * @readreg: Register read function
+ * @iolock: Spin lock of the registers access
+ */
+struct idt_ntb_regctx {
+ void (*writereg)(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize, const u32 val);
+ u32 (*readreg)(void __iomem *cfg_mmio, const ptrdiff_t regoffset,
+ const enum idt_ntb_regsize regsize);
+ spinlock_t iolock;
+};
+
+#endif /* NTB_HW_IDT_REGMAP_H */
--
2.6.6
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 3/3] ntb: Test client drivers for asynchronous NTB devices
2016-07-28 10:01 ` [PATCH v2 0/3] ntb: Asynchronous NTB devices support Serge Semin
2016-07-28 10:01 ` [PATCH v2 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
2016-07-28 10:01 ` [PATCH v2 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
@ 2016-07-28 10:01 ` Serge Semin
2016-08-01 2:49 ` kbuild test robot
2 siblings, 1 reply; 11+ messages in thread
From: Serge Semin @ 2016-07-28 10:01 UTC (permalink / raw)
To: jdmason
Cc: dave.jiang, Allen.Hubbe, Xiangliang.Yu, Sergey.Semin, linux-ntb,
linux-kernel, Serge Semin
There are three drivers to independently test all interfaces implemented by
the IDT 89HPES*NT* NTB driver.
Doorbells are tested by new NTB Doorbell Pingpong client driver. It implements
the so-named algorithm. Driver starts working from setting the peer doorbell of
the last locally set doorbell bit. If there has not been locally set doorbell,
it sets the very first bit. After that the driver unmasks the events of the
just set doorbell bit and waits until the peer sets the same doorbell. When
peer does it, the local driver iterates to the next doorbell bit and starts
delayed work thread, which will set the corresponding bit and perform doorbell
bit umasking on waking up.
Messaging subsystem can be tested by the client driver implementing a simple
transmition/reception algorithm. A message can be send to a peer by writing
data to /sys/kernel/debug/ntb_msg_test/ntbA{N}/data file. The peer can read
it from the same file then.
Memory Windows test driver implements a simple write/read algorithm. The driver
allocates the predefined number of local buffers (inbound memory window -
inwndw{N}). In order to get a translated base address driver sends a
corresponding command to a peer. Then driver initialize the outbound memory
windows (outwndw{N}). The read/write operations can be performed using the
following debug nodes:
/sys/kernel/debug/ntb_mw_test/ntbA{N}/inwndw{N}
/sys/kernel/debug/ntb_mw_test/ntbA{N}/outwndw{N}
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
drivers/ntb/test/Kconfig | 32 +
drivers/ntb/test/Makefile | 9 +-
drivers/ntb/test/ntb_db_test.c | 677 +++++++++++++++++
drivers/ntb/test/ntb_msg_test.c | 736 +++++++++++++++++++
drivers/ntb/test/ntb_mw_test.c | 1539 +++++++++++++++++++++++++++++++++++++++
5 files changed, 2991 insertions(+), 2 deletions(-)
create mode 100644 drivers/ntb/test/ntb_db_test.c
create mode 100644 drivers/ntb/test/ntb_msg_test.c
create mode 100644 drivers/ntb/test/ntb_mw_test.c
diff --git a/drivers/ntb/test/Kconfig b/drivers/ntb/test/Kconfig
index a5d0eda..80f5058 100644
--- a/drivers/ntb/test/Kconfig
+++ b/drivers/ntb/test/Kconfig
@@ -25,3 +25,35 @@ config NTB_PERF
to and from the window without additional software interaction.
If unsure, say N.
+
+config NTB_DB_TEST
+ tristate "NTB Doorbell Test Client"
+ help
+ This is a driver to test doorbell subsystem of NTB bus devices.
+ The design is similar to the ping pong although it exchanges the
+ doorbell bits one-by-one, waiting for the peer response before getting
+ to a next doorbell.
+
+ If unsure, say N.
+
+config NTB_MSG_TEST
+ tristate "NTB Messaging Test Client"
+ help
+ This is a driver to test messaging subsystem of NTB. It just creates
+ one file in the DebugFS for each NTB device of asynchronous
+ architecture. In order to send a message one can just write a text to
+ the file. It will be immediately sent to the peer so user can get it
+ by reading from the corresponding file.
+
+ If unsure, say N.
+
+config NTB_MW_TEST
+ tristate "NTB Memory Windows Test Client"
+ help
+ This is a driver to test memory sharing amongst devices. It creates a
+ set of files in the DebugFS, one of which are used to write a text to
+ outbound memory windows and anothers can be used to read data written
+ by the peer to our inbound memory window.
+
+ If unsure, say N.
+
diff --git a/drivers/ntb/test/Makefile b/drivers/ntb/test/Makefile
index 9e77e0b..6ea6db4 100644
--- a/drivers/ntb/test/Makefile
+++ b/drivers/ntb/test/Makefile
@@ -1,3 +1,8 @@
+# Synchronous hardware clients (Intel/AMD)
obj-$(CONFIG_NTB_PINGPONG) += ntb_pingpong.o
-obj-$(CONFIG_NTB_TOOL) += ntb_tool.o
-obj-$(CONFIG_NTB_PERF) += ntb_perf.o
+obj-$(CONFIG_NTB_TOOL) += ntb_tool.o
+obj-$(CONFIG_NTB_PERF) += ntb_perf.o
+# Asynchronous hardware clients (IDT)
+obj-$(CONFIG_NTB_DB_TEST) += ntb_db_test.o
+obj-$(CONFIG_NTB_MSG_TEST) += ntb_msg_test.o
+obj-$(CONFIG_NTB_MW_TEST) += ntb_mw_test.o
diff --git a/drivers/ntb/test/ntb_db_test.c b/drivers/ntb/test/ntb_db_test.c
new file mode 100644
index 0000000..e93c0c6
--- /dev/null
+++ b/drivers/ntb/test/ntb_db_test.c
@@ -0,0 +1,677 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * PCIe NTB doorbells test Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the NTB doorbells pingpong driver design.
+ * The driver is designed to implement the pingpong algorithm. After a quick
+ * initailization the driver starts from setting the peer doorbell of the last
+ * locally set doorbell bit. If there is not any doorbell locally set, then it
+ * sets the very first bit. After that the driver unmasks the events of the
+ * just set bits and waits until the peer is set the same doorbell. When it's
+ * done, the driver iterates to the next doorbell and starts delayed work
+ * thread, which will set the corresponding bit and perform doorbell umasking
+ * on waking up.
+ */
+
+/* Note: You can load this module with either option 'dyndbg=+p' or define the
+ * next preprocessor constant */
+/*#define DEBUG*/
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+
+#include <linux/ntb.h>
+
+#define DRIVER_NAME "ntb_db_test"
+#define DRIVER_DESCRIPTION "PCIe NTB Doorbells Pingpong Client"
+#define DRIVER_VERSION "1.0"
+
+MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+static unsigned int delay_ms = 1000;
+module_param(delay_ms, uint, 0644);
+MODULE_PARM_DESC(delay_ms,
+ "Milliseconds to delay before setting a next doorbell bit");
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_dir;
+
+/*
+ * Enumeration of the driver states
+ * @PP_WAIT: Driver waits until the peer sets the corresponding doorbell bit
+ * @PP_SLEEP: Driver sleeps before to set the next doorbell bit
+ */
+enum db_pp_state {
+ PP_WAIT = 0,
+ PP_SLEEP = 1
+};
+
+/*
+ * Doorbells pingpong driver context
+ * @ntb: Pointer to the NTB device
+ * @cycle: Doorbells setting cycle made up until now
+ * @valid_ids: Valid Doorbel bits
+ * @delay: Delay between setting the next doorbell bit
+ * @state: Current cycle state
+ * @dwork: Kernel thread used to perform the delayed doorbell bit set
+ * @dbgfs_info: Handler of the DebugFS driver info-file
+ */
+struct pp_ctx {
+ struct ntb_dev *ntb;
+ unsigned long long cycle;
+ u64 valid_ids;
+ unsigned long delay;
+ enum db_pp_state state;
+ struct delayed_work dwork;
+ struct dentry *dbgfs_info;
+};
+#define to_ctx_dwork(work) \
+ container_of(to_delayed_work(work), struct pp_ctx, dwork)
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_pp(ctx, args...) \
+ dev_err(&ctx->ntb->dev, ## args)
+#define dev_warn_pp(ctx, args...) \
+ dev_warn(&ctx->ntb->dev, ## args)
+#define dev_info_pp(ctx, args...) \
+ dev_info(&ctx->ntb->dev, ## args)
+#define dev_dbg_pp(ctx, args...) \
+ dev_dbg(&ctx->ntb->dev, ## args)
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+
+/*===========================================================================
+ * Helper functions
+ *===========================================================================*/
+
+/*
+ * Create a contiguous bitmask starting at bit position @l and ending at
+ * position @h. For example
+ * GENMASK_ULL(39, 21) gives us the 64bit vector 0x000000ffffe00000.
+ */
+#ifndef GENMASK_ULL
+#define GENMASK_ULL(h, l) \
+ (((~0ULL) << (l)) & (~0ULL >> (BITS_PER_LONG_LONG - 1 - (h))))
+#endif /* !GENMASK_ULL */
+
+/*
+ * Set the corresponding bit in the 64-bits wide word
+ */
+#ifndef BIT_ULL
+#define BIT_ULL(nr) (1ULL << (nr))
+#endif /* !BIT_ULL */
+
+/*
+ * Method to find a first set bit in 64-bits wide word. The bits numbering is
+ * from 0 to 63. If there is no any set bit, then 64 is returned.
+ */
+static inline unsigned long find_first_bit64(u64 var)
+{
+ return (0x0ULL == var) ? BITS_PER_LONG_LONG : __ffs64(var);
+}
+
+/*
+ * Method to find a next set bit in 64-bits wide word starting from the
+ * specified position. The bits numbering is from 0 to 63. If there is no any
+ * set bit within the position and the last bit of the word, then 64 is
+ * returned.
+ */
+static inline unsigned long find_next_bit64(u64 var, unsigned long pos)
+{
+ /* Consider only the valuable positions */
+ var &= GENMASK_ULL(BITS_PER_LONG_LONG - 1, pos);
+
+ return find_first_bit64(var);
+}
+
+/*===========================================================================
+ * Pingpong algorithm functions definition
+ *===========================================================================*/
+
+/*
+ * Iterate Doorbell PingPong algorithm work thread
+ * This function clears the currently set doorbell bit, which has been
+ * unmasked before, and masks it back. Then method sets the next doorbell
+ * bit and locally unmasks it.
+ */
+static void pp_iterate_cycle(struct work_struct *work)
+{
+ struct pp_ctx *ctx = to_ctx_dwork(work);
+ struct ntb_dev *ntb = ctx->ntb;
+ u64 db_umsk, db_sts;
+ unsigned long db_id;
+ int ret;
+
+ /* Read the mask of the current disposition */
+ db_umsk = ~ntb_db_read_mask(ntb) & ctx->valid_ids;
+ if (1 != hweight64(db_umsk)) {
+ dev_err_pp(ctx,
+ "Got invalid doorbells mask %#018llx", db_umsk);
+ return;
+ }
+
+ /* Read the currently set doorbells */
+ db_sts = ntb_db_read(ntb);
+ if (0x0 == (db_sts & db_umsk)) {
+ dev_err_pp(ctx, "Got driver bug %#018llx & %#018llx == 0",
+ db_sts, db_umsk);
+ return;
+ }
+
+ /* Find the doorbell id (use db_umsk since db_sts can have several
+ * bits set) */
+ db_id = find_first_bit64(db_umsk);
+
+ dev_dbg_pp(ctx, "PingPong the doorbell bit %lu of cycle %llu",
+ db_id, ctx->cycle);
+
+ /* Mask the currently unmasked doorbell */
+ ret = ntb_db_set_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx, "Failed to mask db %lu by %#018llx",
+ db_id, db_umsk);
+ return;
+ }
+
+ /* Clear the currently set doorbell */
+ ret = ntb_db_clear(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to clear the db bit %lu", db_id);
+ return;
+ }
+
+ /* Iterate the doorbell id to set the next doorbell bit */
+ db_id = find_next_bit64(ctx->valid_ids, db_id + 1);
+ if (BITS_PER_LONG_LONG == db_id) {
+ db_id = find_first_bit64(ctx->valid_ids);
+ ctx->cycle++;
+ }
+
+ /* Calculate the new unmasking field */
+ db_umsk = BIT_ULL(db_id);
+
+ /* Set the new peer doorbell bit */
+ ret = ntb_peer_db_set(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to set the peer doorbell %lu by field %#018llx",
+ db_id, db_umsk);
+ return;
+ }
+
+ /* After this the driver is waiting for the peer response */
+ ctx->state = PP_WAIT;
+
+ /* Unmask the corresponding doorbell bit to receive the event */
+ ret = ntb_db_clear_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to unmask the doorbell %lu by field %#018llx",
+ db_id, db_umsk);
+ return;
+ }
+}
+
+/*
+ * Handle the event of Doorbell set
+ */
+static void pp_db_event(void *data, int vec)
+{
+ struct pp_ctx *ctx = data;
+
+ /* From now the driver is sleeping before sending the response */
+ ctx->state = PP_SLEEP;
+
+ /* Schedule the delayed work of the algorithm */
+ (void)schedule_delayed_work(&ctx->dwork, ctx->delay);
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t pp_dbgfs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp);
+
+/*
+ * Driver DebugFS operations
+ */
+static const struct file_operations pp_dbgfs_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = pp_dbgfs_read
+};
+
+/*
+ * DebugFS read node info callback
+ */
+static ssize_t pp_dbgfs_read(struct file *filp, char __user *ubuf,
+ size_t count, loff_t *offp)
+{
+ struct pp_ctx *ctx = filp->private_data;
+ struct ntb_dev *ntb = ctx->ntb;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, count, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Doorbells PingPong test driver:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "Link state\t- %s\n",
+ (ON == ntb_link_is_up(ntb, NULL, NULL)) ? "Up" : "Down");
+ off += scnprintf(strbuf + off, size - off,
+ "Cycle\t\t- %llu\n", ctx->cycle);
+ off += scnprintf(strbuf + off, size - off,
+ "Algo state\t- %s\n",
+ (PP_SLEEP == ctx->state) ? "sleep" : "wait");
+ off += scnprintf(strbuf + off, size - off,
+ "Delay\t\t- %u ms\n", delay_ms);
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, count, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * Driver DebugFS initialization function
+ */
+static int pp_init_dbgfs(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ const char *devname;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_dir)) {
+ dev_warn_pp(ctx,
+ "Top DebugFS directory has not been created for "
+ DRIVER_NAME);
+ return PTR_ERR(dbgfs_dir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(&ntb->dev);
+
+ /* Create the corresponding file node */
+ ctx->dbgfs_info = debugfs_create_file(devname, S_IRUSR,
+ dbgfs_dir, ctx, &pp_dbgfs_ops);
+ if (IS_ERR(ctx->dbgfs_info)) {
+ dev_err_pp(ctx, "Could not create the DebugFS node %s",
+ devname);
+ return PTR_ERR(ctx->dbgfs_info);
+ }
+
+ dev_dbg_pp(ctx, "Doorbell PingPong DebugFS node is created for %s",
+ devname);
+
+ return SUCCESS;
+}
+
+/*
+ * Driver DebugFS deinitialization function
+ */
+static void pp_deinit_dbgfs(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Remove the DebugFS file */
+ debugfs_remove(ctx->dbgfs_info);
+
+ dev_dbg_pp(ctx, "Doorbell PingPong DebugFS node %s is discarded",
+ dev_name(&ntb->dev));
+}
+
+/*===========================================================================
+ * NTB device/client driver initialization
+ *===========================================================================*/
+
+/*
+ * NTB device events handlers
+ */
+static const struct ntb_ctx_ops pp_ops = {
+ .db_event = pp_db_event
+};
+
+/*
+ * Create the driver context structure
+ */
+static struct pp_ctx *pp_create_ctx(struct ntb_dev *ntb)
+{
+ struct pp_ctx *ctx;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&ntb->dev);
+ ctx = kzalloc_node(sizeof(*ctx), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx)) {
+ dev_err(&ntb->dev,
+ "No memory for NTB PingPong driver context");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Initialize the NTB device descriptor and delayed work */
+ ctx->ntb = ntb;
+ ctx->cycle = 0;
+ ctx->valid_ids = ntb_db_valid_mask(ntb);
+ ctx->delay = msecs_to_jiffies(delay_ms);
+ ctx->state = PP_WAIT;
+ INIT_DELAYED_WORK(&ctx->dwork, pp_iterate_cycle);
+
+ dev_dbg_pp(ctx, "Context structure is created");
+
+ return ctx;
+}
+
+/*
+ * Free the driver context structure
+ */
+static void pp_free_ctx(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Just free the memory allocated for the context structure */
+ kfree(ctx);
+
+ dev_dbg(&ntb->dev, "Context structure is freed");
+}
+
+/*
+ * Correspondingly initialize the ntb device structure
+ */
+static int pp_init_ntb_dev(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ int ret;
+
+ /* Set the NTB device events context */
+ ret = ntb_set_ctx(ntb, ctx, &pp_ops);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx, "Failed to specify the NTB device context");
+ return ret;
+ }
+
+ /* Enable the link */
+ ntb_link_enable(ntb, NTB_SPEED_AUTO, NTB_WIDTH_AUTO);
+ /*ntb_link_event(ntb);*/
+
+ dev_dbg_pp(ctx, "NTB device is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the ntb device structure
+ */
+static void pp_stop_ntb_dev(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Disable the link */
+ ntb_link_disable(ntb);
+
+ /* Clear the context to make sure there won't be any doorbell event */
+ ntb_clear_ctx(ntb);
+
+ dev_dbg_pp(ctx, "NTB device is deinitialized");
+}
+
+/*
+ * Initialize the basic algorithm-related fields
+ */
+static int pp_init_algo(struct pp_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ u64 db_sts, db_umsk;
+ int ret;
+
+ /* Read the current mask */
+ db_umsk = ~ntb_db_read_mask(ntb) & ctx->valid_ids;
+
+ /* If all doorbell have been unmasked then mask them all */
+ if (db_umsk == ctx->valid_ids) {
+ ret = ntb_db_set_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to mask all the doorbells "
+ "%#018llx", db_umsk);
+ return ret;
+ }
+ /* Set the unmasking variable to zero so the algorithm would
+ * initialize the corresponding DB bit */
+ db_umsk = 0;
+ }
+
+ /* If there is no any unmasked bit then set the very first peer doorbell
+ * bit and locally unmask it */
+ if (0x0 == db_umsk) {
+ db_umsk = BIT_ULL(0);
+ /* Set the new peer doorbell bit */
+ ret = ntb_peer_db_set(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to set the peer doorbell %u by field "
+ "%#018llx", 0, db_umsk);
+ return ret;
+ }
+ /* Clear the mask of the corresponding doorbell bit */
+ ret = ntb_db_clear_mask(ntb, db_umsk);
+ if (SUCCESS != ret) {
+ dev_err_pp(ctx,
+ "Failed to unmask the doorbell %u by field "
+ "%#018llx", 0, db_umsk);
+ return ret;
+ }
+ }
+ /* If there is one umasked bit then just read the doorbell status.
+ * If the bit is set then just start the work thread to handle the
+ * disposition otherwise just don't do anything waiting for the peer
+ * to set the doorbell bit */
+ else if (1 == hweight64(db_umsk)) {
+ db_sts = ntb_db_read(ntb);
+ if (0x0 != (db_sts & db_umsk)) {
+ /* Schedule the delayed work of the algorithm */
+ (void)schedule_delayed_work(&ctx->dwork, ctx->delay);
+ }
+ } else /* if (1 < hweight64(db_umsk)) */ {
+ dev_err_pp(ctx, "Invalid mask is found %#018llx", db_umsk);
+ return -EINVAL;
+ }
+
+ dev_dbg_pp(ctx, "Doorbell PingPong algorithm is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Stop the driver algorithm
+ */
+static void pp_stop_algo(struct pp_ctx *ctx)
+{
+ /* Make sure the delayed work is not started */
+ cancel_delayed_work_sync(&ctx->dwork);
+
+ dev_dbg_pp(ctx, "Doorbell PingPong algorithm is stopped");
+}
+
+/*
+ * NTB device probe() callback function
+ */
+static int pp_probe(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct pp_ctx *ctx;
+ int ret;
+
+ /* Both synchronous and asynchronous hardware is supported */
+ if (!ntb_valid_sync_dev_ops(ntb) && !ntb_valid_async_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
+ /* Create the current device context */
+ ctx = pp_create_ctx(ntb);
+ if (IS_ERR_OR_NULL(ctx)) {
+ return PTR_ERR(ctx);
+ }
+
+ /* Initialize the NTB device */
+ ret = pp_init_ntb_dev(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_ctx;
+ }
+
+ /* Initialize the pingpong algorithm */
+ ret = pp_init_algo(ctx);
+ if (SUCCESS != ret) {
+ goto err_stop_ntb_dev;
+ }
+
+ /* Create the DebugFS node */
+ (void)pp_init_dbgfs(ctx);
+
+ /* Start */
+
+ return SUCCESS;
+
+/*err_stop_algo:
+ pp_stop_algo(ctx);
+*/
+err_stop_ntb_dev:
+ pp_stop_ntb_dev(ctx);
+
+err_free_ctx:
+ pp_free_ctx(ctx);
+
+ return ret;
+}
+
+/*
+ * NTB device remove() callback function
+ */
+static void pp_remove(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct pp_ctx *ctx = ntb->ctx;
+
+ /* Remove the DebugFS node */
+ pp_deinit_dbgfs(ctx);
+
+ /* Disable the NTB device link and clear the context */
+ pp_stop_ntb_dev(ctx);
+
+ /* Stop the algorithm */
+ pp_stop_algo(ctx);
+
+ /* Free the allocated context */
+ pp_free_ctx(ctx);
+}
+
+/*
+ * NTB bus client driver structure definition
+ */
+static struct ntb_client pp_client = {
+ .ops = {
+ .probe = pp_probe,
+ .remove = pp_remove,
+ },
+};
+/* module_ntb_client(pp_client); */
+
+/*
+ * Driver initialize method
+ */
+static int __init ntb_pp_init(void)
+{
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_dir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Registers the client driver */
+ return ntb_register_client(&pp_client);
+}
+module_init(ntb_pp_init);
+
+/*
+ * Driver exit method
+ */
+static void __exit ntb_pp_exit(void)
+{
+ /* Unregister the client driver */
+ ntb_unregister_client(&pp_client);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_dir);
+}
+module_exit(ntb_pp_exit);
+
diff --git a/drivers/ntb/test/ntb_msg_test.c b/drivers/ntb/test/ntb_msg_test.c
new file mode 100644
index 0000000..a4aecf4
--- /dev/null
+++ b/drivers/ntb/test/ntb_msg_test.c
@@ -0,0 +1,736 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * PCIe NTB messaging test Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the NTB Messaging driver design.
+ * The driver is designed to implement the simple transmition/reception
+ * algorithm. User can send data to a peer by writing it to
+ * debugfs:ntb_msg_test/ntbA_/data file, and one can read it by reading the
+ * same file on the opposite side.
+ */
+
+/* Note: You can load this module with either option 'dyndbg=+p' or define the
+ * next preprocessor constant */
+/*#define DEBUG*/
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+
+#include <linux/ntb.h>
+
+#define DRIVER_NAME "ntb_msg_test"
+#define DRIVER_DESCRIPTION "PCIe NTB Simple Messaging Client"
+#define DRIVER_VERSION "1.0"
+#define CACHE_NAME "ntb_msg_cache"
+
+MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_topdir;
+
+/*
+ * Doorbells pingpong driver context
+ * @ntb: Pointer to the NTB device
+ * @msg_cache: Messages wrapper slab
+ * @msg_lock: Spin lock to synchrnize access to the messages list
+ * @msg_list: List of received messages
+ * @msgcnt: Number of received messages
+ * @failed: Number of failed transfers
+ * @succeeded: Number of succeeded transfers
+ * @datasize: Maximum size of message data (in bytes) excluding the size byte
+ * @dbgfs_dir: Handler of the driver DebugFS directory
+ */
+struct msg_ctx {
+ struct ntb_dev *ntb;
+ struct kmem_cache *msg_cache;
+ spinlock_t msg_lock;
+ struct list_head msg_list;
+ unsigned long msgcnt;
+ unsigned long failed;
+ unsigned long succeeded;
+ size_t datasize;
+ struct dentry *dbgfs_dir;
+};
+
+/*
+ * Received messages container
+ * @msg: Message
+ * @entry: List entry
+ */
+struct ntb_msg_wrap {
+ struct ntb_msg msg;
+ struct list_head entry;
+};
+
+/*
+ * Message converter is used to translate the struct ntb_msg to the
+ * char sized data structure with size.
+ * @size: Size of the data
+ * @data: Pointer to the data buffer
+ */
+struct ntb_msg_conv {
+ u8 size;
+ char data[];
+};
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_msg(ctx, args...) \
+ dev_err(&ctx->ntb->dev, ## args)
+#define dev_warn_msg(ctx, args...) \
+ dev_warn(&ctx->ntb->dev, ## args)
+#define dev_info_msg(ctx, args...) \
+ dev_info(&ctx->ntb->dev, ## args)
+#define dev_dbg_msg(ctx, args...) \
+ dev_dbg(&ctx->ntb->dev, ## args)
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+
+/*===========================================================================
+ * Incoming messages handlers
+ *===========================================================================*/
+
+/*
+ * Save the receive message
+ */
+static void msg_recv_handler(struct msg_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct ntb_msg_wrap *wrap;
+ struct ntb_msg_conv *conv;
+
+ /* Cast the message to the converted one */
+ conv = (struct ntb_msg_conv *)msg;
+
+ /* Allocate the memory from the slab */
+ wrap = kmem_cache_alloc(ctx->msg_cache, GFP_KERNEL);
+ if (NULL == wrap) {
+ dev_err_msg(ctx,
+ "Failed to allocate memory for incoming message %.*s",
+ conv->size, conv->data);
+ return;
+ }
+
+ /* Copy the message to the buffer */
+ memcpy(&wrap->msg, msg, conv->size + 1);
+
+ /* Add the wrapped message to the list of received messages */
+ spin_lock(&ctx->msg_lock);
+ list_add_tail(&wrap->entry, &ctx->msg_list);
+ /* Increment the number of received messages in the buffer */
+ ctx->msgcnt++;
+ spin_unlock(&ctx->msg_lock);
+
+ dev_dbg_msg(ctx, "Message '%.*s' was received",
+ conv->size, conv->data);
+}
+
+/*
+ * Handler of the transmit errors
+ */
+static void msg_fail_handler(struct msg_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct ntb_msg_conv *conv = (struct ntb_msg_conv *)msg;
+
+ /* Just print the error increment the errors counter */
+ dev_err_msg(ctx,
+ "Failed to send the submessage '%.*s'",
+ conv->size, conv->data);
+ ctx->failed++;
+}
+
+/*
+ * Handler of the succeeded transmits
+ */
+static void msg_sent_handler(struct msg_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct ntb_msg_conv *conv = (struct ntb_msg_conv *)msg;
+
+ /* Just print the debug text and increment the succeeded msgs counter */
+ dev_dbg_msg(ctx,
+ "Submessage '%.*s' has been successfully sent",
+ conv->size, conv->data);
+ ctx->succeeded++;
+}
+
+/*
+ * Message event handler
+ */
+static void msg_event_handler(void *data, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg)
+{
+ struct msg_ctx *ctx = data;
+
+ /* Call the corresponding event handler */
+ switch (ev) {
+ case NTB_MSG_NEW:
+ msg_recv_handler(ctx, msg);
+ break;
+ case NTB_MSG_SENT:
+ msg_sent_handler(ctx, msg);
+ break;
+ case NTB_MSG_FAIL:
+ msg_fail_handler(ctx, msg);
+ break;
+ default:
+ dev_err_msg(ctx, "Got invalid message event %d", ev);
+ break;
+ }
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t msg_dbgfs_data_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t msg_dbgfs_data_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t msg_dbgfs_stat_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+/*
+ * DebugFS data node operations
+ */
+static const struct file_operations msg_dbgfs_data_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = msg_dbgfs_data_read,
+ .write = msg_dbgfs_data_write
+};
+
+/*
+ * DebugFS statistics node operations
+ */
+static const struct file_operations msg_dbgfs_stat_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = msg_dbgfs_stat_read,
+};
+
+/*
+ * DebugFS callback of read messages node
+ */
+static ssize_t msg_dbgfs_data_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct msg_ctx *ctx = filep->private_data;
+ struct list_head *entry, *safe_entry;
+ struct ntb_msg_wrap *wrap;
+ struct ntb_msg_conv *conv;
+ size_t datasize, retsize;
+ char *databuf;
+ ssize_t ret;
+
+ /* Find the size of the retrieved data in messages */
+ datasize = 0;
+ spin_lock(&ctx->msg_lock);
+ list_for_each(entry, &ctx->msg_list) {
+ wrap = list_entry(entry, struct ntb_msg_wrap, entry);
+ conv = (struct ntb_msg_conv *)&wrap->msg;
+ datasize += conv->size;
+ }
+ spin_unlock(&ctx->msg_lock);
+
+ /* Calculate the size of the output buffer */
+ datasize = min(datasize, usize);
+
+ /* Allocate the buffer */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_msg(ctx, "No memory to allocate the output buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data from the messages to the output buffer */
+ retsize = 0;
+ spin_lock(&ctx->msg_lock);
+ list_for_each_safe(entry, safe_entry, &ctx->msg_list) {
+ /* Get the message and copy it to the buffer */
+ wrap = list_entry(entry, struct ntb_msg_wrap, entry);
+ conv = (struct ntb_msg_conv *)&wrap->msg;
+
+ /* If there is no enough space left in the buffer then stop the
+ * loop */
+ if ((datasize - retsize) < conv->size) {
+ break;
+ }
+
+ /* Copy the data to the output buffer */
+ memcpy(&databuf[retsize], conv->data, conv->size);
+
+ /* Increment the size of the retrieved data */
+ retsize += conv->size;
+
+ /* Delete the list entry and free the memory */
+ list_del(&wrap->entry);
+ kmem_cache_free(ctx->msg_cache, wrap);
+
+ /* Decrement the number of messages in the buffer */
+ ctx->msgcnt--;
+ }
+ spin_unlock(&ctx->msg_lock);
+
+ /* Copy the text to the output buffer */
+ ret = simple_read_from_buffer(ubuf, usize, offp, databuf, retsize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS callback of write messages node
+ */
+static ssize_t msg_dbgfs_data_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct msg_ctx *ctx = filep->private_data;
+ struct ntb_dev *ntb = ctx->ntb;
+ struct ntb_msg msg;
+ struct ntb_msg_conv *conv;
+ char *databuf;
+ int pos, copied, sts = SUCCESS;
+ ssize_t ret;
+
+ /* Allocate the memory for sending data */
+ databuf = kmalloc(usize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_msg(ctx, "No memory to allocate the sending data buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data to the output buffer */
+ ret = simple_write_to_buffer(databuf, usize, offp, ubuf, usize);
+ if (0 > ret) {
+ dev_err_msg(ctx, "Failed to copy the data from the User-space");
+ kfree(databuf);
+ return ret;
+ }
+
+ /* Start copying data to the message structure and send it straight away
+ * to the peer */
+ conv = (struct ntb_msg_conv *)&msg;
+ for (pos = 0, copied = 0; pos < usize; pos += copied) {
+ /* Calculate the size of data to copy to the message */
+ copied = min(ctx->datasize, (usize - pos));
+ /* Set the data size of the message */
+ conv->size = copied;
+ /* Copy the data */
+ memcpy(conv->data, &databuf[pos], copied);
+
+ /* Send the data stright away */
+ sts = ntb_msg_post(ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_msg(ctx, "Failed to post the submessage %.*s",
+ copied, conv->data);
+ }
+ }
+
+ return (SUCCESS == sts) ? usize : -EINVAL;
+}
+
+/*
+ * DebugFS callback to read statistics
+ */
+static ssize_t msg_dbgfs_stat_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct msg_ctx *ctx = filep->private_data;
+ struct ntb_dev *ntb = ctx->ntb;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, usize, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ dev_dbg_msg(ctx,
+ "Failed to allocate the memory for statistics "
+ "output buffer");
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Messaging Test driver:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "Link state\t\t- %s\n",
+ (ON == ntb_link_is_up(ntb, NULL, NULL)) ? "Up" : "Down");
+ off += scnprintf(strbuf + off, size - off,
+ "Message count\t\t- %lu\n", ctx->msgcnt);
+ off += scnprintf(strbuf + off, size - off,
+ "Message size\t\t- %u\n", ntb_msg_size(ntb));
+ off += scnprintf(strbuf + off, size - off,
+ "Data size\t\t- %lu\n", (unsigned long)ctx->datasize);
+ off += scnprintf(strbuf + off, size - off,
+ "Successfully sent\t- %lu\n", ctx->succeeded);
+ off += scnprintf(strbuf + off, size - off,
+ "Failed to send\t\t- %lu\n", ctx->failed);
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, usize, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS initialization function
+ */
+static int msg_init_dbgfs(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct dentry *dbgfs_data, *dbgfs_stat;
+ const char *devname;
+ int ret;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_topdir)) {
+ dev_warn_msg(ctx,
+ "Top DebugFS directory has not been created for "
+ DRIVER_NAME);
+ return PTR_ERR(dbgfs_topdir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(&ntb->dev);
+
+ /* Create the device related subdirectory */
+ ctx->dbgfs_dir = debugfs_create_dir(devname, dbgfs_topdir);
+ if (IS_ERR_OR_NULL(ctx->dbgfs_dir)) {
+ dev_warn_msg(ctx,
+ "Failed to create the DebugFS subdirectory %s",
+ devname);
+ return PTR_ERR(ctx->dbgfs_dir);
+ }
+
+ /* Create the file node for data io operations */
+ dbgfs_data = debugfs_create_file("data", S_IRWXU, ctx->dbgfs_dir, ctx,
+ &msg_dbgfs_data_ops);
+ if (IS_ERR(dbgfs_data)) {
+ dev_err_msg(ctx, "Could not create DebugFS data node");
+ ret = PTR_ERR(dbgfs_data);
+ goto err_rm_dir;
+ }
+
+ /* Create the file node for statistics io operations */
+ dbgfs_stat = debugfs_create_file("stat", S_IRWXU, ctx->dbgfs_dir, ctx,
+ &msg_dbgfs_stat_ops);
+ if (IS_ERR(dbgfs_stat)) {
+ dev_err_msg(ctx, "Could not create DebugFS statistics node");
+ ret = PTR_ERR(dbgfs_stat);
+ goto err_rm_dir;
+ }
+
+ dev_dbg_msg(ctx, "NTB Messaging DebugFS nodes are created for %s",
+ devname);
+
+ return SUCCESS;
+
+err_rm_dir:
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ return ret;
+}
+
+/*
+ * DebugFS deinitialization function
+ */
+static void msg_deinit_dbgfs(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Remove the DebugFS directory */
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ dev_dbg_msg(ctx, "NTB Messaging DebugFS nodes %s/ are discarded",
+ dev_name(&ntb->dev));
+}
+
+/*===========================================================================
+ * NTB device/client driver initialization
+ *===========================================================================*/
+
+/*
+ * NTB device events handlers
+ */
+static const struct ntb_ctx_ops msg_ops = {
+ .msg_event = msg_event_handler
+};
+
+/*
+ * Create the driver context structure
+ */
+static struct msg_ctx *msg_create_ctx(struct ntb_dev *ntb)
+{
+ struct msg_ctx *ctx;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&ntb->dev);
+ ctx = kzalloc_node(sizeof(*ctx), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx)) {
+ dev_err(&ntb->dev,
+ "No memory for NTB Messaging driver context");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Create the message cache */
+ ctx->msg_cache = kmem_cache_create(CACHE_NAME,
+ sizeof(struct ntb_msg_wrap), 0, 0, NULL);
+ if (NULL == ctx->msg_cache) {
+ dev_err(&ntb->dev,
+ "Failed to allocate the message wrap structures cache");
+ kfree(ctx);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Initialize the context NTB device pointer */
+ ctx->ntb = ntb;
+
+ /* Initialize the message list lock and the list head */
+ spin_lock_init(&ctx->msg_lock);
+ INIT_LIST_HEAD(&ctx->msg_list);
+
+ /* Initialize the counters */
+ ctx->msgcnt = 0;
+ ctx->failed = 0;
+ ctx->succeeded = 0;
+
+ /* Initialize the data size of one message excluding the size byte */
+ ctx->datasize = 4*ntb_msg_size(ntb) - 1;
+
+ dev_dbg_msg(ctx, "Context structure is created");
+
+ return ctx;
+}
+
+/*
+ * Free the driver context structure
+ */
+static void msg_free_ctx(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct ntb_msg_wrap *wrap;
+ struct list_head *entry, *safe_entry;
+
+ /* Walk through the list of messages and destroy all the allocated
+ * memory */
+ spin_lock(&ctx->msg_lock);
+ list_for_each_safe(entry, safe_entry, &ctx->msg_list) {
+ /* Get the message wrapper */
+ wrap = list_entry(entry, struct ntb_msg_wrap, entry);
+
+ /* Delete the list entry and free the memory */
+ list_del(entry);
+ kmem_cache_free(ctx->msg_cache, wrap);
+
+ /* Decrement the number of messages in the buffer */
+ ctx->msgcnt--;
+ }
+ spin_unlock(&ctx->msg_lock);
+
+ /* Destroy the IDT messages cache */
+ kmem_cache_destroy(ctx->msg_cache);
+
+ /* Free the memory allocated for the context structure */
+ kfree(ctx);
+
+ dev_dbg(&ntb->dev, "Context structure is freed");
+}
+
+/*
+ * Initialize the ntb device structure
+ */
+static int msg_init_ntb_dev(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ int ret;
+
+ /* Set the NTB device events context */
+ ret = ntb_set_ctx(ntb, ctx, &msg_ops);
+ if (SUCCESS != ret) {
+ dev_err_msg(ctx, "Failed to specify the NTB device context");
+ return ret;
+ }
+
+ /* Enable the link */
+ ntb_link_enable(ntb, NTB_SPEED_AUTO, NTB_WIDTH_AUTO);
+ /*ntb_link_event(ntb);*/
+
+ dev_dbg_msg(ctx, "NTB device is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the ntb device structure
+ */
+static void msg_stop_ntb_dev(struct msg_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Disable the link */
+ ntb_link_disable(ntb);
+
+ /* Clear the context */
+ ntb_clear_ctx(ntb);
+
+ dev_dbg_msg(ctx, "NTB device is deinitialized");
+}
+
+/*
+ * NTB device probe() callback function
+ */
+static int msg_probe(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct msg_ctx *ctx;
+ int ret;
+
+ /* Only asynchronous hardware is supported */
+ if (!ntb_valid_async_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
+ /* Create the current device context */
+ ctx = msg_create_ctx(ntb);
+ if (IS_ERR_OR_NULL(ctx)) {
+ return PTR_ERR(ctx);
+ }
+
+ /* Initialize the NTB device */
+ ret = msg_init_ntb_dev(ctx);
+ if (SUCCESS != ret) {
+ msg_free_ctx(ctx);
+ return ret;
+ }
+
+ /* Create the DebugFS node */
+ (void)msg_init_dbgfs(ctx);
+
+ return SUCCESS;
+}
+
+/*
+ * NTB device remove() callback function
+ */
+static void msg_remove(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct msg_ctx *ctx = ntb->ctx;
+
+ /* Remove the DebugFS node */
+ msg_deinit_dbgfs(ctx);
+
+ /* Disable the NTB device link and clear the context */
+ msg_stop_ntb_dev(ctx);
+
+ /* Free the allocated context */
+ msg_free_ctx(ctx);
+}
+
+/*
+ * NTB bus client driver structure definition
+ */
+static struct ntb_client msg_client = {
+ .ops = {
+ .probe = msg_probe,
+ .remove = msg_remove,
+ },
+};
+/* module_ntb_client(msg_client); */
+
+/*
+ * Driver initialize method
+ */
+static int __init ntb_msg_init(void)
+{
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_topdir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Registers the client driver */
+ return ntb_register_client(&msg_client);
+}
+module_init(ntb_msg_init);
+
+/*
+ * Driver exit method
+ */
+static void __exit ntb_msg_exit(void)
+{
+ /* Unregister the client driver */
+ ntb_unregister_client(&msg_client);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_topdir);
+}
+module_exit(ntb_msg_exit);
+
diff --git a/drivers/ntb/test/ntb_mw_test.c b/drivers/ntb/test/ntb_mw_test.c
new file mode 100644
index 0000000..97dbfc9
--- /dev/null
+++ b/drivers/ntb/test/ntb_mw_test.c
@@ -0,0 +1,1539 @@
+/*
+ * This file is provided under a GPLv2 license. When using or
+ * redistributing this file, you may do so under that license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright (C) 2016 T-Platforms All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; if not, one can be found <http://www.gnu.org/licenses/>.
+ *
+ * The full GNU General Public License is included in this distribution in
+ * the file called "COPYING".
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * PCIe NTB memory windows test Linux driver
+ *
+ * Contact Information:
+ * Serge Semin <fancer.lancer@gmail.com>, <Sergey.Semin@t-platforms.ru>
+ */
+
+/*
+ * NOTE of the NTB memory windows test driver design.
+ * The driver implements the simple read/write algorithm. It allocates the
+ * necessary inbound shared memory window by demand from the peer. Then it
+ * sends the physical address of the memory back to the peer. The corresponding
+ * inwndwN and outwndwN files are created at the DebugFS:ntb_mw_test/ntbA_/
+ * directory. The inwndwN file can be used to read the data written by a peer.
+ * The other outwndwN file is used to write data to the peer memory window.
+ */
+
+/* Note: You can load this module with either option 'dyndbg=+p' or define the
+ * next preprocessor constant */
+/*#define DEBUG*/
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/types.h>
+#include <linux/dma-mapping.h>
+
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/bitops.h>
+#include <linux/debugfs.h>
+
+#include <linux/ntb.h>
+
+#define DRIVER_NAME "ntb_mw_test"
+#define DRIVER_DESCRIPTION "PCIe NTB Memory Window Test Client"
+#define DRIVER_VERSION "1.0"
+#define CACHE_NAME "ntb_mw_cache"
+
+MODULE_DESCRIPTION(DRIVER_DESCRIPTION);
+MODULE_VERSION(DRIVER_VERSION);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("T-platforms");
+
+/*
+ * DebugFS directory to place the driver debug file
+ */
+static struct dentry *dbgfs_topdir;
+
+/*
+ * Inbound memory windows (locally allocated) structure
+ * @dma_addr: Address if the locally allocated memory and sent to the peer
+ * @virt_addr: Virtual address of that memory
+ * @size: Size of the allocated memory
+ * @addr_align: Address alignment
+ * @size_align: Size alignment
+ * @size_max: Maximum possible size of the window
+ * @dbgfs_node: DebugFS node to read data from peer
+ * @ctx: Pointer to the driver context
+ */
+struct mw_ctx;
+struct inmw_wrap {
+ dma_addr_t dma_addr;
+ void *virt_addr;
+ resource_size_t size;
+ resource_size_t addr_align;
+ resource_size_t size_align;
+ resource_size_t size_max;
+ struct dentry *dbgfs_node;
+ struct mw_ctx *ctx;
+};
+
+/*
+ * Outbound memory windows (remotely allocated) structure
+ * @enabled: Flag whether the window is enabled
+ * @dma_addr: DMA address of the remotely allocated memory window and
+ * retrieved from the peer
+ * @phys_addr: Physical address of the memory to locally map it (retrieved
+ * from the NTB subsystem, shortly it must be from BAR2 of IDT)
+ * @virt_addr: Virtual address of mapped IOMEM physical address
+ * @size: Size of the peer allocated memory
+ * @addr_align: Alignment of the DMA address allocated by the peer
+ * @size_align: Size alignment of the DMA address allocated by the peer
+ * @size_max: Maximum size of the peer allocated memory
+ * @dbgfs_node: DebugFS node to write data to peer
+ * @ctx: Pointer to the driver context
+ */
+struct outmw_wrap {
+ bool enabled;
+ dma_addr_t dma_addr;
+ phys_addr_t phys_addr;
+ void __iomem *virt_addr;
+ resource_size_t size;
+ resource_size_t addr_align;
+ resource_size_t size_align;
+ resource_size_t size_max;
+ struct dentry *dbgfs_node;
+ struct mw_ctx *ctx;
+};
+
+/*
+ * Doorbells pingpong driver context
+ * @ntb: Pointer to the NTB device
+ * @inmw_cnt: Number of possible inbound memory windows
+ * @outmw_cnt: Number of possible outbound memory windows
+ * @dbgfs_dir: Handler of the DebugFS driver info-file
+ */
+struct mw_ctx {
+ struct ntb_dev *ntb;
+ int inmws_cnt;
+ struct inmw_wrap *inmws;
+ int outmws_cnt;
+ struct outmw_wrap *outmws;
+ struct dentry *dbgfs_dir;
+};
+
+/*
+ * Enumeration of commands
+ * @MW_GETADDRS: Get the addresses of all memory windows peer allocated
+ * @MW_DMAADDR: DMA address of the memory window is sent within this msg
+ * @MW_FREEADDRS: Lock the memory windows shared from the local device
+ * @MW_TYPEMASK: Mask of the message type
+ */
+enum msg_type {
+ MW_GETADDRS,
+ MW_DMAADDR,
+ MW_FREEADDRS,
+ MW_TYPEMASK = 0xFFFFU
+};
+
+/*
+* Helper method to get the type string name
+*/
+static inline char *mw_get_typename(enum msg_type type)
+{
+ switch (type) {
+ case MW_GETADDRS:
+ return "GETADDRS";
+ case MW_DMAADDR:
+ return "DMAADDR";
+ case MW_FREEADDRS:
+ return "FREEADDRS";
+ default:
+ break;
+ }
+
+ return "INVALID";
+}
+
+/*
+ * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
+ */
+#define dev_err_mw(ctx, args...) \
+ dev_err(&ctx->ntb->dev, ## args)
+#define dev_warn_mw(ctx, args...) \
+ dev_warn(&ctx->ntb->dev, ## args)
+#define dev_info_mw(ctx, args...) \
+ dev_info(&ctx->ntb->dev, ## args)
+#define dev_dbg_mw(ctx, args...) \
+ dev_dbg(&ctx->ntb->dev, ## args)
+
+/*
+ * Some common constant used in the driver for better readability:
+ * @ON: Enable something
+ * @OFF: Disable something
+ * @SUCCESS: Success of a function execution
+ * @MIN_MW_CNT: Minimum memory windows count
+ * @MAX_MW_CNT: Maximum memory windows count
+ */
+#define ON ((u32)0x1)
+#define OFF ((u32)0x0)
+#define SUCCESS 0
+#define MIN_MW_CNT ((unsigned char)1)
+#define MAX_MW_CNT ((unsigned char)255)
+
+/*
+ * Shared data converter to support the different CPU architectures
+ */
+#define to_sh32(data) \
+ cpu_to_le32((data))
+#define from_sh32(data) \
+ le32_to_cpu((data))
+
+/*
+ * Cast DMA address to real address pointer
+ *
+ * NOTE It's used in the printf's to get rid of warnings
+ */
+#define CAST_DMA_PTR(addr) \
+ ((void *)(phys_addr_t)(addr))
+
+/*
+ * Module parameters:
+ * @inmw_cnt: Number of inbound memory windows [1; 255]
+ * @outmw_cnt: Number of outbound memory windows [1; 255]
+ * If the specified value exceeds the maximum possible valiue, then it is
+ * initialized with maximum one
+ */
+static unsigned char inmws_cnt = MAX_MW_CNT;
+module_param(inmws_cnt, byte, 0000);
+MODULE_PARM_DESC(inmws_cnt,
+ "Inbound memory windows count. Those are the memory windows, which are "
+ "locally allocated. Their address is sent to the remote host."
+ " - Parameter can be set within [1; 255], where 255 means maximum possible"
+ " number of windows");
+
+/*===========================================================================
+ * Helper methods
+ *===========================================================================*/
+
+/*
+ * Alter the passed driver paremeters
+ */
+static void mw_alter_params(struct mw_ctx *ctx)
+{
+ unsigned char inmws_cnt_bak = ctx->inmws_cnt;
+
+ /* Clamp the inbound memory windows parameter */
+ ctx->inmws_cnt = clamp(inmws_cnt,
+ MIN_MW_CNT, (unsigned char)ctx->inmws_cnt);
+ if (inmws_cnt_bak != ctx->inmws_cnt) {
+ dev_warn_mw(ctx,
+ "Inbound memory windows count is altered from "
+ "%hhu to %hhu", inmws_cnt_bak, ctx->inmws_cnt);
+ }
+
+ dev_dbg_mw(ctx, "Memory windows test driver parameter is verified");
+}
+
+/*
+ * Memory block IO write method
+ */
+static void iomem_write(void __iomem *dst, const void *src, size_t cnt)
+{
+ while (cnt--) {
+ iowrite8(*(u8 *)src, dst);
+ dst++;
+ src++;
+ }
+}
+
+/*
+ * Memory block IO read method
+ */
+static void iomem_read(void __iomem *src, void *dst, size_t cnt)
+{
+ while (cnt--) {
+ *(u8 *)dst = ioread8(src);
+ dst++;
+ src++;
+ }
+}
+
+/*===========================================================================
+ * Message command handlers
+ *===========================================================================*/
+
+/*
+ * Send MW_GETADDRS command method
+ */
+static void mw_send_getaddrs_cmd(struct mw_ctx *ctx)
+{
+ struct ntb_msg msg;
+ int sts;
+
+ /* Clear the message structure */
+ memset(&msg, 0, sizeof(msg));
+
+ /* Set the message type only */
+ msg.type = to_sh32(MW_GETADDRS);
+
+ /* Send the message */
+ sts = ntb_msg_post(ctx->ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx, "Failed to send message to get outbound window "
+ "addresses");
+ }
+}
+
+/*
+ * Send MW_FREEADDRS command method
+ */
+static void mw_send_freeaddrs_cmd(struct mw_ctx *ctx)
+{
+ struct ntb_msg msg;
+ int sts;
+
+ /* Clear the message structure */
+ memset(&msg, 0, sizeof(msg));
+
+ /* Set the message type only */
+ msg.type = to_sh32(MW_FREEADDRS);
+
+ /* Send the message */
+ sts = ntb_msg_post(ctx->ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx, "Failed to send a message to disable the peer "
+ "outbound windows");
+ }
+}
+
+/*
+ * Callback method for response on the command MW_GETADDRS
+ */
+static void mw_send_inmw_addrs(struct mw_ctx *ctx)
+{
+ struct ntb_msg msg;
+ struct inmw_wrap *inmw;
+ int mwindx, sts;
+
+ /* Clear the message structure */
+ memset(&msg, 0, sizeof(msg));
+
+ /* Walk through all the inbound memory windows and send the
+ * corresponding DMA address of the window */
+ for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
+ inmw = &ctx->inmws[mwindx];
+ /* Set the type and the memory window index */
+ msg.type = to_sh32(MW_DMAADDR | ((u32)mwindx << 16));
+
+ /* First set the size of the memory window */
+ msg.payload[0] = to_sh32(inmw->size);
+
+ /* Set the Upper part of the memory window address */
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
+#else
+ /* WARNING! NTB entpoints must either have the same architecture
+ * (x32 or x64) or use lower 4Gb for memory windows */
+ msg.payload[1] = 0;
+#endif /* !CONFIG_ARCH_DMA_ADDR_T_64BIT */
+ /* Set the Lower part of the memory window address */
+ msg.payload[2] = to_sh32((u32)(inmw->dma_addr));
+
+ /* Send the message */
+ sts = ntb_msg_post(ctx->ntb, &msg);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx,
+ "Failed to send a message with window %d "
+ "address", mwindx);
+ }
+ }
+}
+
+/*
+ * Method to set the corresponding memory window and enable it
+ */
+static void mw_set_outmw_addr(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ struct outmw_wrap *outmw;
+ int mwindx, sts;
+
+ /* Read the memory windows index (it's the part of the message type) */
+ mwindx = from_sh32(msg->type) >> 16;
+ if (ctx->outmws_cnt <= mwindx) {
+ dev_err_mw(ctx,
+ "Retrieved invalid outbound memory window index %d",
+ mwindx);
+ return;
+ }
+ outmw = &ctx->outmws[mwindx];
+
+ /* Read the memory window size and check whether it has proper size and
+ * alignment */
+ outmw->size = from_sh32(msg->payload[0]);
+ if (!IS_ALIGNED(outmw->size, outmw->size_align) ||
+ outmw->size_max < outmw->size) {
+ dev_err_mw(ctx,
+ "Retrieved invalid memory window %d size %u "
+ "(max: %u, align: %u)", mwindx, (unsigned int)outmw->size,
+ (unsigned int)outmw->size_max,
+ (unsigned int)outmw->size_align);
+ return;
+ }
+
+ /* Read the DMA address, where the second DWORD is the upper part and
+ * the third DWORD - lower */
+ outmw->dma_addr = from_sh32(msg->payload[2]);
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ outmw->dma_addr |= ((dma_addr_t)from_sh32(msg->payload[1]) << 32);
+#endif /* CONFIG_ARCH_DMA_ADDR_T_64BIT */
+ /* Check whether the retrieved address is properly aligned */
+ if (!IS_ALIGNED(outmw->dma_addr, outmw->addr_align)) {
+ dev_err_mw(ctx,
+ "Outbound memory window address 0x%p is not aligned "
+ "within %lu bytes", CAST_DMA_PTR(outmw->dma_addr),
+ (unsigned long int)outmw->addr_align);
+ return;
+ }
+
+ /* Set the translation address of the outbound memory window */
+ sts = ntb_mw_set_trans(ctx->ntb, mwindx, outmw->dma_addr, outmw->size);
+ if (SUCCESS != sts) {
+ dev_err_mw(ctx, "Failed to set the translated address %p of "
+ "outbound memory window %d",
+ CAST_DMA_PTR(outmw->dma_addr), mwindx);
+ return;
+ }
+
+ /* Enable the memory window */
+ outmw->enabled = true;
+
+ dev_dbg_mw(ctx, "Outbound memory window %d is initialized with "
+ "address 0x%p", mwindx, CAST_DMA_PTR(outmw->dma_addr));
+}
+
+/*
+ * Lock all the outbound memory windows
+ */
+static void mw_lock_outmw_addrs(struct mw_ctx *ctx)
+{
+ int mwindx;
+
+ /* Walk through all the memory windows and lock whem by falsing
+ * the flag */
+ for (mwindx = 0; mwindx < ctx->outmws_cnt; mwindx++) {
+ ctx->outmws[mwindx].enabled = false;
+ }
+
+ dev_dbg_mw(ctx, "Outbound memory windows are locked");
+}
+
+/*===========================================================================
+ * Messages and link events handlers
+ *===========================================================================*/
+
+/*
+ * Handle the retrieved message
+ */
+static void msg_recv_handler(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ enum msg_type type = from_sh32(msg->type) & MW_TYPEMASK;
+
+ /* Check the message types */
+ switch (type) {
+ case MW_GETADDRS:
+ mw_send_inmw_addrs(ctx);
+ break;
+ case MW_DMAADDR:
+ mw_set_outmw_addr(ctx, msg);
+ break;
+ case MW_FREEADDRS:
+ mw_lock_outmw_addrs(ctx);
+ break;
+ default:
+ dev_err_mw(ctx, "Invalid message type retrieved %d", type);
+ return;
+ }
+
+ dev_dbg_mw(ctx, "Message of type %s was received",
+ mw_get_typename(type));
+}
+
+/*
+ * Handler of the transmit errors
+ */
+static void msg_fail_handler(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ enum msg_type type = from_sh32(msg->type) & MW_TYPEMASK;
+
+ /* Just print the error */
+ dev_err_mw(ctx, "Failed to send the message of type %s",
+ mw_get_typename(type));
+}
+
+/*
+ * Handler of the succeeded transmits
+ */
+static void msg_sent_handler(struct mw_ctx *ctx, const struct ntb_msg *msg)
+{
+ enum msg_type type = from_sh32(msg->type) & MW_TYPEMASK;
+
+ /* Just print the debug text and increment the succeeded msgs counter */
+ dev_dbg_mw(ctx, "Message of type %s has been successfully sent",
+ mw_get_typename(type));
+}
+
+/*
+ * Message event handler
+ */
+static void msg_event_handler(void *data, enum NTB_MSG_EVENT ev,
+ struct ntb_msg *msg)
+{
+ struct mw_ctx *ctx = data;
+
+ /* Call the corresponding event handler */
+ switch (ev) {
+ case NTB_MSG_NEW:
+ msg_recv_handler(ctx, msg);
+ break;
+ case NTB_MSG_SENT:
+ msg_sent_handler(ctx, msg);
+ break;
+ case NTB_MSG_FAIL:
+ msg_fail_handler(ctx, msg);
+ break;
+ default:
+ dev_err_mw(ctx, "Got invalid message event %d", ev);
+ break;
+ }
+}
+
+/*
+ * Link Up/Down event handler
+ */
+static void link_event_handler(void *data)
+{
+ struct mw_ctx *ctx = data;
+ int sts;
+
+ /* If link is up then send the message with GETADDRS command, otherwise
+ * the outbound memory windows must be disabled */
+ sts = ntb_link_is_up(ctx->ntb, NULL, NULL);
+ if (ON == sts) {
+ mw_send_getaddrs_cmd(ctx);
+ } else /* if (OFF == sts) */ {
+ mw_lock_outmw_addrs(ctx);
+ }
+
+ dev_dbg_mw(ctx, "Link %s event was retrieved",
+ ON == sts ? "Up" : "Down");
+}
+
+/*===========================================================================
+ * 11. DebugFS callback functions
+ *===========================================================================*/
+
+static ssize_t mw_dbgfs_outmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_outmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_outmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_inmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_inmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+static ssize_t mw_dbgfs_inmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp);
+
+/*
+ * DebugFS outbound memory window node operations
+ */
+static const struct file_operations mw_dbgfs_outmw_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_outmw_read,
+ .write = mw_dbgfs_outmw_write
+};
+
+/*
+ * DebugFS outbound memory window configuration node operations
+ */
+static const struct file_operations mw_dbgfs_outmw_cfg_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_outmw_cfg_read,
+};
+
+/*
+ * DebugFS inbound memory window node operations
+ */
+static const struct file_operations mw_dbgfs_inmw_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_inmw_read,
+ .write = mw_dbgfs_inmw_write
+};
+
+/*
+ * DebugFS inbound memory window configuration node operations
+ */
+static const struct file_operations mw_dbgfs_inmw_cfg_ops = {
+ .owner = THIS_MODULE,
+ .open = simple_open,
+ .read = mw_dbgfs_inmw_cfg_read,
+};
+
+/*
+ * DebugFS read callback of outbound memory window node
+ */
+static ssize_t mw_dbgfs_outmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct outmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+ int sts;
+
+ /* Check whether the link is up and the outbound window is enabled */
+ sts = ntb_link_is_up(ctx->ntb, NULL, NULL);
+ if (OFF == sts || !wrap->enabled) {
+ dev_err_mw(ctx, "NTB link is %s, memory window status is %s",
+ OFF == sts ? "Down" : "Up",
+ wrap->enabled ? "enabled" : "disabled");
+ return -ENODEV;
+ }
+
+ /* Read the first DWORD with the size of the message */
+ datasize = readl(wrap->virt_addr);
+
+ /* Check the read data size */
+ if (wrap->size < datasize) {
+ dev_err_mw(ctx, "Data size %u exceeds the memory window size %u",
+ (unsigned int)datasize, (unsigned int)wrap->size);
+ return -EINVAL;
+ }
+
+ /* Calculate the size of the output buffer */
+ datasize = min(datasize, usize);
+
+ /* If there is nothing to copy then just return from the function */
+ if (0 == datasize) {
+ return 0;
+ }
+
+ /* Allocate the buffer */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the output buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data from the shared memory to the temporary buffer */
+ /* NOTE memcpy_toio could be used instead, but it weirdly works so
+ * the traditional looping is used */
+ iomem_read((wrap->virt_addr + 4), databuf, datasize);
+ /*memcpy_fromio(databuf, wrap->virt_addr + 4, datasize);*/
+
+ /* Copy the data to the output buffer */
+ ret = simple_read_from_buffer(ubuf, usize, offp, databuf, datasize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS write callback of outbound memory window node
+ */
+static ssize_t mw_dbgfs_outmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct outmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+ int sts;
+
+ /* Check whether the link is up and the outbound window is enabled */
+ sts = ntb_link_is_up(ctx->ntb, NULL, NULL);
+ if (OFF == sts || !wrap->enabled) {
+ dev_err_mw(ctx, "NTB link is %s, memory window status is %s",
+ OFF == sts ? "Down" : "Up",
+ wrap->enabled ? "enabled" : "disabled");
+ return -ENODEV;
+ }
+
+ /* Calculate the data size */
+ datasize = min(((size_t)wrap->size - 4), usize);
+
+ /* Allocate the memory for sending data */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the input data buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data to the output buffer */
+ ret = simple_write_to_buffer(databuf, datasize, offp, ubuf, usize);
+ if (0 > ret) {
+ dev_err_mw(ctx, "Failed to copy the data from the User-space");
+ kfree(databuf);
+ return ret;
+ }
+
+ /* First DWORD is the data size */
+ writel((u32)datasize, wrap->virt_addr);
+
+ /* Copy the data to the memory window */
+ /* NOTE memcpy_toio could be used instead, but it weirdly works so
+ * the traditional looping is used */
+ iomem_write((wrap->virt_addr + 4), databuf, datasize);
+ /*memcpy_toio((wrap->virt_addr + 4), databuf, datasize);*/
+
+ /* Ensure that the data is fully copied out by setting the memory
+ * barrier */
+ wmb();
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS read callback of outbound memory window configurations
+ */
+static ssize_t mw_dbgfs_outmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct outmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+ int id;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, usize, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ dev_dbg_mw(ctx,
+ "Failed to allocated the memory for outbound memory "
+ "window configuration");
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Outbound Memory Window configuration:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "Status\t\t\t- %s\n",
+ wrap->enabled ? "enabled" : "disabled");
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address\t\t- 0x%p\n", CAST_DMA_PTR(wrap->dma_addr));
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address alignment\t- %lu\n",
+ (unsigned long int)wrap->addr_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Physycal map address\t- 0x%p\n", (void *)wrap->phys_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "Virtual map address\t- 0x%p\n", (void *)wrap->virt_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "Size of the window\t- %lu\n",
+ (unsigned long int)wrap->size);
+ off += scnprintf(strbuf + off, size - off,
+ "Size alignment\t\t- %lu\n",
+ (unsigned long int)wrap->size_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Maximum size\t\t- %lu\n",
+ (unsigned long int)wrap->size_max);
+ /* Print raw data from the inbound window */
+ off += scnprintf(strbuf + off, size - off,
+ "Raw data (16 bytes)\t- ");
+ for (id = 0; id < 16; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "%02hhx ", ioread8(wrap->virt_addr + id));
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, usize, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS read callback of inbound memory window node
+ */
+static ssize_t mw_dbgfs_inmw_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct inmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+
+ /* Read the first DWORD with the size of the data */
+ datasize = le32_to_cpu(*(u32 *)wrap->virt_addr);
+
+ /* Calculate the size of the output buffer */
+ datasize = min(datasize, usize);
+
+ /* If there is nothing to copy then just return from the function */
+ if (0 == datasize) {
+ return 0;
+ }
+
+ /* Allocate the buffer */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the output buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data from the shared memory to the temporary buffer */
+ memcpy(databuf, wrap->virt_addr + 4, datasize);
+
+ /* Copy the data to the output buffer */
+ ret = simple_read_from_buffer(ubuf, usize, offp, databuf, datasize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS write callback of inbound memory window node
+ */
+static ssize_t mw_dbgfs_inmw_write(struct file *filep, const char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct inmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *databuf;
+ size_t datasize;
+ ssize_t ret = 0;
+
+ /* Calculate the data size */
+ datasize = min((size_t)wrap->size - 4, usize);
+
+ /* Allocate the memory for sending data */
+ databuf = kmalloc(datasize, GFP_KERNEL);
+ if (NULL == databuf) {
+ dev_err_mw(ctx, "No memory to allocate the input data buffer");
+ return -ENOMEM;
+ }
+
+ /* Copy the data to the output buffer */
+ ret = simple_write_to_buffer(databuf, usize, offp, ubuf, usize);
+ if (0 > ret) {
+ dev_err_mw(ctx, "Failed to copy the data from the User-space");
+ kfree(databuf);
+ return ret;
+ }
+
+ /* First DWORD is the data size */
+ *(u32 *)wrap->virt_addr = cpu_to_le32(datasize);
+
+ /* Copy the data to the memory window */
+ memcpy(wrap->virt_addr + 4, databuf, datasize);
+
+ /* Free the memory allocated for the buffer */
+ kfree(databuf);
+
+ return datasize;
+}
+
+/*
+ * DebugFS read callback of outbound memory window configurations
+ */
+static ssize_t mw_dbgfs_inmw_cfg_read(struct file *filep, char __user *ubuf,
+ size_t usize, loff_t *offp)
+{
+ struct inmw_wrap *wrap = filep->private_data;
+ struct mw_ctx *ctx = wrap->ctx;
+ char *strbuf;
+ size_t size;
+ ssize_t ret = 0, off = 0;
+ int id;
+
+ /* Limit the buffer size */
+ size = min_t(size_t, usize, 0x800U);
+
+ /* Allocate the memory for the buffer */
+ strbuf = kmalloc(size, GFP_KERNEL);
+ if (NULL == strbuf) {
+ dev_dbg_mw(ctx,
+ "Failed to allocated the memory for inbound memory "
+ "window configuration");
+ return -ENOMEM;
+ }
+
+ /* Put the data into the string buffer */
+ off += scnprintf(strbuf + off, size - off,
+ "\n\t\tNTB Inbound Memory Window configuration:\n\n");
+
+ /* Current driver state */
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address\t\t- 0x%p\n", CAST_DMA_PTR(wrap->dma_addr));
+ off += scnprintf(strbuf + off, size - off,
+ "DMA address alignment\t- %lu\n",
+ (unsigned long int)wrap->addr_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Virtual address\t\t- 0x%p\n", (void *)wrap->virt_addr);
+ off += scnprintf(strbuf + off, size - off,
+ "Size of the window\t- %lu\n",
+ (unsigned long int)wrap->size);
+ off += scnprintf(strbuf + off, size - off,
+ "Size alignment\t\t- %lu\n",
+ (unsigned long int)wrap->size_align);
+ off += scnprintf(strbuf + off, size - off,
+ "Maximum size\t\t- %lu\n",
+ (unsigned long int)wrap->size_max);
+
+ /* Print raw data from the inbound window */
+ off += scnprintf(strbuf + off, size - off,
+ "Raw data (16 bytes)\t- ");
+ for (id = 0; id < 16; id++) {
+ off += scnprintf(strbuf + off, size - off,
+ "%02hhx ", *(char *)(wrap->virt_addr + id));
+ }
+ off += scnprintf(strbuf + off, size - off, "\n");
+
+
+ /* Copy the buffer to the User Space */
+ ret = simple_read_from_buffer(ubuf, usize, offp, strbuf, off);
+ kfree(strbuf);
+
+ return ret;
+}
+
+/*
+ * DebugFS initialization function
+ */
+#define NAMESIZE 16
+static int mw_init_dbgfs(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct dentry *dbgfs_node;
+ char nodename[NAMESIZE];
+ const char *devname;
+ int outmwindx, inmwindx, ret;
+
+ /* If the top directory is not created then do nothing */
+ if (IS_ERR_OR_NULL(dbgfs_topdir)) {
+ dev_warn_mw(ctx,
+ "Top DebugFS directory has not been created for "
+ DRIVER_NAME);
+ return PTR_ERR(dbgfs_topdir);
+ }
+
+ /* Retrieve the device name */
+ devname = dev_name(&ntb->dev);
+
+ /* Create the device related subdirectory */
+ ctx->dbgfs_dir = debugfs_create_dir(devname, dbgfs_topdir);
+ if (IS_ERR_OR_NULL(ctx->dbgfs_dir)) {
+ dev_warn_mw(ctx,
+ "Failed to create the DebugFS subdirectory %s",
+ devname);
+ return PTR_ERR(ctx->dbgfs_dir);
+ }
+
+ /* Walk through all the outbound memory windows creating the
+ * corresponding nodes */
+ for (outmwindx = 0; outmwindx < ctx->outmws_cnt; outmwindx++) {
+ /* Create the name of the read/write node */
+ snprintf(nodename, NAMESIZE, "outmw%d", outmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->outmws[outmwindx],
+ &mw_dbgfs_outmw_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+
+ /* Create the name of the configuration node */
+ snprintf(nodename, NAMESIZE, "outmwcfg%d", outmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->outmws[outmwindx],
+ &mw_dbgfs_outmw_cfg_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+ }
+
+ /* Walk through all the inbound memory windows creating the
+ * corresponding nodes */
+ for (inmwindx = 0; inmwindx < ctx->inmws_cnt; inmwindx++) {
+ /* Create the name of the read/write node */
+ snprintf(nodename, NAMESIZE, "inmw%d", inmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->inmws[inmwindx],
+ &mw_dbgfs_inmw_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+
+ /* Create the name of the configuration node */
+ snprintf(nodename, NAMESIZE, "inmwcfg%d", inmwindx);
+ /* Create the data read/write node */
+ dbgfs_node = debugfs_create_file(nodename, S_IRWXU,
+ ctx->dbgfs_dir, &ctx->inmws[inmwindx],
+ &mw_dbgfs_inmw_cfg_ops);
+ if (IS_ERR(dbgfs_node)) {
+ dev_err_mw(ctx, "Could not create DebugFS '%s' node",
+ nodename);
+ ret = PTR_ERR(dbgfs_node);
+ goto err_rm_dir;
+ }
+ }
+
+ dev_dbg_mw(ctx, "NTB Memory Windows DebugFS top diretory is created "
+ "for %s", devname);
+
+ return SUCCESS;
+
+err_rm_dir:
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ return ret;
+}
+
+/*
+ * DebugFS deinitialization function
+ */
+static void mw_deinit_dbgfs(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Remove the DebugFS directory */
+ debugfs_remove_recursive(ctx->dbgfs_dir);
+
+ dev_dbg_mw(ctx, "Memory Windows DebugFS nodes %s/ are discarded",
+ dev_name(&ntb->dev));
+}
+
+/*===========================================================================
+ * NTB device/client driver initialization
+ *===========================================================================*/
+
+/*
+ * NTB device events handlers
+ */
+static const struct ntb_ctx_ops mw_ops = {
+ .link_event = link_event_handler,
+ .msg_event = msg_event_handler
+};
+
+/*
+ * Create the outbound memory windows
+ */
+static int mw_create_outmws(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct outmw_wrap *outmw;
+ int ret, mwindx;
+
+ /* Loop over all the outbound memory window descriptors initializing the
+ * corresponding fields */
+ for (mwindx = 0; mwindx < ctx->outmws_cnt; mwindx++) {
+ outmw = &ctx->outmws[mwindx];
+ /* Outbound memory windows are disabled by default */
+ outmw->enabled = false;
+
+ /* Set the context */
+ outmw->ctx = ctx;
+
+ /* Retrieve the physical address of the memory to map */
+ ret = ntb_mw_get_maprsc(ntb, mwindx, &outmw->phys_addr,
+ &outmw->size);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to get map resources of "
+ "outbound window %d", mwindx);
+ mwindx--;
+ goto err_unmap_rsc;
+ }
+
+ /* Map the memory window resources */
+ outmw->virt_addr = ioremap_nocache(outmw->phys_addr, outmw->size);
+
+ /* Retrieve the memory windows maximum size and alignments */
+ ret = ntb_mw_get_align(ntb, mwindx, &outmw->addr_align,
+ &outmw->size_align, &outmw->size_max);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to get alignment options of "
+ "outbound window %d", mwindx);
+ goto err_unmap_rsc;
+ }
+ }
+
+ dev_dbg_mw(ctx, "Outbound memory windows are created");
+
+ return SUCCESS;
+
+err_unmap_rsc:
+ for (; 0 <= mwindx; mwindx--) {
+ outmw = &ctx->outmws[mwindx];
+ iounmap(outmw->virt_addr);
+ }
+
+ return ret;
+}
+
+/*
+ * Free the outbound memory windows
+ */
+static void mw_free_outmws(struct mw_ctx *ctx)
+{
+ struct outmw_wrap *outmw;
+ int mwindx;
+
+ /* Loop over all the outbound memory window descriptors and unmap the
+ * resources */
+ for (mwindx = 0; mwindx < ctx->outmws_cnt; mwindx++) {
+ outmw = &ctx->outmws[mwindx];
+
+ /* Disable the memory window */
+ outmw->enabled = false;
+
+ /* Unmap the resource */
+ iounmap(outmw->virt_addr);
+ }
+
+ dev_dbg_mw(ctx, "Outbound memory windows are freed");
+}
+
+/*
+ * Create the inbound memory windows
+ */
+static int mw_create_inmws(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct inmw_wrap *inmw;
+ int mwindx, ret = SUCCESS;
+
+ /* Loop over all the inbound memory window descriptors initializing the
+ * corresponding fields */
+ for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
+ inmw = &ctx->inmws[mwindx];
+ /* Set the context */
+ inmw->ctx = ctx;
+ /* Retrieve the memory windows maximum size and alignments */
+ ret = ntb_peer_mw_get_align(ntb, mwindx, &inmw->addr_align,
+ &inmw->size_align, &inmw->size_max);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to get alignment options of "
+ "inbound window %d", mwindx);
+ mwindx--;
+ ret = -ENOMEM;
+ goto err_free_dma_bufs;
+ }
+ /* Allocate all the maximum possible size */
+ inmw->size = inmw->size_max;
+
+ /* Allocate the cache coherent DMA memory windows */
+ inmw->virt_addr = dma_zalloc_coherent(&ntb->dev, inmw->size,
+ &inmw->dma_addr, GFP_KERNEL);
+ if (IS_ERR_OR_NULL(inmw->virt_addr)) {
+ dev_err_mw(ctx,
+ "Failed to allocate the inbound buffer for %d",
+ mwindx);
+ mwindx--;
+ ret = -ENOMEM;
+ goto err_free_dma_bufs;
+ }
+ /* Make sure the allocated address is properly aligned */
+ if (!IS_ALIGNED(inmw->dma_addr, inmw->addr_align)) {
+ dev_err_mw(ctx, "DMA address %p of inbound mw %d isn't "
+ "aligned with %lu", CAST_DMA_PTR(inmw->dma_addr),
+ mwindx, (unsigned long int)inmw->addr_align);
+ ret = -EINVAL;
+ goto err_free_dma_bufs;
+ }
+ }
+
+ dev_dbg_mw(ctx, "Inbound memory windows are created");
+
+ return SUCCESS;
+
+err_free_dma_bufs:
+ for (; 0 <= mwindx; mwindx--) {
+ inmw = &ctx->inmws[mwindx];
+ dma_free_coherent(&ntb->dev, inmw->size, inmw->virt_addr,
+ inmw->dma_addr);
+ }
+
+ return ret;
+}
+
+/*
+ * Free the inbound memory windows
+ */
+static void mw_free_inmws(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ struct inmw_wrap *inmw;
+ int mwindx;
+
+ /* Loop over all the inbound memory window descriptors and free
+ * the allocated memory */
+ for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
+ inmw = &ctx->inmws[mwindx];
+
+ /* Free the cache coherent DMA memory window */
+ dma_free_coherent(&ntb->dev, inmw->size, inmw->virt_addr,
+ inmw->dma_addr);
+ }
+
+ dev_dbg_mw(ctx, "Inbound memory windows are freed");
+}
+
+/*
+ * Create the driver context structure
+ */
+static struct mw_ctx *mw_create_ctx(struct ntb_dev *ntb)
+{
+ struct mw_ctx *ctx, *ret;
+ int node;
+
+ /* Allocate the memory at the device NUMA node */
+ node = dev_to_node(&ntb->dev);
+ ctx = kzalloc_node(sizeof(*ctx), GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx)) {
+ dev_err(&ntb->dev,
+ "No memory for NTB Memory windows driver context");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Initialize the context NTB device pointer */
+ ctx->ntb = ntb;
+
+ /* Read the number of memory windows */
+ /* Number of memory windows local NTB device can set to the translated
+ * address register */
+ ctx->outmws_cnt = ntb_mw_count(ntb);
+ /* Number of memory windows peer can set to his translated address
+ * register */
+ ctx->inmws_cnt = ntb_peer_mw_count(ntb);
+
+ /* Alter the inbound memory windows count with respect to the driver
+ * parameter */
+ mw_alter_params(ctx);
+
+ /* Allocate the memory for memory window descriptors */
+ ctx->outmws = kzalloc_node(ctx->outmws_cnt * sizeof(*ctx->outmws),
+ GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx->outmws)) {
+ dev_err_mw(ctx,
+ "Failed to allocate memory for outbound MW descriptors");
+ ret = ERR_PTR(-ENOMEM);
+ goto err_free_ctx;
+ }
+ ctx->inmws = kzalloc_node(ctx->inmws_cnt * sizeof(*ctx->inmws),
+ GFP_KERNEL, node);
+ if (IS_ERR_OR_NULL(ctx->inmws)) {
+ dev_err_mw(ctx,
+ "Failed to allocate memory for inbound MW descriptors");
+ ret = ERR_PTR(-ENOMEM);
+ goto err_free_outmws;
+ }
+
+ dev_dbg_mw(ctx, "Context structure is created");
+
+ return ctx;
+
+/*err_free_inmws:
+ kfree(ctx->inmws);
+*/
+err_free_outmws:
+ kfree(ctx->outmws);
+err_free_ctx:
+ kfree(ctx);
+
+ return ret;
+}
+
+/*
+ * Free the driver context structure
+ */
+static void mw_free_ctx(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Free the outbound memory windows descriptors */
+ kfree(ctx->outmws);
+
+ /* Free the inbound memory windows descriptors */
+ kfree(ctx->inmws);
+
+ /* Free the memory allocated for the context structure */
+ kfree(ctx);
+
+ dev_dbg(&ntb->dev, "Context structure is freed");
+}
+
+/*
+ * Initialize the ntb device structure
+ */
+static int mw_init_ntb_dev(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+ int ret;
+
+ /* Set the NTB device events context */
+ ret = ntb_set_ctx(ntb, ctx, &mw_ops);
+ if (SUCCESS != ret) {
+ dev_err_mw(ctx, "Failed to specify the NTB device context");
+ return ret;
+ }
+
+ /* Enable the link and rise the event to check the link state */
+ ntb_link_enable(ntb, NTB_SPEED_AUTO, NTB_WIDTH_AUTO);
+ /*ntb_link_event(ntb);*/
+
+ dev_dbg_mw(ctx, "NTB device is initialized");
+
+ return SUCCESS;
+}
+
+/*
+ * Deinitialize the ntb device structure
+ */
+static void mw_stop_ntb_dev(struct mw_ctx *ctx)
+{
+ struct ntb_dev *ntb = ctx->ntb;
+
+ /* Clear the context */
+ ntb_clear_ctx(ntb);
+
+ /* Disable the link */
+ ntb_link_disable(ntb);
+
+ dev_dbg_mw(ctx, "NTB device is deinitialized");
+}
+
+/*
+ * Initialize the DMA masks
+ */
+static int __maybe_unused mw_ntb_set_dma_mask(struct ntb_dev *ntb)
+{
+ struct device *dev;
+ int ret = SUCCESS;
+
+ /* Get the NTB device structure */
+ dev = &ntb->dev;
+ /* Try to set the Highmem DMA address */
+ ret = dma_set_mask(dev, DMA_BIT_MASK(64));
+ if (SUCCESS == ret) {
+ /* Next call won't fail of the upper one returned OK */
+ dma_set_coherent_mask(dev, DMA_BIT_MASK(64));
+ return SUCCESS;
+ }
+
+ /* Warn if the HIGHMEM can be used for DMA */
+ dev_warn(dev, "Cannot set the NTB device DMA highmem mask");
+
+ /* Try the Low 32-bits DMA addresses */
+ ret = dma_set_mask(dev, DMA_BIT_MASK(32));
+ if (SUCCESS == ret) {
+ /* The same is here */
+ dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
+ return SUCCESS;
+ }
+
+ dev_err(dev, "Failed to set the NTB device DMA lowmem mask");
+
+ return ret;
+}
+
+/*
+ * NTB device probe() callback function
+ */
+static int mw_probe(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct mw_ctx *ctx;
+ int ret;
+
+ /* Only asynchronous hardware is supported */
+ if (!ntb_valid_async_dev_ops(ntb)) {
+ return -EINVAL;
+ }
+
+ /* Check whether the messaging supports at least 4 DWORDS */
+ ret = ntb_msg_size(ntb);
+ if (4 > ret) {
+ dev_err(&ntb->dev, "NTB Messaging supports just %d < 4 dwords",
+ ret);
+ return -EINVAL;
+ }
+
+ /* Set the NTB device DMA mask */
+ /*ret = mw_ntb_set_dma_mask(ntb);
+ if (SUCCESS != ret) {
+ return ret;
+ }*/
+
+ /* Create the current device context */
+ ctx = mw_create_ctx(ntb);
+ if (IS_ERR_OR_NULL(ctx)) {
+ return PTR_ERR(ctx);
+ }
+
+ /* Allocate the inbound memory windows */
+ ret = mw_create_inmws(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_ctx;
+ }
+
+ /* Map the outbound memory windows */
+ ret = mw_create_outmws(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_inmws;
+ }
+
+ /* Initialize the NTB device */
+ ret = mw_init_ntb_dev(ctx);
+ if (SUCCESS != ret) {
+ goto err_free_outmws;
+ }
+
+ /* Create the DebugFS nodes */
+ (void)mw_init_dbgfs(ctx);
+
+ return SUCCESS;
+
+/*err_stop_ntb_dev:
+ mw_stop_ntb_dev(ctx);
+*/
+err_free_outmws:
+ mw_free_outmws(ctx);
+err_free_inmws:
+ mw_free_inmws(ctx);
+err_free_ctx:
+ mw_free_ctx(ctx);
+
+ return ret;
+
+}
+
+/*
+ * NTB device remove() callback function
+ */
+static void mw_remove(struct ntb_client *client, struct ntb_dev *ntb)
+{
+ struct mw_ctx *ctx = ntb->ctx;
+
+ /* Send the message so the peer would lock outbound memory windows */
+ mw_send_freeaddrs_cmd(ctx);
+
+ /* Remove the DebugFS node */
+ mw_deinit_dbgfs(ctx);
+
+ /* Disable the NTB device link and clear the context */
+ mw_stop_ntb_dev(ctx);
+
+ /* Clear the outbound memory windows */
+ mw_free_outmws(ctx);
+
+ /* Clear the inbound memory windows */
+ mw_free_inmws(ctx);
+
+ /* Free the allocated context */
+ mw_free_ctx(ctx);
+}
+
+/*
+ * NTB bus client driver structure definition
+ */
+static struct ntb_client mw_client = {
+ .ops = {
+ .probe = mw_probe,
+ .remove = mw_remove,
+ },
+};
+/* module_ntb_client(mw_client); */
+
+/*
+ * Driver initialize method
+ */
+static int __init ntb_mw_init(void)
+{
+ /* Create the top DebugFS directory if the FS is initialized */
+ if (debugfs_initialized())
+ dbgfs_topdir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+
+ /* Registers the client driver */
+ return ntb_register_client(&mw_client);
+}
+module_init(ntb_mw_init);
+
+/*
+ * Driver exit method
+ */
+static void __exit ntb_mw_exit(void)
+{
+ /* Unregister the client driver */
+ ntb_unregister_client(&mw_client);
+
+ /* Discard the top DebugFS directory */
+ debugfs_remove_recursive(dbgfs_topdir);
+}
+module_exit(ntb_mw_exit);
+
--
2.6.6
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 3/3] ntb: Test client drivers for asynchronous NTB devices
2016-07-26 19:50 ` [PATCH 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
@ 2016-07-28 12:58 ` kbuild test robot
0 siblings, 0 replies; 11+ messages in thread
From: kbuild test robot @ 2016-07-28 12:58 UTC (permalink / raw)
To: Serge Semin
Cc: kbuild-all, jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu,
Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
[-- Attachment #1: Type: text/plain, Size: 17887 bytes --]
Hi,
[auto build test WARNING on ntb/ntb-next]
[cannot apply to v4.7 next-20160728]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Serge-Semin/ntb-Asynchronous-NTB-devices-support/20160727-082033
base: https://github.com/jonmason/ntb ntb-next
config: sparc64-allyesconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 5.4.0-6) 5.4.0 20160609
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=sparc64
All warnings (new ones prefixed by >>):
In file included from include/linux/swab.h:4:0,
from include/uapi/linux/byteorder/big_endian.h:12,
from include/linux/byteorder/big_endian.h:4,
from arch/sparc/include/uapi/asm/byteorder.h:4,
from arch/sparc/include/asm/bitops_64.h:15,
from arch/sparc/include/asm/bitops.h:4,
from include/linux/bitops.h:36,
from include/linux/kernel.h:10,
from drivers/ntb/test/ntb_mw_test.c:57:
drivers/ntb/test/ntb_mw_test.c: In function 'mw_send_inmw_addrs':
>> drivers/ntb/test/ntb_mw_test.c:353:49: warning: right shift count >= width of type [-Wshift-count-overflow]
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
include/uapi/linux/swab.h:116:32: note: in definition of macro '__swab32'
(__builtin_constant_p((__u32)(x)) ? \
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^
>> drivers/ntb/test/ntb_mw_test.c:353:20: note: in expansion of macro 'to_sh32'
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
>> drivers/ntb/test/ntb_mw_test.c:353:49: warning: right shift count >= width of type [-Wshift-count-overflow]
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
include/uapi/linux/swab.h:17:12: note: in definition of macro '___constant_swab32'
(((__u32)(x) & (__u32)0x000000ffUL) << 24) | \
^
include/uapi/linux/byteorder/big_endian.h:32:43: note: in expansion of macro '__swab32'
#define __cpu_to_le32(x) ((__force __le32)__swab32((x)))
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^
>> drivers/ntb/test/ntb_mw_test.c:353:20: note: in expansion of macro 'to_sh32'
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
>> drivers/ntb/test/ntb_mw_test.c:353:49: warning: right shift count >= width of type [-Wshift-count-overflow]
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
include/uapi/linux/swab.h:18:12: note: in definition of macro '___constant_swab32'
(((__u32)(x) & (__u32)0x0000ff00UL) << 8) | \
^
include/uapi/linux/byteorder/big_endian.h:32:43: note: in expansion of macro '__swab32'
#define __cpu_to_le32(x) ((__force __le32)__swab32((x)))
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^
>> drivers/ntb/test/ntb_mw_test.c:353:20: note: in expansion of macro 'to_sh32'
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
>> drivers/ntb/test/ntb_mw_test.c:353:49: warning: right shift count >= width of type [-Wshift-count-overflow]
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
include/uapi/linux/swab.h:19:12: note: in definition of macro '___constant_swab32'
(((__u32)(x) & (__u32)0x00ff0000UL) >> 8) | \
^
include/uapi/linux/byteorder/big_endian.h:32:43: note: in expansion of macro '__swab32'
#define __cpu_to_le32(x) ((__force __le32)__swab32((x)))
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^
>> drivers/ntb/test/ntb_mw_test.c:353:20: note: in expansion of macro 'to_sh32'
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
>> drivers/ntb/test/ntb_mw_test.c:353:49: warning: right shift count >= width of type [-Wshift-count-overflow]
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
include/uapi/linux/swab.h:20:12: note: in definition of macro '___constant_swab32'
(((__u32)(x) & (__u32)0xff000000UL) >> 24)))
^
include/uapi/linux/byteorder/big_endian.h:32:43: note: in expansion of macro '__swab32'
#define __cpu_to_le32(x) ((__force __le32)__swab32((x)))
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^
>> drivers/ntb/test/ntb_mw_test.c:353:20: note: in expansion of macro 'to_sh32'
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
>> drivers/ntb/test/ntb_mw_test.c:353:49: warning: right shift count >= width of type [-Wshift-count-overflow]
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
include/uapi/linux/swab.h:118:12: note: in definition of macro '__swab32'
__fswab32(x))
^
include/linux/byteorder/generic.h:87:21: note: in expansion of macro '__cpu_to_le32'
#define cpu_to_le32 __cpu_to_le32
^
>> drivers/ntb/test/ntb_mw_test.c:353:20: note: in expansion of macro 'to_sh32'
msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
^
drivers/ntb/test/ntb_mw_test.c: In function 'mw_set_outmw_addr':
>> drivers/ntb/test/ntb_mw_test.c:407:61: warning: left shift count >= width of type [-Wshift-count-overflow]
outmw->dma_addr |= ((dma_addr_t)from_sh32(msg->payload[1]) << 32);
^
>> drivers/ntb/test/ntb_mw_test.c:413:23: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"within %u bytes", (void *)outmw->dma_addr,
^
drivers/ntb/test/ntb_mw_test.c:189:29: note: in definition of macro 'dev_err_mw'
dev_err(&ctx->ntb->dev, ## args)
^
drivers/ntb/test/ntb_mw_test.c:422:33: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"outbound memory window %d", (void *)outmw->dma_addr,
^
drivers/ntb/test/ntb_mw_test.c:189:29: note: in definition of macro 'dev_err_mw'
dev_err(&ctx->ntb->dev, ## args)
^
In file included from include/linux/printk.h:289:0,
from include/linux/kernel.h:13,
from drivers/ntb/test/ntb_mw_test.c:57:
drivers/ntb/test/ntb_mw_test.c:431:27: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"address 0x%p", mwindx, (void *)outmw->dma_addr);
^
include/linux/dynamic_debug.h:87:9: note: in definition of macro 'dynamic_dev_dbg'
##__VA_ARGS__); \
^
>> drivers/ntb/test/ntb_mw_test.c:195:2: note: in expansion of macro 'dev_dbg'
dev_dbg(&ctx->ntb->dev, ## args)
^
>> drivers/ntb/test/ntb_mw_test.c:430:2: note: in expansion of macro 'dev_dbg_mw'
dev_dbg_mw(ctx, "Outbound memory window %d is initialized with "
^
drivers/ntb/test/ntb_mw_test.c: In function 'mw_dbgfs_outmw_cfg_read':
drivers/ntb/test/ntb_mw_test.c:766:30: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"DMA address\t\t- 0x%p\n", (void *)wrap->dma_addr);
^
drivers/ntb/test/ntb_mw_test.c: In function 'mw_dbgfs_inmw_cfg_read':
drivers/ntb/test/ntb_mw_test.c:915:30: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"DMA address\t\t- 0x%p\n", (void *)wrap->dma_addr);
^
drivers/ntb/test/ntb_mw_test.c: In function 'mw_create_inmws':
drivers/ntb/test/ntb_mw_test.c:1197:25: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"aligned with %lu", (void *)inmw->dma_addr, mwindx,
^
drivers/ntb/test/ntb_mw_test.c:189:29: note: in definition of macro 'dev_err_mw'
dev_err(&ctx->ntb->dev, ## args)
^
vim +353 drivers/ntb/test/ntb_mw_test.c
189 dev_err(&ctx->ntb->dev, ## args)
190 #define dev_warn_mw(ctx, args...) \
191 dev_warn(&ctx->ntb->dev, ## args)
192 #define dev_info_mw(ctx, args...) \
193 dev_info(&ctx->ntb->dev, ## args)
194 #define dev_dbg_mw(ctx, args...) \
> 195 dev_dbg(&ctx->ntb->dev, ## args)
196
197 /*
198 * Some common constant used in the driver for better readability:
199 * @ON: Enable something
200 * @OFF: Disable something
201 * @SUCCESS: Success of a function execution
202 * @MIN_MW_CNT: Minimum memory windows count
203 * @MAX_MW_CNT: Maximum memory windows count
204 */
205 #define ON ((u32)0x1)
206 #define OFF ((u32)0x0)
207 #define SUCCESS 0
208 #define MIN_MW_CNT ((unsigned char)1)
209 #define MAX_MW_CNT ((unsigned char)255)
210
211 /*
212 * Shared data converter to support the different CPU architectures
213 */
214 #define to_sh32(data) \
215 cpu_to_le32((data))
216 #define from_sh32(data) \
217 le32_to_cpu((data))
218
219 /*
220 * Module parameters:
221 * @inmw_cnt: Number of inbound memory windows [1; 255]
222 * @outmw_cnt: Number of outbound memory windows [1; 255]
223 * If the specified value exceeds the maximum possible valiue, then it is
224 * initialized with maximum one
225 */
226 static unsigned char inmws_cnt = MAX_MW_CNT;
227 module_param(inmws_cnt, byte, 0000);
228 MODULE_PARM_DESC(inmws_cnt,
229 "Inbound memory windows count. Those are the memory windows, which are "
230 "locally allocated. Their address is sent to the remote host."
231 " - Parameter can be set within [1; 255], where 255 means maximum possible"
232 " number of windows");
233
234 /*===========================================================================
235 * Helper methods
236 *===========================================================================*/
237
238 /*
239 * Alter the passed driver paremeters
240 */
241 static void mw_alter_params(struct mw_ctx *ctx)
242 {
243 unsigned char inmws_cnt_bak = ctx->inmws_cnt;
244
245 /* Clamp the inbound memory windows parameter */
246 ctx->inmws_cnt = clamp(inmws_cnt,
247 MIN_MW_CNT, (unsigned char)ctx->inmws_cnt);
248 if (inmws_cnt_bak != ctx->inmws_cnt) {
249 dev_warn_mw(ctx,
250 "Inbound memory windows count is altered from "
251 "%hhu to %hhu", inmws_cnt_bak, ctx->inmws_cnt);
252 }
253
254 dev_dbg_mw(ctx, "Memory windows test driver parameter is verified");
255 }
256
257 /*
258 * Memory block IO write method
259 */
260 static void iomem_write(void __iomem *dst, const void *src, size_t cnt)
261 {
262 while (cnt--) {
263 iowrite8(*(u8 *)src, dst);
264 dst++;
265 src++;
266 }
267 }
268
269 /*
270 * Memory block IO read method
271 */
272 static void iomem_read(void __iomem *src, void *dst, size_t cnt)
273 {
274 while (cnt--) {
275 *(u8 *)dst = ioread8(src);
276 dst++;
277 src++;
278 }
279 }
280
281 /*===========================================================================
282 * Message command handlers
283 *===========================================================================*/
284
285 /*
286 * Send MW_GETADDRS command method
287 */
288 static void mw_send_getaddrs_cmd(struct mw_ctx *ctx)
289 {
290 struct ntb_msg msg;
291 int sts;
292
293 /* Clear the message structure */
294 memset(&msg, 0, sizeof(msg));
295
296 /* Set the message type only */
297 msg.type = to_sh32(MW_GETADDRS);
298
299 /* Send the message */
300 sts = ntb_msg_post(ctx->ntb, &msg);
301 if (SUCCESS != sts) {
302 dev_err_mw(ctx, "Failed to send message to get outbound window "
303 "addresses");
304 }
305 }
306
307 /*
308 * Send MW_FREEADDRS command method
309 */
310 static void mw_send_freeaddrs_cmd(struct mw_ctx *ctx)
311 {
312 struct ntb_msg msg;
313 int sts;
314
315 /* Clear the message structure */
316 memset(&msg, 0, sizeof(msg));
317
318 /* Set the message type only */
319 msg.type = to_sh32(MW_FREEADDRS);
320
321 /* Send the message */
322 sts = ntb_msg_post(ctx->ntb, &msg);
323 if (SUCCESS != sts) {
324 dev_err_mw(ctx, "Failed to send a message to disable the peer "
325 "outbound windows");
326 }
327 }
328
329 /*
330 * Callback method for response on the command MW_GETADDRS
331 */
332 static void mw_send_inmw_addrs(struct mw_ctx *ctx)
333 {
334 struct ntb_msg msg;
335 struct inmw_wrap *inmw;
336 int mwindx, sts;
337
338 /* Clear the message structure */
339 memset(&msg, 0, sizeof(msg));
340
341 /* Walk through all the inbound memory windows and send the
342 * corresponding DMA address of the window */
343 for (mwindx = 0; mwindx < ctx->inmws_cnt; mwindx++) {
344 inmw = &ctx->inmws[mwindx];
345 /* Set the type and the memory window index */
346 msg.type = to_sh32(MW_DMAADDR | ((u32)mwindx << 16));
347
348 /* First set the size of the memory window */
349 msg.payload[0] = to_sh32(inmw->size);
350
351 /* Set the Upper part of the memory window address */
352 #ifdef CONFIG_64BIT
> 353 msg.payload[1] = to_sh32((u32)(inmw->dma_addr >> 32));
354 #else
355 /* WARNING! NTB entpoints must either have the same architecture
356 * (x32 or x64) or use lower 4Gb for memory windows */
357 msg.payload[1] = 0;
358 #endif /* !CONFIG_64BIT */
359 /* Set the Lower part of the memory window address */
360 msg.payload[2] = to_sh32((u32)(inmw->dma_addr));
361
362 /* Send the message */
363 sts = ntb_msg_post(ctx->ntb, &msg);
364 if (SUCCESS != sts) {
365 dev_err_mw(ctx,
366 "Failed to send a message with window %d "
367 "address", mwindx);
368 }
369 }
370 }
371
372 /*
373 * Method to set the corresponding memory window and enable it
374 */
375 static void mw_set_outmw_addr(struct mw_ctx *ctx, const struct ntb_msg *msg)
376 {
377 struct outmw_wrap *outmw;
378 int mwindx, sts;
379
380 /* Read the memory windows index (it's the part of the message type) */
381 mwindx = from_sh32(msg->type) >> 16;
382 if (ctx->outmws_cnt <= mwindx) {
383 dev_err_mw(ctx,
384 "Retrieved invalid outbound memory window index %d",
385 mwindx);
386 return;
387 }
388 outmw = &ctx->outmws[mwindx];
389
390 /* Read the memory window size and check whether it has proper size and
391 * alignment */
392 outmw->size = from_sh32(msg->payload[0]);
393 if (!IS_ALIGNED(outmw->size, outmw->size_align) ||
394 outmw->size_max < outmw->size) {
395 dev_err_mw(ctx,
396 "Retrieved invalid memory window %d size %u "
397 "(max: %u, align: %u)", mwindx, (unsigned int)outmw->size,
398 (unsigned int)outmw->size_max,
399 (unsigned int)outmw->size_align);
400 return;
401 }
402
403 /* Read the DMA address, where the second DWORD is the upper part and
404 * the third DWORD - lower */
405 outmw->dma_addr = from_sh32(msg->payload[2]);
406 #ifdef CONFIG_64BIT
> 407 outmw->dma_addr |= ((dma_addr_t)from_sh32(msg->payload[1]) << 32);
408 #endif /* CONFIG_64BIT */
409 /* Check whether the retrieved address is properly aligned */
410 if (!IS_ALIGNED(outmw->dma_addr, outmw->addr_align)) {
411 dev_err_mw(ctx,
412 "Outbound memory window address %p is not aligned "
> 413 "within %u bytes", (void *)outmw->dma_addr,
414 (unsigned int)outmw->addr_align);
415 return;
416 }
417
418 /* Set the translation address of the outbound memory window */
419 sts = ntb_mw_set_trans(ctx->ntb, mwindx, outmw->dma_addr, outmw->size);
420 if (SUCCESS != sts) {
421 dev_err_mw(ctx, "Failed to set the translated address %p of "
422 "outbound memory window %d", (void *)outmw->dma_addr,
423 mwindx);
424 return;
425 }
426
427 /* Enable the memory window */
428 outmw->enabled = true;
429
> 430 dev_dbg_mw(ctx, "Outbound memory window %d is initialized with "
> 431 "address 0x%p", mwindx, (void *)outmw->dma_addr);
432 }
433
434 /*
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 46476 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 3/3] ntb: Test client drivers for asynchronous NTB devices
2016-07-28 10:01 ` [PATCH v2 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
@ 2016-08-01 2:49 ` kbuild test robot
0 siblings, 0 replies; 11+ messages in thread
From: kbuild test robot @ 2016-08-01 2:49 UTC (permalink / raw)
To: Serge Semin
Cc: kbuild-all, jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu,
Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
[-- Attachment #1: Type: text/plain, Size: 5539 bytes --]
Hi Serge,
[auto build test WARNING on ntb/ntb-next]
[cannot apply to v4.7 next-20160729]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Serge-Semin/ntb-Asynchronous-NTB-devices-support/20160728-183315
base: https://github.com/jonmason/ntb ntb-next
config: i386-randconfig-x011-08010912 (attached as .config)
compiler: gcc-6 (Debian 6.1.1-9) 6.1.1 20160705
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All warnings (new ones prefixed by >>):
drivers/ntb/test/ntb_mw_test.c: In function 'mw_set_outmw_addr':
drivers/ntb/test/ntb_mw_test.c:225:3: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
((void *)(phys_addr_t)(addr))
^
>> drivers/ntb/test/ntb_mw_test.c:189:29: note: in expansion of macro 'CAST_DMA_PTR'
dev_err(&ctx->ntb->dev, ## args)
^~~~
>> drivers/ntb/test/ntb_mw_test.c:419:3: note: in expansion of macro 'dev_err_mw'
dev_err_mw(ctx,
^~~~~~~~~~
drivers/ntb/test/ntb_mw_test.c:225:3: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
((void *)(phys_addr_t)(addr))
^
>> drivers/ntb/test/ntb_mw_test.c:189:29: note: in expansion of macro 'CAST_DMA_PTR'
dev_err(&ctx->ntb->dev, ## args)
^~~~
drivers/ntb/test/ntb_mw_test.c:429:3: note: in expansion of macro 'dev_err_mw'
dev_err_mw(ctx, "Failed to set the translated address %p of "
^~~~~~~~~~
drivers/ntb/test/ntb_mw_test.c:225:3: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
((void *)(phys_addr_t)(addr))
^
>> include/linux/device.h:1206:41: note: in expansion of macro 'CAST_DMA_PTR'
dev_printk(KERN_DEBUG, dev, format, ##arg); \
^~~
drivers/ntb/test/ntb_mw_test.c:195:2: note: in expansion of macro 'dev_dbg'
dev_dbg(&ctx->ntb->dev, ## args)
^~~~~~~
drivers/ntb/test/ntb_mw_test.c:438:2: note: in expansion of macro 'dev_dbg_mw'
dev_dbg_mw(ctx, "Outbound memory window %d is initialized with "
^~~~~~~~~~
drivers/ntb/test/ntb_mw_test.c: In function 'mw_dbgfs_outmw_cfg_read':
drivers/ntb/test/ntb_mw_test.c:225:3: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
((void *)(phys_addr_t)(addr))
^
drivers/ntb/test/ntb_mw_test.c:774:30: note: in expansion of macro 'CAST_DMA_PTR'
"DMA address\t\t- 0x%p\n", CAST_DMA_PTR(wrap->dma_addr));
^~~~~~~~~~~~
drivers/ntb/test/ntb_mw_test.c:779:37: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
"Physycal map address\t- 0x%p\n", (void *)wrap->phys_addr);
^
drivers/ntb/test/ntb_mw_test.c: In function 'mw_dbgfs_inmw_cfg_read':
drivers/ntb/test/ntb_mw_test.c:225:3: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
((void *)(phys_addr_t)(addr))
^
drivers/ntb/test/ntb_mw_test.c:923:30: note: in expansion of macro 'CAST_DMA_PTR'
"DMA address\t\t- 0x%p\n", CAST_DMA_PTR(wrap->dma_addr));
^~~~~~~~~~~~
drivers/ntb/test/ntb_mw_test.c: In function 'mw_create_inmws':
drivers/ntb/test/ntb_mw_test.c:225:3: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
((void *)(phys_addr_t)(addr))
^
>> drivers/ntb/test/ntb_mw_test.c:189:29: note: in expansion of macro 'CAST_DMA_PTR'
dev_err(&ctx->ntb->dev, ## args)
^~~~
drivers/ntb/test/ntb_mw_test.c:1204:4: note: in expansion of macro 'dev_err_mw'
dev_err_mw(ctx, "DMA address %p of inbound mw %d isn't "
^~~~~~~~~~
vim +/CAST_DMA_PTR +189 drivers/ntb/test/ntb_mw_test.c
183 }
184
185 /*
186 * Wrapper dev_err/dev_warn/dev_info/dev_dbg macros
187 */
188 #define dev_err_mw(ctx, args...) \
> 189 dev_err(&ctx->ntb->dev, ## args)
190 #define dev_warn_mw(ctx, args...) \
191 dev_warn(&ctx->ntb->dev, ## args)
192 #define dev_info_mw(ctx, args...) \
193 dev_info(&ctx->ntb->dev, ## args)
194 #define dev_dbg_mw(ctx, args...) \
195 dev_dbg(&ctx->ntb->dev, ## args)
196
197 /*
198 * Some common constant used in the driver for better readability:
199 * @ON: Enable something
200 * @OFF: Disable something
201 * @SUCCESS: Success of a function execution
202 * @MIN_MW_CNT: Minimum memory windows count
203 * @MAX_MW_CNT: Maximum memory windows count
204 */
205 #define ON ((u32)0x1)
206 #define OFF ((u32)0x0)
207 #define SUCCESS 0
208 #define MIN_MW_CNT ((unsigned char)1)
209 #define MAX_MW_CNT ((unsigned char)255)
210
211 /*
212 * Shared data converter to support the different CPU architectures
213 */
214 #define to_sh32(data) \
215 cpu_to_le32((data))
216 #define from_sh32(data) \
217 le32_to_cpu((data))
218
219 /*
220 * Cast DMA address to real address pointer
221 *
222 * NOTE It's used in the printf's to get rid of warnings
223 */
224 #define CAST_DMA_PTR(addr) \
> 225 ((void *)(phys_addr_t)(addr))
226
227 /*
228 * Module parameters:
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 28404 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-08-01 2:50 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-07-26 19:50 [PATCH 0/3] ntb: Asynchronous NTB devices support Serge Semin
2016-07-26 19:50 ` [PATCH 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
2016-07-26 19:50 ` [PATCH 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
2016-07-27 16:29 ` kbuild test robot
2016-07-26 19:50 ` [PATCH 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
2016-07-28 12:58 ` kbuild test robot
2016-07-28 10:01 ` [PATCH v2 0/3] ntb: Asynchronous NTB devices support Serge Semin
2016-07-28 10:01 ` [PATCH v2 1/3] ntb: Add asynchronous devices support to NTB-bus interface Serge Semin
2016-07-28 10:01 ` [PATCH v2 2/3] ntb: IDT 89HPES*NT* PCIe-switches NTB device driver Serge Semin
2016-07-28 10:01 ` [PATCH v2 3/3] ntb: Test client drivers for asynchronous NTB devices Serge Semin
2016-08-01 2:49 ` kbuild test robot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).