* [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows
@ 2018-11-22 9:01 Wesley Sheng
2018-11-22 9:02 ` [PATCH 1/3] ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers Wesley Sheng
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Wesley Sheng @ 2018-11-22 9:01 UTC (permalink / raw)
To: kurt.schwemmer, logang, jdmason, dave.jiang, allenbh, linux-pci,
linux-ntb, linux-kernel
Cc: wesleyshenggit, wesley.sheng
Hi, Everyone,
This patch series adds support of >=4G memory windows.
Current Switchtec's BAR setup registers are limited to 32bits,
corresponding to the maximum MW (memory window) size is <4G.
Increase the MW sizes with the addition of the BAR Setup Extension
Register for the upper 32bits of a 64bits MW size. This increases the MW
range to between 4K and 2^63.
Additionally, we've made the following changes:
* debug print 64bit aligned crosslink BAR numbers
* Fix the array size of NT req id mapping table
Tested with ntb_test.sh successfully based on NTB fixes series from
Logan Gunthorpe <logang@deltatee.com> at
https://github.com/sbates130272/linux-p2pmem on branch of
ntb_multiport_fixes
Regards,
Wesley
Paul Selles (2):
ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers
ntb_hw_switchtec: Added support of >=4G memory windows
Wesley Sheng (1):
ntb_hw_switchtec: NT req id mapping table register entry number should
be 512
drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 11 ++++++++---
include/linux/switchtec.h | 10 +++++++---
2 files changed, 15 insertions(+), 6 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/3] ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers
2018-11-22 9:01 [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
@ 2018-11-22 9:02 ` Wesley Sheng
2018-11-22 9:02 ` [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Wesley Sheng @ 2018-11-22 9:02 UTC (permalink / raw)
To: kurt.schwemmer, logang, jdmason, dave.jiang, allenbh, linux-pci,
linux-ntb, linux-kernel
Cc: wesleyshenggit, wesley.sheng
From: Paul Selles <paul.selles@microchip.com>
Switchtec NTB crosslink BARs are 64bit addressed but they are printed as
32bit addressed BARs. Fix debug log to increment the BAR numbers by 2 to
reflect the 64bit address alignment.
Signed-off-by: Paul Selles <paul.selles@microchip.com>
Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
---
drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
index 5ee5f40..9916bc5 100644
--- a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
+++ b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
@@ -1120,7 +1120,7 @@ static int crosslink_enum_partition(struct switchtec_ntb *sndev,
dev_dbg(&sndev->stdev->dev,
"Crosslink BAR%d addr: %llx\n",
- i, bar_addr);
+ i*2, bar_addr);
if (bar_addr != bar_space * i)
continue;
--
2.7.4
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows
2018-11-22 9:01 [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
2018-11-22 9:02 ` [PATCH 1/3] ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers Wesley Sheng
@ 2018-11-22 9:02 ` Wesley Sheng
2018-11-23 15:49 ` kbuild test robot
2018-11-27 16:06 ` Jon Mason
2018-11-22 9:02 ` [PATCH 3/3] ntb_hw_switchtec: NT req id mapping table register entry number should be 512 Wesley Sheng
2018-11-22 16:54 ` [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Logan Gunthorpe
3 siblings, 2 replies; 7+ messages in thread
From: Wesley Sheng @ 2018-11-22 9:02 UTC (permalink / raw)
To: kurt.schwemmer, logang, jdmason, dave.jiang, allenbh, linux-pci,
linux-ntb, linux-kernel
Cc: wesleyshenggit, wesley.sheng
From: Paul Selles <paul.selles@microchip.com>
Current Switchtec's BAR setup registers are limited to 32bits,
corresponding to the maximum MW (memory window) size is <4G.
Increase the MW sizes with the addition of the BAR Setup Extension
Register for the upper 32bits of a 64bits MW size. This increases the MW
range to between 4K and 2^63.
Reported-by: Boris Glimcher <boris.glimcher@emc.com>
Signed-off-by: Paul Selles <paul.selles@microchip.com>
Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
---
drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 9 +++++++--
include/linux/switchtec.h | 6 +++++-
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
index 9916bc5..32850fb 100644
--- a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
+++ b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
@@ -264,6 +264,7 @@ static void switchtec_ntb_mw_clr_direct(struct switchtec_ntb *sndev, int idx)
ctl_val &= ~NTB_CTRL_BAR_DIR_WIN_EN;
iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
iowrite32(0, &ctl->bar_entry[bar].win_size);
+ iowrite32(0, &ctl->bar_ext_entry[bar].win_size);
iowrite64(sndev->self_partition, &ctl->bar_entry[bar].xlate_addr);
}
@@ -286,7 +287,9 @@ static void switchtec_ntb_mw_set_direct(struct switchtec_ntb *sndev, int idx,
ctl_val |= NTB_CTRL_BAR_DIR_WIN_EN;
iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
- iowrite32(xlate_pos | size, &ctl->bar_entry[bar].win_size);
+ iowrite32(xlate_pos | (size & 0xFFFFF000),
+ &ctl->bar_entry[bar].win_size);
+ iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
iowrite64(sndev->self_partition | addr,
&ctl->bar_entry[bar].xlate_addr);
}
@@ -1053,7 +1056,9 @@ static int crosslink_setup_mws(struct switchtec_ntb *sndev, int ntb_lut_idx,
ctl_val |= NTB_CTRL_BAR_DIR_WIN_EN;
iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
- iowrite32(xlate_pos | size, &ctl->bar_entry[bar].win_size);
+ iowrite32(xlate_pos | (size & 0xFFFFF000),
+ &ctl->bar_entry[bar].win_size);
+ iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
iowrite64(sndev->peer_partition | addr,
&ctl->bar_entry[bar].xlate_addr);
}
diff --git a/include/linux/switchtec.h b/include/linux/switchtec.h
index eee0412..1e6e333 100644
--- a/include/linux/switchtec.h
+++ b/include/linux/switchtec.h
@@ -248,7 +248,11 @@ struct ntb_ctrl_regs {
u32 win_size;
u64 xlate_addr;
} bar_entry[6];
- u32 reserved2[216];
+ struct {
+ u32 win_size;
+ u32 reserved[3];
+ } bar_ext_entry[6];
+ u32 reserved2[192];
u32 req_id_table[256];
u32 reserved3[512];
u64 lut_entry[512];
--
2.7.4
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/3] ntb_hw_switchtec: NT req id mapping table register entry number should be 512
2018-11-22 9:01 [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
2018-11-22 9:02 ` [PATCH 1/3] ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers Wesley Sheng
2018-11-22 9:02 ` [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
@ 2018-11-22 9:02 ` Wesley Sheng
2018-11-22 16:54 ` [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Logan Gunthorpe
3 siblings, 0 replies; 7+ messages in thread
From: Wesley Sheng @ 2018-11-22 9:02 UTC (permalink / raw)
To: kurt.schwemmer, logang, jdmason, dave.jiang, allenbh, linux-pci,
linux-ntb, linux-kernel
Cc: wesleyshenggit, wesley.sheng
The number of available NT req id mapping table entries per NTB control
register is 512. The driver mistakenly limits the number to 256.
Fix the array size of NT req id mapping table.
Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
---
include/linux/switchtec.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/linux/switchtec.h b/include/linux/switchtec.h
index 1e6e333..52a079b 100644
--- a/include/linux/switchtec.h
+++ b/include/linux/switchtec.h
@@ -253,8 +253,8 @@ struct ntb_ctrl_regs {
u32 reserved[3];
} bar_ext_entry[6];
u32 reserved2[192];
- u32 req_id_table[256];
- u32 reserved3[512];
+ u32 req_id_table[512];
+ u32 reserved3[256];
u64 lut_entry[512];
} __packed;
--
2.7.4
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows
2018-11-22 9:01 [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
` (2 preceding siblings ...)
2018-11-22 9:02 ` [PATCH 3/3] ntb_hw_switchtec: NT req id mapping table register entry number should be 512 Wesley Sheng
@ 2018-11-22 16:54 ` Logan Gunthorpe
3 siblings, 0 replies; 7+ messages in thread
From: Logan Gunthorpe @ 2018-11-22 16:54 UTC (permalink / raw)
To: Wesley Sheng, kurt.schwemmer, jdmason, dave.jiang, allenbh,
linux-pci, linux-ntb, linux-kernel
Cc: wesleyshenggit
Hey,
This entire series looks good to me.
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Logan
On 2018-11-22 2:01 a.m., Wesley Sheng wrote:
> Hi, Everyone,
>
> This patch series adds support of >=4G memory windows.
>
> Current Switchtec's BAR setup registers are limited to 32bits,
> corresponding to the maximum MW (memory window) size is <4G.
> Increase the MW sizes with the addition of the BAR Setup Extension
> Register for the upper 32bits of a 64bits MW size. This increases the MW
> range to between 4K and 2^63.
>
> Additionally, we've made the following changes:
>
> * debug print 64bit aligned crosslink BAR numbers
> * Fix the array size of NT req id mapping table
>
> Tested with ntb_test.sh successfully based on NTB fixes series from
> Logan Gunthorpe <logang@deltatee.com> at
> https://github.com/sbates130272/linux-p2pmem on branch of
> ntb_multiport_fixes
>
> Regards,
> Wesley
>
>
>
> Paul Selles (2):
> ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers
> ntb_hw_switchtec: Added support of >=4G memory windows
>
> Wesley Sheng (1):
> ntb_hw_switchtec: NT req id mapping table register entry number should
> be 512
>
> drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 11 ++++++++---
> include/linux/switchtec.h | 10 +++++++---
> 2 files changed, 15 insertions(+), 6 deletions(-)
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows
2018-11-22 9:02 ` [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
@ 2018-11-23 15:49 ` kbuild test robot
2018-11-27 16:06 ` Jon Mason
1 sibling, 0 replies; 7+ messages in thread
From: kbuild test robot @ 2018-11-23 15:49 UTC (permalink / raw)
To: Wesley Sheng
Cc: kbuild-all, kurt.schwemmer, logang, jdmason, dave.jiang, allenbh,
linux-pci, linux-ntb, linux-kernel, wesleyshenggit, wesley.sheng
[-- Attachment #1: Type: text/plain, Size: 2178 bytes --]
Hi Paul,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v4.20-rc3 next-20181123]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Wesley-Sheng/ntb_hw_switchtec-Added-support-of-4G-memory-windows/20181123-231700
config: i386-randconfig-x077-201846 (attached as .config)
compiler: gcc-7 (Debian 7.3.0-1) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All warnings (new ones prefixed by >>):
drivers/ntb/hw/mscc/ntb_hw_switchtec.c: In function 'switchtec_ntb_mw_set_direct':
>> drivers/ntb/hw/mscc/ntb_hw_switchtec.c:292:17: warning: right shift count >= width of type [-Wshift-count-overflow]
iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
^~
drivers/ntb/hw/mscc/ntb_hw_switchtec.c: In function 'crosslink_setup_mws':
drivers/ntb/hw/mscc/ntb_hw_switchtec.c:1061:18: warning: right shift count >= width of type [-Wshift-count-overflow]
iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
^~
vim +292 drivers/ntb/hw/mscc/ntb_hw_switchtec.c
277
278 static void switchtec_ntb_mw_set_direct(struct switchtec_ntb *sndev, int idx,
279 dma_addr_t addr, resource_size_t size)
280 {
281 int xlate_pos = ilog2(size);
282 int bar = sndev->peer_direct_mw_to_bar[idx];
283 struct ntb_ctrl_regs __iomem *ctl = sndev->mmio_peer_ctrl;
284 u32 ctl_val;
285
286 ctl_val = ioread32(&ctl->bar_entry[bar].ctl);
287 ctl_val |= NTB_CTRL_BAR_DIR_WIN_EN;
288
289 iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
290 iowrite32(xlate_pos | (size & 0xFFFFF000),
291 &ctl->bar_entry[bar].win_size);
> 292 iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
293 iowrite64(sndev->self_partition | addr,
294 &ctl->bar_entry[bar].xlate_addr);
295 }
296
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 31802 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows
2018-11-22 9:02 ` [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
2018-11-23 15:49 ` kbuild test robot
@ 2018-11-27 16:06 ` Jon Mason
1 sibling, 0 replies; 7+ messages in thread
From: Jon Mason @ 2018-11-27 16:06 UTC (permalink / raw)
To: wesley.sheng
Cc: Kurt Schwemmer, Logan Gunthorpe, Dave Jiang, Allen Hubbe,
linux-pci, linux-ntb, linux-kernel, wesleyshenggit
On Wed, Nov 21, 2018 at 9:19 PM Wesley Sheng <wesley.sheng@microchip.com> wrote:
>
> From: Paul Selles <paul.selles@microchip.com>
>
> Current Switchtec's BAR setup registers are limited to 32bits,
> corresponding to the maximum MW (memory window) size is <4G.
>
> Increase the MW sizes with the addition of the BAR Setup Extension
> Register for the upper 32bits of a 64bits MW size. This increases the MW
> range to between 4K and 2^63.
>
> Reported-by: Boris Glimcher <boris.glimcher@emc.com>
> Signed-off-by: Paul Selles <paul.selles@microchip.com>
> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com>
> ---
> drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 9 +++++++--
> include/linux/switchtec.h | 6 +++++-
> 2 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
> index 9916bc5..32850fb 100644
> --- a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
> +++ b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c
> @@ -264,6 +264,7 @@ static void switchtec_ntb_mw_clr_direct(struct switchtec_ntb *sndev, int idx)
> ctl_val &= ~NTB_CTRL_BAR_DIR_WIN_EN;
> iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
> iowrite32(0, &ctl->bar_entry[bar].win_size);
> + iowrite32(0, &ctl->bar_ext_entry[bar].win_size);
> iowrite64(sndev->self_partition, &ctl->bar_entry[bar].xlate_addr);
> }
>
> @@ -286,7 +287,9 @@ static void switchtec_ntb_mw_set_direct(struct switchtec_ntb *sndev, int idx,
> ctl_val |= NTB_CTRL_BAR_DIR_WIN_EN;
>
> iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
> - iowrite32(xlate_pos | size, &ctl->bar_entry[bar].win_size);
> + iowrite32(xlate_pos | (size & 0xFFFFF000),
> + &ctl->bar_entry[bar].win_size);
> + iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
Thanks for the patches. Overall the look good. Per the kbuild email,
size_t is 32bits on 32bit arch. So, this is going to have compile
warnings on those. Please address this and resubmit.
Also, patches 1 and 3 are bug fixes. Please do the following, reorder
the patches to make the bug fixes first and add a "Fixes" line to the
commit messages (see
https://www.kernel.org/doc/html/latest/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes).
This will allow me to split up the series and get the bug fixes into
v4.20 (and the stable trees).
Thanks,
Jon
> iowrite64(sndev->self_partition | addr,
> &ctl->bar_entry[bar].xlate_addr);
> }
> @@ -1053,7 +1056,9 @@ static int crosslink_setup_mws(struct switchtec_ntb *sndev, int ntb_lut_idx,
> ctl_val |= NTB_CTRL_BAR_DIR_WIN_EN;
>
> iowrite32(ctl_val, &ctl->bar_entry[bar].ctl);
> - iowrite32(xlate_pos | size, &ctl->bar_entry[bar].win_size);
> + iowrite32(xlate_pos | (size & 0xFFFFF000),
> + &ctl->bar_entry[bar].win_size);
> + iowrite32(size >> 32, &ctl->bar_ext_entry[bar].win_size);
> iowrite64(sndev->peer_partition | addr,
> &ctl->bar_entry[bar].xlate_addr);
> }
> diff --git a/include/linux/switchtec.h b/include/linux/switchtec.h
> index eee0412..1e6e333 100644
> --- a/include/linux/switchtec.h
> +++ b/include/linux/switchtec.h
> @@ -248,7 +248,11 @@ struct ntb_ctrl_regs {
> u32 win_size;
> u64 xlate_addr;
> } bar_entry[6];
> - u32 reserved2[216];
> + struct {
> + u32 win_size;
> + u32 reserved[3];
> + } bar_ext_entry[6];
> + u32 reserved2[192];
> u32 req_id_table[256];
> u32 reserved3[512];
> u64 lut_entry[512];
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-11-27 16:06 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-22 9:01 [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
2018-11-22 9:02 ` [PATCH 1/3] ntb_hw_switchtec: debug print 64bit aligned crosslink BAR Numbers Wesley Sheng
2018-11-22 9:02 ` [PATCH 2/3] ntb_hw_switchtec: Added support of >=4G memory windows Wesley Sheng
2018-11-23 15:49 ` kbuild test robot
2018-11-27 16:06 ` Jon Mason
2018-11-22 9:02 ` [PATCH 3/3] ntb_hw_switchtec: NT req id mapping table register entry number should be 512 Wesley Sheng
2018-11-22 16:54 ` [PATCH 0/3] ntb_hw_switchtec: Added support of >=4G memory windows Logan Gunthorpe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).