From: Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> To: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-ntb-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org, linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, linux-kselftest-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Jon Mason <jdmason-zqedIHPRSYI@public.gmane.org>, Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> Cc: Allen Hubbe <allenbh-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Dave Jiang <dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>, Serge Semin <fancer.lancer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Eric Pilmore <epilmore-Op3I1peydIbQT0dZR+AlfA@public.gmane.org>, Bjorn Helgaas <bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> Subject: [PATCH v4 04/10] NTB: Introduce functions to calculate multi-port resource index Date: Mon, 22 Apr 2019 15:05:22 -0600 [thread overview] Message-ID: <20190422210528.15289-5-logang@deltatee.com> (raw) In-Reply-To: <20190422210528.15289-1-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> When using multi-ports each port uses resources (dbs, msgs, mws, etc) on every other port. Creating a mapping for these resources such that each port has a corresponding resource on every other port is a bit tricky. Introduce the ntb_peer_resource_idx() function for this purpose. It returns the peer resource number that will correspond with the local peer index on the remote peer. Also, introduce ntb_peer_highest_mw_idx() which will use ntb_peer_resource_idx() but return the MW index starting with the highest index and working down. Signed-off-by: Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> Cc: Jon Mason <jdmason-zqedIHPRSYI@public.gmane.org> Cc: Dave Jiang <dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> Cc: Allen Hubbe <allenbh-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> --- include/linux/ntb.h | 70 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/include/linux/ntb.h b/include/linux/ntb.h index 91cf492b16a0..66552830544b 100644 --- a/include/linux/ntb.h +++ b/include/linux/ntb.h @@ -1557,4 +1557,74 @@ static inline int ntb_peer_msg_write(struct ntb_dev *ntb, int pidx, int midx, return ntb->ops->peer_msg_write(ntb, pidx, midx, msg); } +/** + * ntb_peer_resource_idx() - get a resource index for a given peer idx + * @ntb: NTB device context. + * @pidx: Peer port index. + * + * When constructing a graph of peers, each remote peer must use a different + * resource index (mw, doorbell, etc) to communicate with each other + * peer. + * + * In a two peer system, this function should always return 0 such that + * resource 0 points to the remote peer on both ports. + * + * In a 5 peer system, this function will return the following matrix + * + * pidx \ port 0 1 2 3 4 + * 0 0 0 1 2 3 + * 1 0 1 1 2 3 + * 2 0 1 2 2 3 + * 3 0 1 2 3 3 + * + * For example, if this function is used to program peer's memory + * windows, port 0 will program MW 0 on all it's peers to point to itself. + * port 1 will program MW 0 in port 0 to point to itself and MW 1 on all + * other ports. etc. + * + * For the legacy two host case, ntb_port_number() and ntb_peer_port_number() + * both return zero and therefore this function will always return zero. + * So MW 0 on each host would be programmed to point to the other host. + * + * Return: the resource index to use for that peer. + */ +static inline int ntb_peer_resource_idx(struct ntb_dev *ntb, int pidx) +{ + int local_port, peer_port; + + if (pidx >= ntb_peer_port_count(ntb)) + return -EINVAL; + + local_port = ntb_logical_port_number(ntb); + peer_port = ntb_peer_logical_port_number(ntb, pidx); + + if (peer_port < local_port) + return local_port - 1; + else + return local_port; +} + +/** + * ntb_peer_highest_mw_idx() - get a memory window index for a given peer idx + * using the highest index memory windows first + * + * @ntb: NTB device context. + * @pidx: Peer port index. + * + * Like ntb_peer_resource_idx(), except it returns indexes starting with + * last memory window index. + * + * Return: the resource index to use for that peer. + */ +static inline int ntb_peer_highest_mw_idx(struct ntb_dev *ntb, int pidx) +{ + int ret; + + ret = ntb_peer_resource_idx(ntb, pidx); + if (ret < 0) + return ret; + + return ntb_mw_count(ntb, pidx) - ret - 1; +} + #endif -- 2.20.1
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang@deltatee.com> To: linux-kernel@vger.kernel.org, linux-ntb@googlegroups.com, linux-pci@vger.kernel.org, iommu@lists.linux-foundation.org, linux-kselftest@vger.kernel.org, Jon Mason <jdmason@kudzu.us>, Joerg Roedel <joro@8bytes.org> Cc: Allen Hubbe <allenbh@gmail.com>, Dave Jiang <dave.jiang@intel.com>, Serge Semin <fancer.lancer@gmail.com>, Eric Pilmore <epilmore@gigaio.com>, Bjorn Helgaas <bhelgaas@google.com>, Logan Gunthorpe <logang@deltatee.com> Subject: [PATCH v4 04/10] NTB: Introduce functions to calculate multi-port resource index Date: Mon, 22 Apr 2019 15:05:22 -0600 [thread overview] Message-ID: <20190422210528.15289-5-logang@deltatee.com> (raw) Message-ID: <20190422210522.ZLO8GeGWA79qat2psQTmarDN24GJxH7gsghStwACNCk@z> (raw) In-Reply-To: <20190422210528.15289-1-logang@deltatee.com> When using multi-ports each port uses resources (dbs, msgs, mws, etc) on every other port. Creating a mapping for these resources such that each port has a corresponding resource on every other port is a bit tricky. Introduce the ntb_peer_resource_idx() function for this purpose. It returns the peer resource number that will correspond with the local peer index on the remote peer. Also, introduce ntb_peer_highest_mw_idx() which will use ntb_peer_resource_idx() but return the MW index starting with the highest index and working down. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Cc: Jon Mason <jdmason@kudzu.us> Cc: Dave Jiang <dave.jiang@intel.com> Cc: Allen Hubbe <allenbh@gmail.com> --- include/linux/ntb.h | 70 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/include/linux/ntb.h b/include/linux/ntb.h index 91cf492b16a0..66552830544b 100644 --- a/include/linux/ntb.h +++ b/include/linux/ntb.h @@ -1557,4 +1557,74 @@ static inline int ntb_peer_msg_write(struct ntb_dev *ntb, int pidx, int midx, return ntb->ops->peer_msg_write(ntb, pidx, midx, msg); } +/** + * ntb_peer_resource_idx() - get a resource index for a given peer idx + * @ntb: NTB device context. + * @pidx: Peer port index. + * + * When constructing a graph of peers, each remote peer must use a different + * resource index (mw, doorbell, etc) to communicate with each other + * peer. + * + * In a two peer system, this function should always return 0 such that + * resource 0 points to the remote peer on both ports. + * + * In a 5 peer system, this function will return the following matrix + * + * pidx \ port 0 1 2 3 4 + * 0 0 0 1 2 3 + * 1 0 1 1 2 3 + * 2 0 1 2 2 3 + * 3 0 1 2 3 3 + * + * For example, if this function is used to program peer's memory + * windows, port 0 will program MW 0 on all it's peers to point to itself. + * port 1 will program MW 0 in port 0 to point to itself and MW 1 on all + * other ports. etc. + * + * For the legacy two host case, ntb_port_number() and ntb_peer_port_number() + * both return zero and therefore this function will always return zero. + * So MW 0 on each host would be programmed to point to the other host. + * + * Return: the resource index to use for that peer. + */ +static inline int ntb_peer_resource_idx(struct ntb_dev *ntb, int pidx) +{ + int local_port, peer_port; + + if (pidx >= ntb_peer_port_count(ntb)) + return -EINVAL; + + local_port = ntb_logical_port_number(ntb); + peer_port = ntb_peer_logical_port_number(ntb, pidx); + + if (peer_port < local_port) + return local_port - 1; + else + return local_port; +} + +/** + * ntb_peer_highest_mw_idx() - get a memory window index for a given peer idx + * using the highest index memory windows first + * + * @ntb: NTB device context. + * @pidx: Peer port index. + * + * Like ntb_peer_resource_idx(), except it returns indexes starting with + * last memory window index. + * + * Return: the resource index to use for that peer. + */ +static inline int ntb_peer_highest_mw_idx(struct ntb_dev *ntb, int pidx) +{ + int ret; + + ret = ntb_peer_resource_idx(ntb, pidx); + if (ret < 0) + return ret; + + return ntb_mw_count(ntb, pidx) - ret - 1; +} + #endif -- 2.20.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2019-04-22 21:05 UTC|newest] Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-22 21:05 [PATCH v4 00/10] Support using MSI interrupts in ntb_transport Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe [not found] ` <20190422210528.15289-1-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> 2019-04-22 21:05 ` [PATCH v4 01/10] PCI/MSI: Support allocating virtual MSI interrupts Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 02/10] PCI/switchtec: Add module parameter to request more interrupts Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 03/10] NTB: Introduce helper functions to calculate logical port number Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe [this message] 2019-04-22 21:05 ` [PATCH v4 04/10] NTB: Introduce functions to calculate multi-port resource index Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 08/10] NTB: Add ntb_msi_test support to ntb_test Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 05/10] NTB: Rename ntb.c to support multiple source files in the module Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 06/10] NTB: Introduce MSI library Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 07/10] NTB: Introduce NTB MSI Test Client Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 09/10] NTB: Add MSI interrupt support to ntb_transport Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe 2019-04-22 21:05 ` [PATCH v4 10/10] NTB: Describe the ntb_msi_test client in the documentation Logan Gunthorpe 2019-04-22 21:05 ` Logan Gunthorpe
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190422210528.15289-5-logang@deltatee.com \ --to=logang-otvngxwrz7hwk0htik3j/w@public.gmane.org \ --cc=allenbh-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \ --cc=bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \ --cc=dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \ --cc=epilmore-Op3I1peydIbQT0dZR+AlfA@public.gmane.org \ --cc=fancer.lancer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \ --cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \ --cc=jdmason-zqedIHPRSYI@public.gmane.org \ --cc=joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org \ --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-kselftest-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-ntb-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org \ --cc=linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).