* [GIT PULL] please pull infiniband.git
@ 2007-07-12 23:07 Roland Dreier
2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier
0 siblings, 1 reply; 31+ messages in thread
From: Roland Dreier @ 2007-07-12 23:07 UTC (permalink / raw)
To: torvalds; +Cc: general, linux-kernel
Linus, please pull from
master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus
This tree is also available from kernel.org mirrors at:
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus
This will get the first batch of changes for the 2.6.23 merge window:
Andrew Morton (1):
IB: Fix ib_umem_get() when npages == 0
Arthur Jones (3):
IB/ipath: Update MAINTAINERS entry
IB/ipath: Test interrupts at driver startup
IB/ipath: Remove bogus RD_ATOMIC checks from modify_qp
Bryan O'Sullivan (1):
IB/ipath: Include <linux/vmalloc.h> to fix ppc64 build
Dave Olson (5):
IB/ipath: Support the IBA6110 revision 4
IB/ipath: Fix the mtrr_add args for chips with 2 buffer sizes
IB/ipath: Use S_ABORT not cancel and abort on exit freeze mode after recovery
IB/ipath: Be more cautious about coming out of freeze mode
IB/ipath: Change version wording to be less confusing with release number
Dotan Barak (2):
mlx4_core: Get the maximum message size from reported device capabilities
IB/core: Take sizeof the correct pointer when calling kmalloc()
Hal Rosenstock (1):
IB/mad: Enhance SMI for switch support
Hoang-Nam Nguyen (3):
IB/ehca: Change scaling_code parameter description to match default value
IB/ehca: Report RDMA atomic attributes in query_qp()
IB/ehca: Improve latency by unlocking after triggering the hardware
Jack Morgenstein (2):
IB/mlx4: Implement query QP
IB/mlx4: Implement query SRQ
Jan Engelhardt (1):
IB: Use menuconfig for InfiniBand menu
Joachim Fenkes (9):
IB/ehca: Refactor "maybe missed event" code
IB/ehca: HW level, HW caps and MTU autodetection
IB/ehca: QP code restructuring in preparation for SRQ
IB/ehca: add Shared Receive Queue support
IB/ehca: Lock renaming, static initializers
IB/ehca: Refactor sync between completions and destroy_cq using atomic_t
IB/ehca: Change idr spinlocks into rwlocks
IB/ehca: Return QP pointer in poll_cq()
IB/ehca: Notify consumers of LID/PKEY/SM changes after nondisruptive events
Joan Eslinger (1):
IB/ipath: Change use of constants for TID type to defined values
John Gregor (2):
IB/ipath: Remove incompletely implemented ipath_runtime flags and code
IB/ipath: Update copyright dates
Mark Debbage (2):
IB/ipath: Correct checking of swminor version field when using subports
IB/ipath: Make handling of one subport consistent
Michael Albaugh (4):
IB/ipath: Support blinking LEDs with an led_override file
IB/ipath: Lock and always use shadow copies of GPIO register
IB/ipath: Log "active" time and some errors to EEPROM
IB/ipath: Add capability to modify PBC word
Michael S. Tsirkin (2):
IB/mlx4: Include linux/mutex.h from mlx4_ib.h
mlx4_core: Include linux/mutex.h from mlx4.h
Ralph Campbell (10):
IB/ipath: Fix problem with next WQE after a UC completion
IB/ipath: Fix local loopback bug when waiting for resources
IB/ipath: Set M bit in BTH according to IB spec
IB/ipath: Fix RDMA read retry code
IB/ipath: Wait for PIO available interrupt
IB/ipath: Fix possible data corruption if multiple SGEs used for receive
IB/ipath: Duplicate RDMA reads can cause responder to NAK inappropriately
IB/ipath: Add barrier before updating WC head in shared memory
IB/ipath: Lower default number of kernel send buffers
IB/ipath: Remove support for preproduction HTX InfiniPath cards
Robert Walsh (5):
IB/ipath: Fix maximum MTU reporting
IB/ipath: Fill in some missing FMR-related fields in query_device
IB/ipath: Send ACK invalid where appropriate
IB/ipath: ipath_poll fixups and enhancements
IB/ipath: Clean send flags properly on QP reset
Roland Dreier (5):
IB: Remove garbage non-ASCII characters from comments
IB: Update mailing list address
IPoIB/cm: Fix warning if IPV6 is not enabled
IPoIB: Recycle loopback skbs instead of freeing and reallocating
IB: Update MAINTAINERS with Hal's new email address
Sean Hefty (7):
IB/ipath: return correct PortGUID in NodeInfo
IB/sa: Make sure SA queries use default P_Key
IB/cm: Use spin_lock_irq() instead of spin_lock_irqsave() when possible
IB/cm: Include HCA ACK delay in local ACK timeout
IB/cm: cm_msgs.h should include ib_cm.h
IB/cm: Fix handling of duplicate SIDR REQs
IB/cm: Send no match if a SIDR REQ does not match a listen
Shani Moideen (1):
IB/mthca: Replace memset(<addr>, 0, PAGE_SIZE) with clear_page(<addr>)
Stefan Roscher (2):
IB/ehca: Support UD low-latency QPs
IB/ehca: Set SEND_GRH flag for all non-LL UD QPs on eHCA2
Steve Wise (6):
RDMA/cxgb3: Streaming -> RDMA mode transition fixes
RDMA/cxgb3: TERMINATE WRs can hang the tx ofld queue
RDMA/cxgb3: Don't count neg_adv abort_req_rss messages as real aborts
RDMA/cxgb3: ctrl-qp init/clear shouldn't set the gen bit
RDMA/cxgb3: Don't post TID_RELEASE message
RDMA/cxgb3: Don't abort after failures sending the mpa reply
WANG Cong (1):
RDMA/cxgb3: Check return of kmalloc() in iwch_register_device()
MAINTAINERS | 15 +-
drivers/infiniband/Kconfig | 15 +-
drivers/infiniband/core/agent.c | 19 +-
drivers/infiniband/core/cm.c | 247 ++++---
drivers/infiniband/core/cm_msgs.h | 1 +
drivers/infiniband/core/cma.c | 1 -
drivers/infiniband/core/mad.c | 50 ++-
drivers/infiniband/core/multicast.c | 2 +-
drivers/infiniband/core/sa.h | 2 +-
drivers/infiniband/core/sa_query.c | 87 ++-
drivers/infiniband/core/smi.c | 16 +-
drivers/infiniband/core/smi.h | 2 +
drivers/infiniband/core/sysfs.c | 2 +-
drivers/infiniband/core/ucm.c | 1 -
drivers/infiniband/core/umem.c | 1 +
drivers/infiniband/hw/amso1100/Kconfig | 2 +-
drivers/infiniband/hw/cxgb3/Kconfig | 2 +-
drivers/infiniband/hw/cxgb3/cxio_hal.c | 6 +-
drivers/infiniband/hw/cxgb3/cxio_wr.h | 3 +-
drivers/infiniband/hw/cxgb3/iwch_cm.c | 108 ++--
drivers/infiniband/hw/cxgb3/iwch_cm.h | 1 +
drivers/infiniband/hw/cxgb3/iwch_provider.c | 7 +-
drivers/infiniband/hw/cxgb3/iwch_qp.c | 7 +-
drivers/infiniband/hw/ehca/Kconfig | 2 +-
drivers/infiniband/hw/ehca/ehca_av.c | 6 +-
drivers/infiniband/hw/ehca/ehca_classes.h | 75 ++-
drivers/infiniband/hw/ehca/ehca_classes_pSeries.h | 4 +-
drivers/infiniband/hw/ehca/ehca_cq.c | 50 +-
drivers/infiniband/hw/ehca/ehca_hca.c | 61 ++-
drivers/infiniband/hw/ehca/ehca_irq.c | 140 +++--
drivers/infiniband/hw/ehca/ehca_irq.h | 1 -
drivers/infiniband/hw/ehca/ehca_iverbs.h | 18 +
drivers/infiniband/hw/ehca/ehca_main.c | 98 +++-
drivers/infiniband/hw/ehca/ehca_qp.c | 751 +++++++++++++++------
drivers/infiniband/hw/ehca/ehca_reqs.c | 85 ++-
drivers/infiniband/hw/ehca/ehca_tools.h | 1 +
drivers/infiniband/hw/ehca/ehca_uverbs.c | 13 +-
drivers/infiniband/hw/ehca/hcp_if.c | 58 +-
drivers/infiniband/hw/ehca/hcp_if.h | 1 -
drivers/infiniband/hw/ehca/hipz_hw.h | 19 +
drivers/infiniband/hw/ehca/ipz_pt_fn.h | 28 +-
drivers/infiniband/hw/ipath/Kconfig | 2 +-
drivers/infiniband/hw/ipath/ipath_common.h | 33 +-
drivers/infiniband/hw/ipath/ipath_cq.c | 7 +-
drivers/infiniband/hw/ipath/ipath_debug.h | 2 +-
drivers/infiniband/hw/ipath/ipath_diag.c | 41 +-
drivers/infiniband/hw/ipath/ipath_driver.c | 187 +++++-
drivers/infiniband/hw/ipath/ipath_eeprom.c | 303 ++++++++-
drivers/infiniband/hw/ipath/ipath_file_ops.c | 205 ++++--
drivers/infiniband/hw/ipath/ipath_fs.c | 9 +-
drivers/infiniband/hw/ipath/ipath_iba6110.c | 101 ++--
drivers/infiniband/hw/ipath/ipath_iba6120.c | 92 ++-
drivers/infiniband/hw/ipath/ipath_init_chip.c | 26 +-
drivers/infiniband/hw/ipath/ipath_intr.c | 141 ++++-
drivers/infiniband/hw/ipath/ipath_kernel.h | 85 +++-
drivers/infiniband/hw/ipath/ipath_keys.c | 2 +-
drivers/infiniband/hw/ipath/ipath_layer.c | 2 +-
drivers/infiniband/hw/ipath/ipath_layer.h | 2 +-
drivers/infiniband/hw/ipath/ipath_mad.c | 11 +-
drivers/infiniband/hw/ipath/ipath_mmap.c | 2 +-
drivers/infiniband/hw/ipath/ipath_mr.c | 2 +-
drivers/infiniband/hw/ipath/ipath_qp.c | 19 +-
drivers/infiniband/hw/ipath/ipath_rc.c | 116 +++-
drivers/infiniband/hw/ipath/ipath_registers.h | 2 +-
drivers/infiniband/hw/ipath/ipath_ruc.c | 36 +-
drivers/infiniband/hw/ipath/ipath_srq.c | 4 +-
drivers/infiniband/hw/ipath/ipath_stats.c | 25 +-
drivers/infiniband/hw/ipath/ipath_sysfs.c | 43 ++-
drivers/infiniband/hw/ipath/ipath_uc.c | 9 +-
drivers/infiniband/hw/ipath/ipath_ud.c | 6 +-
drivers/infiniband/hw/ipath/ipath_user_pages.c | 2 +-
drivers/infiniband/hw/ipath/ipath_verbs.c | 29 +-
drivers/infiniband/hw/ipath/ipath_verbs.h | 3 +-
drivers/infiniband/hw/ipath/ipath_verbs_mcast.c | 2 +-
drivers/infiniband/hw/ipath/ipath_wc_ppc64.c | 2 +-
drivers/infiniband/hw/ipath/ipath_wc_x86_64.c | 29 +-
drivers/infiniband/hw/mlx4/Kconfig | 1 -
drivers/infiniband/hw/mlx4/main.c | 6 +-
drivers/infiniband/hw/mlx4/mlx4_ib.h | 4 +
drivers/infiniband/hw/mlx4/qp.c | 137 ++++
drivers/infiniband/hw/mlx4/srq.c | 18 +
drivers/infiniband/hw/mthca/Kconfig | 2 +-
drivers/infiniband/hw/mthca/mthca_allocator.c | 2 +-
drivers/infiniband/hw/mthca/mthca_eq.c | 2 +-
drivers/infiniband/ulp/ipoib/Kconfig | 2 +-
drivers/infiniband/ulp/ipoib/ipoib_cm.c | 4 +-
drivers/infiniband/ulp/ipoib/ipoib_ib.c | 33 +-
drivers/infiniband/ulp/iser/Kconfig | 2 +-
drivers/infiniband/ulp/srp/Kconfig | 2 +-
drivers/net/cxgb3/version.h | 2 +-
drivers/net/mlx4/fw.c | 3 +
drivers/net/mlx4/fw.h | 1 +
drivers/net/mlx4/main.c | 1 +
drivers/net/mlx4/mlx4.h | 1 +
drivers/net/mlx4/qp.c | 21 +
drivers/net/mlx4/srq.c | 30 +
include/linux/mlx4/device.h | 2 +
include/linux/mlx4/qp.h | 3 +
include/rdma/ib_cm.h | 1 -
include/rdma/ib_mad.h | 3 +
100 files changed, 2812 insertions(+), 1061 deletions(-)
^ permalink raw reply [flat|nested] 31+ messages in thread* Further 2.6.23 merge plans... 2007-07-12 23:07 [GIT PULL] please pull infiniband.git Roland Dreier @ 2007-07-12 23:15 ` Roland Dreier 2007-07-13 0:17 ` [ofa-general] " Hal Rosenstock ` (5 more replies) 0 siblings, 6 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-12 23:15 UTC (permalink / raw) To: linux-kernel, general As you can see, I just sent my first 2.6.23 pull request for Linus. There are still a few more things I plan to do in before the merge window closes (in ~10 days): - Write a patch to add P_Key handling to user_mad in the way we discussed (add an ioctl to enable P_Key mode without breaking old apps) -- I hope to do this tomorrow so we can get some review and testing before merging it. - Take a look at Sean's local SA caching patches. I merged everything else from Sean's tree, but I'm still undecided about these. I haven't read them carefully yet, but even aside from that I don't have a good feeling about whether there's consensus about this yet. Any opinions about merging, for or against, would be appreciated here. - Merge up pending hardware driver changes, including the cxgb3 and ehca patches I have in my queue, plus Jack's catastrophic error patch for mlx4. - Try to get to resolution on the IPoIB "CM without SRQ" solution. Also, if there's something I didn't list and didn't already include in the tree I asked Linus to pull, please remind me. I probably dropped it. - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier @ 2007-07-13 0:17 ` Hal Rosenstock 2007-07-13 1:14 ` Sean Hefty ` (4 subsequent siblings) 5 siblings, 0 replies; 31+ messages in thread From: Hal Rosenstock @ 2007-07-13 0:17 UTC (permalink / raw) To: Roland Dreier; +Cc: linux-kernel, general On Thu, 2007-07-12 at 19:15, Roland Dreier wrote: > As you can see, I just sent my first 2.6.23 pull request for Linus. > There are still a few more things I plan to do in before the merge > window closes (in ~10 days): > > - Write a patch to add P_Key handling to user_mad in the way we > discussed (add an ioctl to enable P_Key mode without breaking old > apps) -- I hope to do this tomorrow so we can get some review and > testing before merging it. Unfortunately, I'll mostly just be able to review it. Not sure how much testing I will be able to do but we'll see... -- Hal > - Take a look at Sean's local SA caching patches. I merged > everything else from Sean's tree, but I'm still undecided about > these. I haven't read them carefully yet, but even aside from that > I don't have a good feeling about whether there's consensus about > this yet. Any opinions about merging, for or against, would be > appreciated here. > > - Merge up pending hardware driver changes, including the cxgb3 and > ehca patches I have in my queue, plus Jack's catastrophic error > patch for mlx4. > > - Try to get to resolution on the IPoIB "CM without SRQ" solution. > > Also, if there's something I didn't list and didn't already include in > the tree I asked Linus to pull, please remind me. I probably dropped it. > > - R. > _______________________________________________ > general mailing list > general@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier 2007-07-13 0:17 ` [ofa-general] " Hal Rosenstock @ 2007-07-13 1:14 ` Sean Hefty [not found] ` <15ddcffd0707172020j5b68fcb2v7d3ca77863998020@mail.gmail.com> 2007-07-13 5:47 ` Michael S. Tsirkin ` (3 subsequent siblings) 5 siblings, 1 reply; 31+ messages in thread From: Sean Hefty @ 2007-07-13 1:14 UTC (permalink / raw) To: Roland Dreier; +Cc: linux-kernel, general > - Take a look at Sean's local SA caching patches. I merged > everything else from Sean's tree, but I'm still undecided about > these. I haven't read them carefully yet, but even aside from that > I don't have a good feeling about whether there's consensus about > this yet. Any opinions about merging, for or against, would be > appreciated here. Obviously I'm biased here, but we've definitely seen local caching of path records (PR) greatly improve performance for large MPI job runs. (Our largest jobs wouldn't run without it.) The development of the feature was requested and paid for by the US national labs. Infinicon/Silverstorm/QLogic also had this feature in their IB stack for scalability reasons as well. PR caching is done in the stack today by IPoIB. The implementation is hidden under the current kernel ib_sa interface, is disabled by default, and automatically fails over to standard PR queries if needed. Removing the cache later should be fairly easy. But to be fair, it will be difficult to enable both QoS and local PR caching. To me, this would be the strongest reason against using it. However, QoS places additional burden on the SA, which will make scaling even more challenging. - Sean ^ permalink raw reply [flat|nested] 31+ messages in thread
[parent not found: <15ddcffd0707172020j5b68fcb2v7d3ca77863998020@mail.gmail.com>]
* Re: [ofa-general] Further 2.6.23 merge plans... [not found] ` <15ddcffd0707172020j5b68fcb2v7d3ca77863998020@mail.gmail.com> @ 2007-07-18 3:23 ` Roland Dreier 2007-07-18 5:26 ` Sean Hefty 0 siblings, 1 reply; 31+ messages in thread From: Roland Dreier @ 2007-07-18 3:23 UTC (permalink / raw) To: Or Gerlitz; +Cc: Sean Hefty, linux-kernel, general > > But to be fair, it will be difficult to enable both QoS and local PR > > caching. To me, this would be the strongest reason against using it. > > However, QoS places additional burden on the SA, which will make scaling > > even more challenging. > > my understanding is that the local sa does a path-query where all the fields > except for the SGID are wildcard-ed. This means we expect the result to be a > table of all the paths from this port to every other port on the fabrics for > every pkey which this port is a member of etc, correct? > > How do you plug here the QoS concept of SID in the path query? are you > expecting the SA to realize what are all the services for which this port is > a "member"? does the proposed definision for QoS management at the SA > defines "services per gids" isn't it "what SL to user per Service"? Or, thanks for rescuing this post. I think this is an important question. If we merge the local SA stuff, then are we creating a problem for dealing with QoS? Are we going to have to revert the local SA stuff once the QoS stuff is available? Or is there at least a sketch of a plan on how to handle this? - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [ofa-general] Further 2.6.23 merge plans... 2007-07-18 3:23 ` Roland Dreier @ 2007-07-18 5:26 ` Sean Hefty 2007-07-18 17:05 ` Sean Hefty 0 siblings, 1 reply; 31+ messages in thread From: Sean Hefty @ 2007-07-18 5:26 UTC (permalink / raw) To: 'Roland Dreier', Or Gerlitz; +Cc: linux-kernel, general >I think this is an important question. If we merge the local SA >stuff, then are we creating a problem for dealing with QoS? Yes - I do believe that merging PR caching and QoS together will be difficult. I don't think the problems are insurmountable, but I can't say that I have a definite solution for how to deal with this. My current thoughts are that the purpose of the cache is to increase SA scalability on large clusters. We've seen issues running MPI, trying to establish all-to-all connections, on our 256 node cluster. (With 4 processes per node, this results in about 500,000+ PR queries hitting the SA.) The SA was swamped with work, and it wasn't trying to enforce QoS requirements across the cluster. I just don't see how an SA that is already having trouble scaling to this number of nodes will be able to perform the additional task of providing QoS across the cluster. It may be that, at least initially, an administrator may need to select between enabling PR caching or QoS. >Are we going to have to revert the local SA stuff once the QoS stuff is >available? In the best case, the local SA will need enhancements added to the base support. In the worst case, a user would have to choose between QoS or PR caching. If all users choose QoS, then it would make sense to remove the local SA. >Or is there at least a sketch of a plan on how to handle this? This is only a rough idea, and it depends on how the QoS is implemented. The idea is to create a local QoS module on each node. The local QoS modules would be programmed with basic QoS information. For example, which types of queries to handle locally, versus which ones to forward to the SA. Locally handled queries would return PRs based on some QoS mapping table. (I haven't looked into any details of this.) Ideally, local QoS modules would be programmed by a QoS master. This would require a new vendor-specific protocol, but would allow for a simple distributed QoS manager. We will have a better idea of the issues and possible solutions once the QoS spec is released, and we can hold discussions on it. I will be working more details on QoS enhancements starting in the next couple of weeks. - Sean ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-18 5:26 ` Sean Hefty @ 2007-07-18 17:05 ` Sean Hefty 2007-07-18 20:54 ` Sean Hefty 0 siblings, 1 reply; 31+ messages in thread From: Sean Hefty @ 2007-07-18 17:05 UTC (permalink / raw) To: Sean Hefty; +Cc: 'Roland Dreier', Or Gerlitz, linux-kernel, general > We will have a better idea of the issues and possible solutions once the QoS > spec is released, and we can hold discussions on it. I will be working more > details on QoS enhancements starting in the next couple of weeks. Based on discussions so far, maybe the best path forward from here is to delay until 2.6.24. This will let us add this version to OFED 1.3 for more widespread testing, plus give us the time that we need to come up with a plan to integrate QoS with the local SA. I don't think we'll have a final implementation for QoS support by that time, but at least we'll have a better idea of the problems. These patches are based on the same design used with OFED 1.2, but a fair number of lines of code still changed, plus it added InformInfo registration. I don't believe anyone other than me has tested these patches with the local SA enabled. It's typically running on my systems, but because it automatically fails over to standard SA queries, it would be easy for me to miss problems. - Sean ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-18 17:05 ` Sean Hefty @ 2007-07-18 20:54 ` Sean Hefty 0 siblings, 0 replies; 31+ messages in thread From: Sean Hefty @ 2007-07-18 20:54 UTC (permalink / raw) To: Sean Hefty; +Cc: 'Roland Dreier', linux-kernel, general > Based on discussions so far, maybe the best path forward from here is to > delay until 2.6.24. This will let us add this version to OFED 1.3 for > more widespread testing, plus give us the time that we need to come up > with a plan to integrate QoS with the local SA. I spoke with Matt on this, and he agreed with this plan. - Sean ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier 2007-07-13 0:17 ` [ofa-general] " Hal Rosenstock 2007-07-13 1:14 ` Sean Hefty @ 2007-07-13 5:47 ` Michael S. Tsirkin 2007-07-13 18:14 ` Roland Dreier 2007-07-13 18:56 ` [ofa-general] " Shirley Ma ` (2 subsequent siblings) 5 siblings, 1 reply; 31+ messages in thread From: Michael S. Tsirkin @ 2007-07-13 5:47 UTC (permalink / raw) To: Roland Dreier; +Cc: linux-kernel, general > Also, if there's something I didn't list and didn't already include in > the tree I asked Linus to pull, please remind me. I probably dropped it. Any plans to do something with multiple EQ support in mthca? -- MST ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-13 5:47 ` Michael S. Tsirkin @ 2007-07-13 18:14 ` Roland Dreier 2007-07-13 18:50 ` [ofa-general] " Shirley Ma 2007-07-14 17:54 ` Michael S. Tsirkin 0 siblings, 2 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-13 18:14 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: linux-kernel, general > Any plans to do something with multiple EQ support in mthca? I haven't done any work on it or seen anything from anyone else, so I expect this will have to wait for 2.6.24. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-13 18:14 ` Roland Dreier @ 2007-07-13 18:50 ` Shirley Ma 2007-07-17 18:06 ` Roland Dreier 2007-07-14 17:54 ` Michael S. Tsirkin 1 sibling, 1 reply; 31+ messages in thread From: Shirley Ma @ 2007-07-13 18:50 UTC (permalink / raw) To: Roland Dreier; +Cc: general, general-bounces, linux-kernel, Michael S. Tsirkin Hello Roland, > > Any plans to do something with multiple EQ support in mthca? > > I haven't done any work on it or seen anything from anyone else, so I > expect this will have to wait for 2.6.24. We are working on IPoIB to use multiple EQ for multiple links/connetions scalability. Does this mean this will wait for 2.6.24? Thanks Shirley ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-13 18:50 ` [ofa-general] " Shirley Ma @ 2007-07-17 18:06 ` Roland Dreier 0 siblings, 0 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-17 18:06 UTC (permalink / raw) To: Shirley Ma; +Cc: general, general-bounces, linux-kernel, Michael S. Tsirkin > We are working on IPoIB to use multiple EQ for multiple > links/connetions scalability. Does this mean this will wait for 2.6.24? I think so -- I don't want to merge something that first appears in the last few days of the merge window. The idea is to get your stuff queued up *before* the merge window opens. - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-13 18:14 ` Roland Dreier 2007-07-13 18:50 ` [ofa-general] " Shirley Ma @ 2007-07-14 17:54 ` Michael S. Tsirkin [not found] ` <OF72F6B9D1.F60C4EEF-ON8725731A.00506757-8825731A.0024BD1C@us.ibm.com> 2007-07-16 16:42 ` Roland Dreier 1 sibling, 2 replies; 31+ messages in thread From: Michael S. Tsirkin @ 2007-07-14 17:54 UTC (permalink / raw) To: Roland Dreier; +Cc: Michael S. Tsirkin, linux-kernel, general > Quoting Roland Dreier <rdreier@cisco.com>: > Subject: Re: Further 2.6.23 merge plans... > > > Any plans to do something with multiple EQ support in mthca? > > I haven't done any work on it or seen anything from anyone else, so I > expect this will have to wait for 2.6.24. I'm surprised to hear this. How about this: http://lists.openfabrics.org/pipermail/general/2007-May/035757.html -- MST ^ permalink raw reply [flat|nested] 31+ messages in thread
[parent not found: <OF72F6B9D1.F60C4EEF-ON8725731A.00506757-8825731A.0024BD1C@us.ibm.com>]
* Re: [ofa-general] Re: Further 2.6.23 merge plans... [not found] ` <OF72F6B9D1.F60C4EEF-ON8725731A.00506757-8825731A.0024BD1C@us.ibm.com> @ 2007-07-16 14:55 ` Michael S. Tsirkin 0 siblings, 0 replies; 31+ messages in thread From: Michael S. Tsirkin @ 2007-07-16 14:55 UTC (permalink / raw) To: Shirley Ma Cc: Michael S. Tsirkin, general, general-bounces, linux-kernel, Roland Dreier > Quoting Shirley Ma <xma@us.ibm.com>: > Subject: Re: [ofa-general] Re: Further 2.6.23 merge plans... > > Michael, > > I would like to try this patch for one adapter/2 ports scalability performance > for IPoIB. Is this patch appliable to OFED-1.2? Most likely yes. -- MST ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-14 17:54 ` Michael S. Tsirkin [not found] ` <OF72F6B9D1.F60C4EEF-ON8725731A.00506757-8825731A.0024BD1C@us.ibm.com> @ 2007-07-16 16:42 ` Roland Dreier 2007-07-16 20:05 ` Michael S. Tsirkin 1 sibling, 1 reply; 31+ messages in thread From: Roland Dreier @ 2007-07-16 16:42 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: linux-kernel, general > > I haven't done any work on it or seen anything from anyone else, so I > > expect this will have to wait for 2.6.24. > I'm surprised to hear this. How about this: > http://lists.openfabrics.org/pipermail/general/2007-May/035757.html Sure, I remember that. But I haven't seen anything to suggest that anyone has given any further thought to the issues that were raised in that thread. - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-16 16:42 ` Roland Dreier @ 2007-07-16 20:05 ` Michael S. Tsirkin 2007-07-17 17:53 ` Roland Dreier 0 siblings, 1 reply; 31+ messages in thread From: Michael S. Tsirkin @ 2007-07-16 20:05 UTC (permalink / raw) To: Roland Dreier; +Cc: Michael S. Tsirkin, linux-kernel, general > Quoting Roland Dreier <rdreier@cisco.com>: > Subject: Re: Further 2.6.23 merge plans... > > > > I haven't done any work on it or seen anything from anyone else, so I > > > expect this will have to wait for 2.6.24. > > > I'm surprised to hear this. How about this: > > http://lists.openfabrics.org/pipermail/general/2007-May/035757.html > > Sure, I remember that. But I haven't seen anything to suggest that > anyone has given any further thought to the issues that were raised in > that thread. Well, the only issue I recall is about the # of EQs we want to allocate. Was there something else? Maybe code can be merged as-is (2 EQs) and the number be tuned later as applications start using vectors? -- MST ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-16 20:05 ` Michael S. Tsirkin @ 2007-07-17 17:53 ` Roland Dreier 0 siblings, 0 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-17 17:53 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: linux-kernel, general > Well, the only issue I recall is about the # of EQs we want to allocate. > Was there something else? Yes, some ideas about how applications should pick which EQ to use. And how to handle CPU affinity. And whether we want to try to do something NUMA-aware. - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier ` (2 preceding siblings ...) 2007-07-13 5:47 ` Michael S. Tsirkin @ 2007-07-13 18:56 ` Shirley Ma 2007-07-16 16:47 ` Roland Dreier 2007-07-15 12:26 ` Tziporet Koren 2007-07-17 18:07 ` Roland Dreier 5 siblings, 1 reply; 31+ messages in thread From: Shirley Ma @ 2007-07-13 18:56 UTC (permalink / raw) To: Roland Dreier; +Cc: general, general-bounces, linux-kernel Hello Roland, FYI, we are working on several IPoIB performance improvement patches which are not on the list. Some of the patches are under test, some of the patches are going to be submitted soon. They are: 1. skb aggregations for both dev xmit(networking layer) and IPoIB send (it will be submitted soon, for both UD and RC mode) 2. multiple interrupt vectors in IPoIB for multiple links scalability (working on patch for both UD and RC mode) 3. split CQ and send completion aggregation (for both UD and RC mode) 4. LRO for IPoIB when generic LRO is available in networking layer. (UD mode only) Some of them might be made in 2.6.23 timeline, some of them might not, it depends on our test progress and community review feedback. Thanks Shirley ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-13 18:56 ` [ofa-general] " Shirley Ma @ 2007-07-16 16:47 ` Roland Dreier 0 siblings, 0 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-16 16:47 UTC (permalink / raw) To: Shirley Ma; +Cc: general, general-bounces, linux-kernel > FYI, we are working on several IPoIB performance improvement > patches which are not on the list. Some of the patches are under test, > some of the patches are going to be submitted soon. They are: There is less than a week left in the merge window, and none of these changes has been reviewed yet. So being realistic, I don't think we can expect to get any of this into 2.6.23. - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier ` (3 preceding siblings ...) 2007-07-13 18:56 ` [ofa-general] " Shirley Ma @ 2007-07-15 12:26 ` Tziporet Koren 2007-07-16 16:42 ` Roland Dreier 2007-07-17 18:07 ` Roland Dreier 5 siblings, 1 reply; 31+ messages in thread From: Tziporet Koren @ 2007-07-15 12:26 UTC (permalink / raw) To: Roland Dreier; +Cc: linux-kernel, general Roland Dreier wrote: > As you can see, I just sent my first 2.6.23 pull request for Linus. > There are still a few more things I plan to do in before the merge > window closes (in ~10 days): > > Till when can we insert mlx4 with FMRs? Tziporet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-15 12:26 ` Tziporet Koren @ 2007-07-16 16:42 ` Roland Dreier 0 siblings, 0 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-16 16:42 UTC (permalink / raw) To: tziporet; +Cc: linux-kernel, general > Till when can we insert mlx4 with FMRs? 2.6.22 came out on July 8, so I would expect 2.6.23-rc1 (the end of the merge window) to be July 22. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier ` (4 preceding siblings ...) 2007-07-15 12:26 ` Tziporet Koren @ 2007-07-17 18:07 ` Roland Dreier 2007-07-17 20:43 ` Matt Leininger 2007-07-17 21:44 ` Michael S. Tsirkin 5 siblings, 2 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-17 18:07 UTC (permalink / raw) To: linux-kernel; +Cc: general > - Take a look at Sean's local SA caching patches. I merged > everything else from Sean's tree, but I'm still undecided about > these. I haven't read them carefully yet, but even aside from that > I don't have a good feeling about whether there's consensus about > this yet. Any opinions about merging, for or against, would be > appreciated here. Does anyone other than Sean have an opinion here? If you want this feature, if you've tested it, if you don't think it's ready yet, whatever, please speak up -- I don't feel comfortable making a decision on my own here (although I will if I have to). ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-17 18:07 ` Roland Dreier @ 2007-07-17 20:43 ` Matt Leininger 2007-07-17 20:45 ` Roland Dreier 2007-07-17 21:44 ` Michael S. Tsirkin 1 sibling, 1 reply; 31+ messages in thread From: Matt Leininger @ 2007-07-17 20:43 UTC (permalink / raw) To: Roland Dreier; +Cc: linux-kernel, general On Tue, 2007-07-17 at 11:07 -0700, Roland Dreier wrote: > > - Take a look at Sean's local SA caching patches. I merged > > everything else from Sean's tree, but I'm still undecided about > > these. I haven't read them carefully yet, but even aside from that > > I don't have a good feeling about whether there's consensus about > > this yet. Any opinions about merging, for or against, would be > > appreciated here. > > Does anyone other than Sean have an opinion here? If you want this > feature, if you've tested it, if you don't think it's ready yet, > whatever, please speak up -- I don't feel comfortable making a > decision on my own here (although I will if I have to). Roland, I would like to see these features moved upstream. DOE funded this work as part of the items we see needing on our large scale IB deployment (both present and future). So from at least one big customer perspective we see this as useful. I'll let others comment on specific code/implementation issues. Thanks, - Matt > _______________________________________________ > general mailing list > general@lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- Matt Leininger, Ph.D. Lawrence Livermore National Laboratory leininger2@llnl.gov V 925-422-4110 ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-17 20:43 ` Matt Leininger @ 2007-07-17 20:45 ` Roland Dreier 2007-07-18 6:36 ` Or Gerlitz 0 siblings, 1 reply; 31+ messages in thread From: Roland Dreier @ 2007-07-17 20:45 UTC (permalink / raw) To: leininger2; +Cc: linux-kernel, general > I would like to see these features moved upstream. DOE funded this > work as part of the items we see needing on our large scale IB > deployment (both present and future). So from at least one big customer > perspective we see this as useful. Does your reference to "present deployment" mean you are running this code now? - R. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Further 2.6.23 merge plans... 2007-07-17 20:45 ` Roland Dreier @ 2007-07-18 6:36 ` Or Gerlitz 0 siblings, 0 replies; 31+ messages in thread From: Or Gerlitz @ 2007-07-18 6:36 UTC (permalink / raw) To: Roland Dreier; +Cc: leininger2, linux-kernel, general Roland Dreier wrote: > > I would like to see these features moved upstream. DOE funded this > > work as part of the items we see needing on our large scale IB > > deployment (both present and future). So from at least one big customer > > perspective we see this as useful. > > Does your reference to "present deployment" mean you are running this > code now? Indeed, my understanding is that the DOE uses an Open MPI device (I think its called PTE) which is implemented directly over libibverbs and hence no path queries are issued at all, if this is indeed the case, for them its more of a "for-the-future" thing. Or. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Further 2.6.23 merge plans... 2007-07-17 18:07 ` Roland Dreier 2007-07-17 20:43 ` Matt Leininger @ 2007-07-17 21:44 ` Michael S. Tsirkin 2007-07-18 7:34 ` [ofa-general] " Tziporet Koren 1 sibling, 1 reply; 31+ messages in thread From: Michael S. Tsirkin @ 2007-07-17 21:44 UTC (permalink / raw) To: Roland Dreier; +Cc: linux-kernel, general > Quoting Roland Dreier <rdreier@cisco.com>: > Subject: Re: Further 2.6.23 merge plans... > > > - Take a look at Sean's local SA caching patches. I merged > > everything else from Sean's tree, but I'm still undecided about > > these. I haven't read them carefully yet, but even aside from that > > I don't have a good feeling about whether there's consensus about > > this yet. Any opinions about merging, for or against, would be > > appreciated here. > > Does anyone other than Sean have an opinion here? If you want this > feature, if you've tested it, if you don't think it's ready yet, > whatever, please speak up -- I don't feel comfortable making a > decision on my own here (although I will if I have to). We have the patches applied in ofed 1.2.c with default module parameter set to caching disabled (ofed 1.2 had a different version of the patches, but caching is disabled by default there, too). At least in this configuration (caching disabled), all issues I've seen seem to be fixed now, and tests seem to be running smoothly. So I think it's safe to merge it up if the module parameter is set to cache disabled by default. No idea what happens if it's enabled though :) -- MST ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-17 21:44 ` Michael S. Tsirkin @ 2007-07-18 7:34 ` Tziporet Koren 2007-07-18 7:38 ` Michael S. Tsirkin 0 siblings, 1 reply; 31+ messages in thread From: Tziporet Koren @ 2007-07-18 7:34 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: Roland Dreier, linux-kernel, general Michael S. Tsirkin wrote: > We have the patches applied in ofed 1.2.c with default module parameter set to > caching disabled (ofed 1.2 had a different version of the patches, but caching > is disabled by default there, too). At least in this configuration > (caching disabled), all issues I've seen seem to be fixed now, and tests seem to > be running smoothly. > As far as I know Intel run with SA cache enabled on large clusters with Intel MPI Tziporet ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-18 7:34 ` [ofa-general] " Tziporet Koren @ 2007-07-18 7:38 ` Michael S. Tsirkin 2007-07-18 8:48 ` Tziporet Koren 0 siblings, 1 reply; 31+ messages in thread From: Michael S. Tsirkin @ 2007-07-18 7:38 UTC (permalink / raw) To: Tziporet Koren; +Cc: Michael S. Tsirkin, Roland Dreier, linux-kernel, general > Quoting Tziporet Koren <tziporet@dev.mellanox.co.il>: > Subject: Re: [ofa-general] Re: Further 2.6.23 merge plans... > > Michael S. Tsirkin wrote: > >We have the patches applied in ofed 1.2.c with default module parameter set > >to caching disabled (ofed 1.2 had a different version of the patches, but > >caching is disabled by default there, too). At least > >in this configuration (caching disabled), all issues I've seen seem to be > >fixed now, and tests seem to be running smoothly. > > As far as I know Intel run with SA cache enabled on large clusters with > Intel MPI With OFED 1.2 version of the code, right? -- MST ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-18 7:38 ` Michael S. Tsirkin @ 2007-07-18 8:48 ` Tziporet Koren 2007-07-18 16:16 ` Sean Hefty 0 siblings, 1 reply; 31+ messages in thread From: Tziporet Koren @ 2007-07-18 8:48 UTC (permalink / raw) To: Michael S. Tsirkin; +Cc: Tziporet Koren, Roland Dreier, linux-kernel, general Michael S. Tsirkin wrote: >> As far as I know Intel run with SA cache enabled on large clusters with >> Intel MPI >> > > With OFED 1.2 version of the code, right? > > Yes. But maybe they also used the new module - Sean? ^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-18 8:48 ` Tziporet Koren @ 2007-07-18 16:16 ` Sean Hefty 2007-07-18 16:20 ` Roland Dreier 0 siblings, 1 reply; 31+ messages in thread From: Sean Hefty @ 2007-07-18 16:16 UTC (permalink / raw) To: 'Tziporet Koren', Michael S. Tsirkin Cc: Roland Dreier, general, linux-kernel >> With OFED 1.2 version of the code, right? >> >> >Yes. >But maybe they also used the new module - Sean? We actually use the OFED 1.2 version. So, this feature is in use, but not this specific implementation. - Sean ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [ofa-general] Re: Further 2.6.23 merge plans... 2007-07-18 16:16 ` Sean Hefty @ 2007-07-18 16:20 ` Roland Dreier 0 siblings, 0 replies; 31+ messages in thread From: Roland Dreier @ 2007-07-18 16:20 UTC (permalink / raw) To: Sean Hefty Cc: 'Tziporet Koren', Michael S. Tsirkin, general, linux-kernel > We actually use the OFED 1.2 version. So, this feature is in use, but not this > specific implementation. Hmm... how much testing has the implementation being proposed for merging actually had? It might still be OK if the answer is that it hasn't been tested at scale but that the basic code works and should behave the same as the code that was tested because the underlying design is the same... is at least that much true? ^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2007-07-18 20:54 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-07-12 23:07 [GIT PULL] please pull infiniband.git Roland Dreier
2007-07-12 23:15 ` Further 2.6.23 merge plans Roland Dreier
2007-07-13 0:17 ` [ofa-general] " Hal Rosenstock
2007-07-13 1:14 ` Sean Hefty
[not found] ` <15ddcffd0707172020j5b68fcb2v7d3ca77863998020@mail.gmail.com>
2007-07-18 3:23 ` Roland Dreier
2007-07-18 5:26 ` Sean Hefty
2007-07-18 17:05 ` Sean Hefty
2007-07-18 20:54 ` Sean Hefty
2007-07-13 5:47 ` Michael S. Tsirkin
2007-07-13 18:14 ` Roland Dreier
2007-07-13 18:50 ` [ofa-general] " Shirley Ma
2007-07-17 18:06 ` Roland Dreier
2007-07-14 17:54 ` Michael S. Tsirkin
[not found] ` <OF72F6B9D1.F60C4EEF-ON8725731A.00506757-8825731A.0024BD1C@us.ibm.com>
2007-07-16 14:55 ` [ofa-general] " Michael S. Tsirkin
2007-07-16 16:42 ` Roland Dreier
2007-07-16 20:05 ` Michael S. Tsirkin
2007-07-17 17:53 ` Roland Dreier
2007-07-13 18:56 ` [ofa-general] " Shirley Ma
2007-07-16 16:47 ` Roland Dreier
2007-07-15 12:26 ` Tziporet Koren
2007-07-16 16:42 ` Roland Dreier
2007-07-17 18:07 ` Roland Dreier
2007-07-17 20:43 ` Matt Leininger
2007-07-17 20:45 ` Roland Dreier
2007-07-18 6:36 ` Or Gerlitz
2007-07-17 21:44 ` Michael S. Tsirkin
2007-07-18 7:34 ` [ofa-general] " Tziporet Koren
2007-07-18 7:38 ` Michael S. Tsirkin
2007-07-18 8:48 ` Tziporet Koren
2007-07-18 16:16 ` Sean Hefty
2007-07-18 16:20 ` Roland Dreier
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).