From: James Simmons <jsimmons@infradead.org> To: lustre-devel@lists.lustre.org Subject: [lustre-devel] [PATCH 14/22] lnet: Force full discovery cycle Date: Tue, 2 Jun 2020 20:59:53 -0400 Message-ID: <1591146001-27171-15-git-send-email-jsimmons@infradead.org> (raw) In-Reply-To: <1591146001-27171-1-git-send-email-jsimmons@infradead.org> From: Amir Shehata <ashehata@whamcloud.com> There are scenarios where there could be a discrepancy between cached peer information and reality. In these cases what could end-up happening is incomplete interface information might be cached because one side determined that the peer didn't require a PUSH. This will lead to undesired MR behavior, where not all the interfaces are used for a period of time. Therefore, it is safer to always force a full discovery cycle: GET/PUSH to ensure both sides are up-to-date. In the NMR case, when discovery is turned off, make sure to flag discovery as complete to avoid stalling the state machine. WC-bug-id: https://jira.whamcloud.com/browse/LU-13477 Lustre-commit: 9bafd530d7858 ("LU-13477 lnet: Force full discovery cycle") Signed-off-by: Amir Shehata <ashehata@whamcloud.com> Reviewed-on: https://review.whamcloud.com/38322 Reviewed-by: Chris Horn <chris.horn@hpe.com> Reviewed-by: Serguei Smirnov <ssmirnov@whamcloud.com> Reviewed-by: Oleg Drokin <green@whamcloud.com> Signed-off-by: James Simmons <jsimmons@infradead.org> --- net/lnet/lnet/peer.c | 40 ++++++++++++++++++++++++---------------- net/lnet/lnet/router.c | 5 +++-- 2 files changed, 27 insertions(+), 18 deletions(-) diff --git a/net/lnet/lnet/peer.c b/net/lnet/lnet/peer.c index 60749da..5869a20 100644 --- a/net/lnet/lnet/peer.c +++ b/net/lnet/lnet/peer.c @@ -1179,6 +1179,11 @@ struct lnet_peer_ni * lp = lpni->lpni_peer_net->lpn_peer; while (!lnet_peer_is_uptodate(lp)) { + spin_lock(&lp->lp_lock); + /* force a full discovery cycle */ + lp->lp_state |= LNET_PEER_FORCE_PING | LNET_PEER_FORCE_PUSH; + spin_unlock(&lp->lp_lock); + rc = lnet_discover_peer_locked(lpni, cpt, true); if (rc) goto out_decref; @@ -3043,6 +3048,19 @@ static int lnet_peer_push_failed(struct lnet_peer *lp) return rc ? rc : LNET_REDISCOVER_PEER; } +/* Mark the peer as discovered. */ +static int lnet_peer_discovered(struct lnet_peer *lp) +__must_hold(&lp->lp_lock) +{ + lp->lp_state |= LNET_PEER_DISCOVERED; + lp->lp_state &= ~(LNET_PEER_DISCOVERING | + LNET_PEER_REDISCOVER); + + CDEBUG(D_NET, "peer %s\n", libcfs_nid2str(lp->lp_primary_nid)); + + return 0; +} + /* Active side of push. */ static int lnet_peer_send_push(struct lnet_peer *lp) __must_hold(&lp->lp_lock) @@ -3056,6 +3074,12 @@ static int lnet_peer_send_push(struct lnet_peer *lp) /* Don't push to a non-multi-rail peer. */ if (!(lp->lp_state & LNET_PEER_MULTI_RAIL)) { lp->lp_state &= ~LNET_PEER_FORCE_PUSH; + /* if peer's NIDs are uptodate then peer is discovered */ + if (lp->lp_state & LNET_PEER_NIDS_UPTODATE) { + rc = lnet_peer_discovered(lp); + return rc; + } + return 0; } @@ -3147,22 +3171,6 @@ static void lnet_peer_discovery_error(struct lnet_peer *lp, int error) } /* - * Mark the peer as discovered. - */ -static int lnet_peer_discovered(struct lnet_peer *lp) -__must_hold(&lp->lp_lock) -{ - lp->lp_state |= LNET_PEER_DISCOVERED; - lp->lp_state &= ~(LNET_PEER_DISCOVERING | - LNET_PEER_REDISCOVER); - - CDEBUG(D_NET, "peer %s\n", libcfs_nid2str(lp->lp_primary_nid)); - - return 0; -} - - -/* * Discovering this peer is taking too long. Cancel any Ping or Push * that discovery is waiting on by unlinking the relevant MDs. The * lnet_discovery_event_handler() will proceed from here and complete diff --git a/net/lnet/lnet/router.c b/net/lnet/lnet/router.c index af50e51..903d027 100644 --- a/net/lnet/lnet/router.c +++ b/net/lnet/lnet/router.c @@ -1154,9 +1154,10 @@ bool lnet_router_checker_active(void) spin_unlock(&rtr->lp_lock); continue; } - /* make sure we actively discover the router */ + /* make sure we fully discover the router */ rtr->lp_state &= ~LNET_PEER_NIDS_UPTODATE; - rtr->lp_state |= LNET_PEER_RTR_DISCOVERY; + rtr->lp_state |= LNET_PEER_FORCE_PING | LNET_PEER_FORCE_PUSH | + LNET_PEER_RTR_DISCOVERY; spin_unlock(&rtr->lp_lock); /* find the peer_ni associated with the primary NID */ -- 1.8.3.1
next prev parent reply index Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-03 0:59 [lustre-devel] [PATCH 00/22] lustre: OpenSFS backport patches for May 29 2020 James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 01/22] lnet: libcfs: fix CPT handling for UP systems James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 02/22] lustre: use BIT() macro where appropriate in include James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 03/22] lustre: use BIT() macro where appropriate James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 04/22] lustre: ptlrpc: change LONG_UNLINK to PTLRPC_REQ_LONG_UNLINK James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 05/22] lustre: llite: use %pd to report dentry names James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 06/22] lnet: tidy lnet_discover and fix mem accounting bug James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 07/22] lustre: llite: prevent MAX_DIO_SIZE 32-bit truncation James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 08/22] lustre: llite: integrate statx() API with Lustre James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 09/22] lustre: ldlm: no current source if lu_ref_del not in same tsk James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 10/22] lnet: always pass struct lnet_md by reference James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 11/22] lustre: llite: fix read if readahead window smaller than rpc size James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 12/22] lustre: obdclass: bind zombie export cleanup workqueue James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 13/22] lnet: handle discovery off properly James Simmons 2020-06-03 0:59 ` James Simmons [this message] 2020-06-03 0:59 ` [lustre-devel] [PATCH 15/22] lnet: set route aliveness properly James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 16/22] lnet: Correct the default LND timeout James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 17/22] lnet: Add lnet_lnd_timeout to sysfs James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 18/22] lnet: lnd: Allow independent ko2iblnd timeout James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 19/22] lnet: lnd: Allow independent socklnd timeout James Simmons 2020-06-03 0:59 ` [lustre-devel] [PATCH 20/22] lnet: lnd: gracefully handle unexpected events James Simmons 2020-06-03 1:00 ` [lustre-devel] [PATCH 21/22] lustre: update version to 2.13.54 James Simmons 2020-06-03 1:00 ` [lustre-devel] [PATCH 22/22] lnet: procs: print new line based on distro James Simmons
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1591146001-27171-15-git-send-email-jsimmons@infradead.org \ --to=jsimmons@infradead.org \ --cc=lustre-devel@lists.lustre.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Lustre-devel archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/lustre-devel/0 lustre-devel/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 lustre-devel lustre-devel/ https://lore.kernel.org/lustre-devel \ lustre-devel@lists.lustre.org public-inbox-index lustre-devel Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.lustre.lists.lustre-devel AGPL code for this site: git clone https://public-inbox.org/public-inbox.git