* [RFC] NFSd laundromat containerization @ 2012-05-11 13:50 Stanislav Kinsbursky 2012-05-11 13:53 ` bfields 0 siblings, 1 reply; 12+ messages in thread From: Stanislav Kinsbursky @ 2012-05-11 13:50 UTC (permalink / raw) To: bfields; +Cc: linux-nfs, Jeff Layton Hello. I'm currently looking on NFSd laundromat work, and it looks like have to be performed per networks namespace context. It's easy to make corresponding delayed work per network namespace and thus gain per-net data pointer in laundromat function. But here a problem appears: network namespace is required to skip clients from other network namespaces while iterating over global lists (client_lru and friends). I see two possible solutions: 1) Make these list per network namespace context. In this case network namespace will not be required - per-net data will be enough. 2) Put network namespace link on per-net data (this one is easier, but uglier). Would be appreciated for any comments. -- Best regards, Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-11 13:50 [RFC] NFSd laundromat containerization Stanislav Kinsbursky @ 2012-05-11 13:53 ` bfields 2012-05-11 14:02 ` Jeff Layton 2012-05-12 8:59 ` Stanislav Kinsbursky 0 siblings, 2 replies; 12+ messages in thread From: bfields @ 2012-05-11 13:53 UTC (permalink / raw) To: Stanislav Kinsbursky; +Cc: linux-nfs, Jeff Layton On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: > Hello. > I'm currently looking on NFSd laundromat work, and it looks like > have to be performed per networks namespace context. > It's easy to make corresponding delayed work per network namespace > and thus gain per-net data pointer in laundromat function. > But here a problem appears: network namespace is required to skip > clients from other network namespaces while iterating over global > lists (client_lru and friends). > I see two possible solutions: > 1) Make these list per network namespace context. In this case > network namespace will not be required - per-net data will be > enough. > 2) Put network namespace link on per-net data (this one is easier, but uglier). I'd rather there be as few shared data structures between network namespaces as possible--I think that will simplify things. So, of those two choices, #1. --b. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-11 13:53 ` bfields @ 2012-05-11 14:02 ` Jeff Layton 2012-05-11 14:15 ` Stanislav Kinsbursky 2012-05-12 8:59 ` Stanislav Kinsbursky 1 sibling, 1 reply; 12+ messages in thread From: Jeff Layton @ 2012-05-11 14:02 UTC (permalink / raw) To: bfields; +Cc: Stanislav Kinsbursky, linux-nfs On Fri, 11 May 2012 09:53:07 -0400 "bfields@fieldses.org" <bfields@fieldses.org> wrote: > On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: > > Hello. > > I'm currently looking on NFSd laundromat work, and it looks like > > have to be performed per networks namespace context. > > It's easy to make corresponding delayed work per network namespace > > and thus gain per-net data pointer in laundromat function. > > But here a problem appears: network namespace is required to skip > > clients from other network namespaces while iterating over global > > lists (client_lru and friends). > > I see two possible solutions: > > 1) Make these list per network namespace context. In this case > > network namespace will not be required - per-net data will be > > enough. > > 2) Put network namespace link on per-net data (this one is easier, but uglier). > > I'd rather there be as few shared data structures between network > namespaces as possible--I think that will simplify things. > > So, of those two choices, #1. > Agreed, that's sort of how I envisioned things going. In general, you'll want to move things that were one global structures to struct nfsd_net, and fix up the code to manage that on a per-ns basis. The catch here is that the laundromat is somewhat intertwined with the grace period, and you need to consider how to handle the grace period between different namespaces. Do we keep a single grace period per-machine as we have today? Or do we move to a per-ns grace period that is started whenever someone starts up knfsd within the container? -- Jeff Layton <jlayton@redhat.com> ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-11 14:02 ` Jeff Layton @ 2012-05-11 14:15 ` Stanislav Kinsbursky 2012-05-11 15:09 ` bfields 0 siblings, 1 reply; 12+ messages in thread From: Stanislav Kinsbursky @ 2012-05-11 14:15 UTC (permalink / raw) To: Jeff Layton; +Cc: bfields, linux-nfs On 11.05.2012 18:02, Jeff Layton wrote: > On Fri, 11 May 2012 09:53:07 -0400 > "bfields@fieldses.org"<bfields@fieldses.org> wrote: > >> On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: >>> Hello. >>> I'm currently looking on NFSd laundromat work, and it looks like >>> have to be performed per networks namespace context. >>> It's easy to make corresponding delayed work per network namespace >>> and thus gain per-net data pointer in laundromat function. >>> But here a problem appears: network namespace is required to skip >>> clients from other network namespaces while iterating over global >>> lists (client_lru and friends). >>> I see two possible solutions: >>> 1) Make these list per network namespace context. In this case >>> network namespace will not be required - per-net data will be >>> enough. >>> 2) Put network namespace link on per-net data (this one is easier, but uglier). >> >> I'd rather there be as few shared data structures between network >> namespaces as possible--I think that will simplify things. >> >> So, of those two choices, #1. >> > > Agreed, that's sort of how I envisioned things going. In general, > you'll want to move things that were one global structures to struct > nfsd_net, and fix up the code to manage that on a per-ns basis. > Ok. I'll do it in first way. Anyway, this patch set with laundromat will depends on grace period containerization. > The catch here is that the laundromat is somewhat intertwined with the > grace period, and you need to consider how to handle the grace period > between different namespaces. Do we keep a single grace period > per-machine as we have today? Or do we move to a per-ns grace period > that is started whenever someone starts up knfsd within the container? > Second one. You can have a look at "Lockd: grace period containerization" patch set I've sent last week. -- Best regards, Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-11 14:15 ` Stanislav Kinsbursky @ 2012-05-11 15:09 ` bfields 0 siblings, 0 replies; 12+ messages in thread From: bfields @ 2012-05-11 15:09 UTC (permalink / raw) To: Stanislav Kinsbursky; +Cc: Jeff Layton, linux-nfs On Fri, May 11, 2012 at 06:15:31PM +0400, Stanislav Kinsbursky wrote: > On 11.05.2012 18:02, Jeff Layton wrote: > >On Fri, 11 May 2012 09:53:07 -0400 > >"bfields@fieldses.org"<bfields@fieldses.org> wrote: > > > >>On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: > >>>Hello. > >>>I'm currently looking on NFSd laundromat work, and it looks like > >>>have to be performed per networks namespace context. > >>>It's easy to make corresponding delayed work per network namespace > >>>and thus gain per-net data pointer in laundromat function. > >>>But here a problem appears: network namespace is required to skip > >>>clients from other network namespaces while iterating over global > >>>lists (client_lru and friends). > >>>I see two possible solutions: > >>>1) Make these list per network namespace context. In this case > >>>network namespace will not be required - per-net data will be > >>>enough. > >>>2) Put network namespace link on per-net data (this one is easier, but uglier). > >> > >>I'd rather there be as few shared data structures between network > >>namespaces as possible--I think that will simplify things. > >> > >>So, of those two choices, #1. > >> > > > >Agreed, that's sort of how I envisioned things going. In general, > >you'll want to move things that were one global structures to struct > >nfsd_net, and fix up the code to manage that on a per-ns basis. > > > > Ok. I'll do it in first way. > Anyway, this patch set with laundromat will depends on grace period > containerization. > > >The catch here is that the laundromat is somewhat intertwined with the > >grace period, and you need to consider how to handle the grace period > >between different namespaces. Do we keep a single grace period > >per-machine as we have today? Or do we move to a per-ns grace period > >that is started whenever someone starts up knfsd within the container? > > > > Second one. You can have a look at "Lockd: grace period > containerization" patch set I've sent last week. I think that makes the most sense. It allows people to shoot themselves in the foot in the case they're exporting the same content from two different containers. We'll need to figure out some way to help them. But one of the common use cases seems to be exporting a set of filesystems where one filesystem is exported by only one server at a time, but people want to be able to move that one filesystem export from one server to another, and this solves that problem. For the first pass at this let's keep the servers as separate as possible. --b. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-11 13:53 ` bfields 2012-05-11 14:02 ` Jeff Layton @ 2012-05-12 8:59 ` Stanislav Kinsbursky 2012-05-12 14:16 ` bfields 1 sibling, 1 reply; 12+ messages in thread From: Stanislav Kinsbursky @ 2012-05-12 8:59 UTC (permalink / raw) To: bfields; +Cc: linux-nfs, Jeff Layton On 11.05.2012 17:53, bfields@fieldses.org wrote: > On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: >> Hello. >> I'm currently looking on NFSd laundromat work, and it looks like >> have to be performed per networks namespace context. >> It's easy to make corresponding delayed work per network namespace >> and thus gain per-net data pointer in laundromat function. >> But here a problem appears: network namespace is required to skip >> clients from other network namespaces while iterating over global >> lists (client_lru and friends). >> I see two possible solutions: >> 1) Make these list per network namespace context. In this case >> network namespace will not be required - per-net data will be >> enough. >> 2) Put network namespace link on per-net data (this one is easier, but uglier). > > I'd rather there be as few shared data structures between network > namespaces as possible--I think that will simplify things. > > So, of those two choices, #1. > Guys, I would like to discuss few ideas about caches and lists containerization. Currently, it look to me, that these hash tables: reclaim_str, conf_id, conf_str, unconf_str, unconf_id, sessionid and these lists: client_lru, close_lru have to be per net, while hash tables file, ownerstr, lockowner_ino and del_recall_lru lists have not, because they are about file system access. If I'd containerize it this way, then looks like nfs4_lock_state() and nfs4_unlock_state() functions will protect only non-containerized data, while containerized data have to protected by some per-net lock. How this approach looks to you? -- Best regards, Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-12 8:59 ` Stanislav Kinsbursky @ 2012-05-12 14:16 ` bfields 2012-05-12 14:40 ` Stanislav Kinsbursky 2012-05-14 9:00 ` Stanislav Kinsbursky 0 siblings, 2 replies; 12+ messages in thread From: bfields @ 2012-05-12 14:16 UTC (permalink / raw) To: Stanislav Kinsbursky; +Cc: linux-nfs, Jeff Layton On Sat, May 12, 2012 at 12:59:05PM +0400, Stanislav Kinsbursky wrote: > On 11.05.2012 17:53, bfields@fieldses.org wrote: > >On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: > >>Hello. > >>I'm currently looking on NFSd laundromat work, and it looks like > >>have to be performed per networks namespace context. > >>It's easy to make corresponding delayed work per network namespace > >>and thus gain per-net data pointer in laundromat function. > >>But here a problem appears: network namespace is required to skip > >>clients from other network namespaces while iterating over global > >>lists (client_lru and friends). > >>I see two possible solutions: > >>1) Make these list per network namespace context. In this case > >>network namespace will not be required - per-net data will be > >>enough. > >>2) Put network namespace link on per-net data (this one is easier, but uglier). > > > >I'd rather there be as few shared data structures between network > >namespaces as possible--I think that will simplify things. > > > >So, of those two choices, #1. > > > > Guys, I would like to discuss few ideas about caches and lists containerization. > Currently, it look to me, that these hash tables: > > reclaim_str, conf_id, conf_str, unconf_str, unconf_id, sessionid > > and these lists: > > client_lru, close_lru > > have to be per net, while hash tables > > file, ownerstr, lockowner_ino > > and > > del_recall_lru lists > > have not, because they are about file system access. Actually, ownerstr and lockowner_ino should also both be per-container. So it's only file and del_recall_lru that should be global. (And del_recall_lru might work either way, actually.) > If I'd containerize it this way, then looks like nfs4_lock_state() > and nfs4_unlock_state() functions will protect only > non-containerized data, while containerized data have to protected > by some per-net lock. Sounds about right. --b. > > How this approach looks to you? > > > -- > Best regards, > Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-12 14:16 ` bfields @ 2012-05-12 14:40 ` Stanislav Kinsbursky 2012-05-14 9:00 ` Stanislav Kinsbursky 1 sibling, 0 replies; 12+ messages in thread From: Stanislav Kinsbursky @ 2012-05-12 14:40 UTC (permalink / raw) To: bfields; +Cc: linux-nfs, Jeff Layton On 12.05.2012 18:16, bfields@fieldses.org wrote: > Actually, ownerstr and lockowner_ino should also both be per-container. > > So it's only file and del_recall_lru that should be global. (And > del_recall_lru might work either way, actually.) Thanks. Will do so. BTW, have you noticed the patch, which protects service lists during per-net shutdown? Title: "SUNRPC: protect service sockets lists during per-net shutdown" -- Best regards, Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-12 14:16 ` bfields 2012-05-12 14:40 ` Stanislav Kinsbursky @ 2012-05-14 9:00 ` Stanislav Kinsbursky 2012-05-14 10:19 ` Stanislav Kinsbursky 2012-05-15 13:40 ` Jeff Layton 1 sibling, 2 replies; 12+ messages in thread From: Stanislav Kinsbursky @ 2012-05-14 9:00 UTC (permalink / raw) To: bfields; +Cc: linux-nfs, Jeff Layton On 12.05.2012 18:16, bfields@fieldses.org wrote: > On Sat, May 12, 2012 at 12:59:05PM +0400, Stanislav Kinsbursky wrote: >> On 11.05.2012 17:53, bfields@fieldses.org wrote: >>> On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: >>>> Hello. >>>> I'm currently looking on NFSd laundromat work, and it looks like >>>> have to be performed per networks namespace context. >>>> It's easy to make corresponding delayed work per network namespace >>>> and thus gain per-net data pointer in laundromat function. >>>> But here a problem appears: network namespace is required to skip >>>> clients from other network namespaces while iterating over global >>>> lists (client_lru and friends). >>>> I see two possible solutions: >>>> 1) Make these list per network namespace context. In this case >>>> network namespace will not be required - per-net data will be >>>> enough. >>>> 2) Put network namespace link on per-net data (this one is easier, but uglier). >>> >>> I'd rather there be as few shared data structures between network >>> namespaces as possible--I think that will simplify things. >>> >>> So, of those two choices, #1. >>> >> >> Guys, I would like to discuss few ideas about caches and lists containerization. >> Currently, it look to me, that these hash tables: >> >> reclaim_str, conf_id, conf_str, unconf_str, unconf_id, sessionid >> >> and these lists: >> >> client_lru, close_lru >> >> have to be per net, while hash tables >> >> file, ownerstr, lockowner_ino >> >> and >> >> del_recall_lru lists >> >> have not, because they are about file system access. > > Actually, ownerstr and lockowner_ino should also both be per-container. > > So it's only file and del_recall_lru that should be global. (And > del_recall_lru might work either way, actually.) > >> If I'd containerize it this way, then looks like nfs4_lock_state() >> and nfs4_unlock_state() functions will protect only >> non-containerized data, while containerized data have to protected >> by some per-net lock. > > Sounds about right. > Bruce, Jeff, I've implemented these per-net hashes and lists (file hash and del_recall_lru remains global). But now I'm confused with locking. For example, let's consider file hash and del_recall_lru lists. It looks like they are protected by recall_lock. But in nfsd_forget_delegations() this lock is not taken. Is it a bug? If yes and recall_lock is used for file hash protection, then why do we need to protect nfsd_process_n_delegations() by nfs4_un/lock_state() calls? Actually, the problem I'm trying to solve is to replace global client_lock by per-net one where possible. But I don't clearly understand, what it protects. Could you, guys, clarify the state locking to me. -- Best regards, Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-14 9:00 ` Stanislav Kinsbursky @ 2012-05-14 10:19 ` Stanislav Kinsbursky 2012-05-15 13:40 ` Jeff Layton 1 sibling, 0 replies; 12+ messages in thread From: Stanislav Kinsbursky @ 2012-05-14 10:19 UTC (permalink / raw) To: bfields; +Cc: linux-nfs, Jeff Layton On 14.05.2012 13:00, Stanislav Kinsbursky wrote: > On 12.05.2012 18:16, bfields@fieldses.org wrote: >> On Sat, May 12, 2012 at 12:59:05PM +0400, Stanislav Kinsbursky wrote: >>> On 11.05.2012 17:53, bfields@fieldses.org wrote: >>>> On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: >>>>> Hello. >>>>> I'm currently looking on NFSd laundromat work, and it looks like >>>>> have to be performed per networks namespace context. >>>>> It's easy to make corresponding delayed work per network namespace >>>>> and thus gain per-net data pointer in laundromat function. >>>>> But here a problem appears: network namespace is required to skip >>>>> clients from other network namespaces while iterating over global >>>>> lists (client_lru and friends). >>>>> I see two possible solutions: >>>>> 1) Make these list per network namespace context. In this case >>>>> network namespace will not be required - per-net data will be >>>>> enough. >>>>> 2) Put network namespace link on per-net data (this one is easier, but uglier). >>>> >>>> I'd rather there be as few shared data structures between network >>>> namespaces as possible--I think that will simplify things. >>>> >>>> So, of those two choices, #1. >>>> >>> >>> Guys, I would like to discuss few ideas about caches and lists containerization. >>> Currently, it look to me, that these hash tables: >>> >>> reclaim_str, conf_id, conf_str, unconf_str, unconf_id, sessionid >>> >>> and these lists: >>> >>> client_lru, close_lru >>> >>> have to be per net, while hash tables >>> >>> file, ownerstr, lockowner_ino >>> >>> and >>> >>> del_recall_lru lists >>> >>> have not, because they are about file system access. >> >> Actually, ownerstr and lockowner_ino should also both be per-container. >> >> So it's only file and del_recall_lru that should be global. (And >> del_recall_lru might work either way, actually.) >> >>> If I'd containerize it this way, then looks like nfs4_lock_state() >>> and nfs4_unlock_state() functions will protect only >>> non-containerized data, while containerized data have to protected >>> by some per-net lock. >> >> Sounds about right. >> > > Bruce, Jeff, I've implemented these per-net hashes and lists (file hash and > del_recall_lru remains global). > But now I'm confused with locking. > > For example, let's consider file hash and del_recall_lru lists. > It looks like they are protected by recall_lock. But in > nfsd_forget_delegations() this lock is not taken. Is it a bug? > If yes and recall_lock is used for file hash protection, then why do we need to > protect nfsd_process_n_delegations() by nfs4_un/lock_state() calls? > > Actually, the problem I'm trying to solve is to replace global client_lock by > per-net one where possible. But I don't clearly understand, what it protects. > > Could you, guys, clarify the state locking to me. > It looks like I can replace global lock with per-net one almost everywhere. But it also looks like there are several places, where additional (per-fs) locking have to be performed (like nfs4proc calls: open, read, setattr, write). Is there any other places, where fs locking have to be used in addition to per-net one? -- Best regards, Stanislav Kinsbursky ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-14 9:00 ` Stanislav Kinsbursky 2012-05-14 10:19 ` Stanislav Kinsbursky @ 2012-05-15 13:40 ` Jeff Layton 2012-05-15 13:55 ` Jeff Layton 1 sibling, 1 reply; 12+ messages in thread From: Jeff Layton @ 2012-05-15 13:40 UTC (permalink / raw) To: Stanislav Kinsbursky; +Cc: bfields, linux-nfs On Mon, 14 May 2012 13:00:17 +0400 Stanislav Kinsbursky <skinsbursky@parallels.com> wrote: > On 12.05.2012 18:16, bfields@fieldses.org wrote: > > On Sat, May 12, 2012 at 12:59:05PM +0400, Stanislav Kinsbursky wrote: > >> On 11.05.2012 17:53, bfields@fieldses.org wrote: > >>> On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: > >>>> Hello. > >>>> I'm currently looking on NFSd laundromat work, and it looks like > >>>> have to be performed per networks namespace context. > >>>> It's easy to make corresponding delayed work per network namespace > >>>> and thus gain per-net data pointer in laundromat function. > >>>> But here a problem appears: network namespace is required to skip > >>>> clients from other network namespaces while iterating over global > >>>> lists (client_lru and friends). > >>>> I see two possible solutions: > >>>> 1) Make these list per network namespace context. In this case > >>>> network namespace will not be required - per-net data will be > >>>> enough. > >>>> 2) Put network namespace link on per-net data (this one is easier, but uglier). > >>> > >>> I'd rather there be as few shared data structures between network > >>> namespaces as possible--I think that will simplify things. > >>> > >>> So, of those two choices, #1. > >>> > >> > >> Guys, I would like to discuss few ideas about caches and lists containerization. > >> Currently, it look to me, that these hash tables: > >> > >> reclaim_str, conf_id, conf_str, unconf_str, unconf_id, sessionid > >> > >> and these lists: > >> > >> client_lru, close_lru > >> > >> have to be per net, while hash tables > >> > >> file, ownerstr, lockowner_ino > >> > >> and > >> > >> del_recall_lru lists > >> > >> have not, because they are about file system access. > > > > Actually, ownerstr and lockowner_ino should also both be per-container. > > > > So it's only file and del_recall_lru that should be global. (And > > del_recall_lru might work either way, actually.) > > > >> If I'd containerize it this way, then looks like nfs4_lock_state() > >> and nfs4_unlock_state() functions will protect only > >> non-containerized data, while containerized data have to protected > >> by some per-net lock. > > > > Sounds about right. > > > > Bruce, Jeff, I've implemented these per-net hashes and lists (file hash and > del_recall_lru remains global). > But now I'm confused with locking. > > For example, let's consider file hash and del_recall_lru lists. > It looks like they are protected by recall_lock. But in > nfsd_forget_delegations() this lock is not taken. Is it a bug? It looks like a bug to me. If another thread is modifying the file_hashtbl while you're calling nfsd_forget_delegations, then you could oops here. Perhaps we only ever modify that table while holding the state mutex in which case the code won't oops, but the recall lock seems rather superfluous at that point. I'd have to unwind the locking and see... > If yes and recall_lock is used for file hash protection, then why do we need to > protect nfsd_process_n_delegations() by nfs4_un/lock_state() calls? > > Actually, the problem I'm trying to solve is to replace global client_lock by > per-net one where possible. But I don't clearly understand, what it protects. > > Could you, guys, clarify the state locking to me. > I wish I could -- I'm still wrapping my brain around it too... -- Jeff Layton <jlayton@redhat.com> ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [RFC] NFSd laundromat containerization 2012-05-15 13:40 ` Jeff Layton @ 2012-05-15 13:55 ` Jeff Layton 0 siblings, 0 replies; 12+ messages in thread From: Jeff Layton @ 2012-05-15 13:55 UTC (permalink / raw) To: Jeff Layton; +Cc: Stanislav Kinsbursky, bfields, linux-nfs On Tue, 15 May 2012 09:40:08 -0400 Jeff Layton <jlayton@redhat.com> wrote: > On Mon, 14 May 2012 13:00:17 +0400 > Stanislav Kinsbursky <skinsbursky@parallels.com> wrote: > > > On 12.05.2012 18:16, bfields@fieldses.org wrote: > > > On Sat, May 12, 2012 at 12:59:05PM +0400, Stanislav Kinsbursky wrote: > > >> On 11.05.2012 17:53, bfields@fieldses.org wrote: > > >>> On Fri, May 11, 2012 at 05:50:44PM +0400, Stanislav Kinsbursky wrote: > > >>>> Hello. > > >>>> I'm currently looking on NFSd laundromat work, and it looks like > > >>>> have to be performed per networks namespace context. > > >>>> It's easy to make corresponding delayed work per network namespace > > >>>> and thus gain per-net data pointer in laundromat function. > > >>>> But here a problem appears: network namespace is required to skip > > >>>> clients from other network namespaces while iterating over global > > >>>> lists (client_lru and friends). > > >>>> I see two possible solutions: > > >>>> 1) Make these list per network namespace context. In this case > > >>>> network namespace will not be required - per-net data will be > > >>>> enough. > > >>>> 2) Put network namespace link on per-net data (this one is easier, but uglier). > > >>> > > >>> I'd rather there be as few shared data structures between network > > >>> namespaces as possible--I think that will simplify things. > > >>> > > >>> So, of those two choices, #1. > > >>> > > >> > > >> Guys, I would like to discuss few ideas about caches and lists containerization. > > >> Currently, it look to me, that these hash tables: > > >> > > >> reclaim_str, conf_id, conf_str, unconf_str, unconf_id, sessionid > > >> > > >> and these lists: > > >> > > >> client_lru, close_lru > > >> > > >> have to be per net, while hash tables > > >> > > >> file, ownerstr, lockowner_ino > > >> > > >> and > > >> > > >> del_recall_lru lists > > >> > > >> have not, because they are about file system access. > > > > > > Actually, ownerstr and lockowner_ino should also both be per-container. > > > > > > So it's only file and del_recall_lru that should be global. (And > > > del_recall_lru might work either way, actually.) > > > > > >> If I'd containerize it this way, then looks like nfs4_lock_state() > > >> and nfs4_unlock_state() functions will protect only > > >> non-containerized data, while containerized data have to protected > > >> by some per-net lock. > > > > > > Sounds about right. > > > > > > > Bruce, Jeff, I've implemented these per-net hashes and lists (file hash and > > del_recall_lru remains global). > > But now I'm confused with locking. > > > > For example, let's consider file hash and del_recall_lru lists. > > It looks like they are protected by recall_lock. But in > > nfsd_forget_delegations() this lock is not taken. Is it a bug? > > It looks like a bug to me. If another thread is modifying the > file_hashtbl while you're calling nfsd_forget_delegations, then you > could oops here. > > Perhaps we only ever modify that table while holding the state mutex in > which case the code won't oops, but the recall lock seems rather > superfluous at that point. > > I'd have to unwind the locking and see... > > > If yes and recall_lock is used for file hash protection, then why do we need to > > protect nfsd_process_n_delegations() by nfs4_un/lock_state() calls? > > > > Actually, the problem I'm trying to solve is to replace global client_lock by > > per-net one where possible. But I don't clearly understand, what it protects. > > > > Could you, guys, clarify the state locking to me. > > > > I wish I could -- I'm still wrapping my brain around it too... > Ok, yeah that is a bug AFAICT. You really need to hold the recall_lock while walking that list, but that makes unhash_delegation tricky -- it can call fput and iput which can block (right?). One possibility is to just have the loop move the entries to a private list. Then you can walk that list w/o holding the lock and do deleg_func on each entry. -- Jeff Layton <jlayton@redhat.com> ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-05-15 13:55 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-05-11 13:50 [RFC] NFSd laundromat containerization Stanislav Kinsbursky 2012-05-11 13:53 ` bfields 2012-05-11 14:02 ` Jeff Layton 2012-05-11 14:15 ` Stanislav Kinsbursky 2012-05-11 15:09 ` bfields 2012-05-12 8:59 ` Stanislav Kinsbursky 2012-05-12 14:16 ` bfields 2012-05-12 14:40 ` Stanislav Kinsbursky 2012-05-14 9:00 ` Stanislav Kinsbursky 2012-05-14 10:19 ` Stanislav Kinsbursky 2012-05-15 13:40 ` Jeff Layton 2012-05-15 13:55 ` Jeff Layton
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.